Commonly - and as I am just starting RE, I face only single-layer encryption algorithms , such as upx , aspack etc ..
Solving these is fully described everywhere . The problem begins when I deal with multi-layer encryption - especially concerning malwares, since I have followed some tuts - usually not detailed enough, they seems to go through a tedious process to get the OEP. For example , they start firstly to deal with a common packers - which is the easy part :p and then they begin to set breakpoints everywhere "in calls and jumps " and tracing through the file here and there, which is for me the hard part that i have described earlier. At this point, I have no clue what they are seeking for or what they are aiming , then after some work , they get the OEP ?!
So what logic did they follow in that process and also because I know that the subject is large , I'm also interested in some keywords ?