Among the large amount of malwares we view, we have seen a few this week that were heavily obfuscated by some sort of "polymorphic packer." Interestingly, unlike the results of most packers/protectors, the application code wasn't just decrypted and executed from a section, but rather was executed onto the heap. The applications were apparently compiled in Visual C++ and didn't have any position-independent code. (The code needs to be loaded at a given address; otherwise, it won't be able to execute correctly.)
It seems that this obfuscating tool requires the malware to be linked with a low imagebase (lower than the usual 0x400000), in order to be packed. Many reverse engineering tools dedicated to unpacking weren't meant to work on code that is on the heap, and will fail to work correctly in this situation. The protected PE file has a standard PE structure and imagebase.
On the other hand, it has a few hundred KB lines of obfuscated code (around 500 KB in the samples we had). All samples had different junk code, and obfuscation, but it was still pretty easy to find the start of the decrypted application after analysis of a few samples. We have successfully rebuilt completely decrypted samples for further analysis.
Creating a new PE file from scratch with information from the heap is rather easy, and with a little shellcode, it's possible to load all the necessary dlls, to be able to rebuild a new, clean Import Table without problems.
You can find graphs from 3 different samples using the same protector:
Once the code is totally unpacked, you can find the whole code flow of the protector. Here is an example from the 3rd graph above:
Finally, here are two blocks of code from the first sample:
Most of the custom packers we see are much simpler than this. More importantly, they are not "polymorphic" at all.
Even though these particular samples were simple to unpack, the layers were interesting to look at, and the code-on-heap feature (already done before, but in a different way) was a fun surprise on a boring night. 