Calculating EFLAGS for various x86 opcodes

I've found it hard to find a source for these in one place so I've put some code together to calculate these for me. You can find the code I used to generate these results here: github.com/samrussell/TestFlags

Intro

This is all based off en.wikipedia.org/wiki/FLAGS_register for the flag definitions and felixcloutier.com/x86 for nicely parsing the x86 docs. These are both good resources for getting familiar with what's going on under the covers here.

It's important to note that not all instructions modify all flags. For example, the INC instruction does not change CF if it overflows the register, and there are apps that take advantage of this fact (e.g. the LZ91 packer LZEXE)

Base flag values:

Your EFLAGS register is likely to be at 0x202 by default for a number of reasons:

  • The flag 0x0002 is always set (reserved)
  • The flag 0x0008 is always unset (reserved)
  • The flag 0x0020 is always unset (reserved)
  • The flag 0x0100 flag should be unset unless you are single stepping something (trap flag)
  • The flag 0x0200 should be set (interrupts enabled)
  • The flag 0x0400 is probably unset (direction flag for LODSB style commands)

We can ignore the higher flags

ADD

According to the intel docs:

The OF, SF, ZF, AF, CF, and PF flags are set according to the result.

Here's the ADD instruction in action:

ADD 0, 0: 246
ADD 1, 0: 202
ADD 1, 1: 202
ADD 1, ffffffff: 257
ADD 1, f: 212
ADD 10, fffffff0: 247
ADD 1, fffffff0: 282
ADD f, fffffff0: 286
ADD fffffff0, fffffff0: 283
ADD ffffffff, 80000000: a07
ADD fffffffe, 80000000: a03
ADD 1, 7fffffff: a96
ADD 2, 7fffffff: a92

So here's what's going on:

  • 0x0001: CF gets set when we loop around from 0xFFFFFFFF to 0 again
  • 0x0004: PF gets set when an even number of bits are set
  • 0x0010: AF gets set when we loop the bottom 4 bits around (e.g. from 0x0F to 0x10)
  • 0x0040: ZF gets set when the result is zero
  • 0x0080: SF gets set when the high bit is set (i.e. we have a negative number)
  • 0x0800: OF gets set when two positive numbers go negative or vice versa

SUB and CMP

The CMP command is defined as executing a SUB command and setting the flags (but not saving the result), so we should expect them to be the same:

Here's some results:

SUB 0, 0: 246
SUB 1, 0: 202
SUB 1, 1: 246
SUB 0, 1: 297
SUB 10, 1: 216
SUB 0, 10: 287
SUB ffffffff, f: 286
SUB 80000000, 1: a16
SUB 80000002, 1: 282
SUB 7fffffff, fffffff0: a87
SUB 7fffff00, fffffff0: 203

And with CMP

CMP 0, 0: 246
CMP 1, 0: 202
CMP 1, 1: 246
CMP 0, 1: 297
CMP 10, 1: 216
CMP 0, 10: 287
CMP ffffffff, f: 286
CMP 80000000, 1: a16
CMP 80000002, 1: 282
CMP 7fffffff, fffffff0: a87
CMP 7fffff00, fffffff0: 203

Identical. As with the ADD command, we can flip 6 different flags through various different means (including setting the elusive AF by making a carry/borrow between bits 3 and 4)

AND and TEST

The TEST command is defined as executing an AND command and setting the flags (but not saving the result), so we should expect them to be the same. It's worth noting that all of these bitwise commands (AND, OR etc) do the following:

  • OF and CF are cleared
  • SF, ZF, and PF flags are set "according to the result"
  • AF flag is undefined...

It makes sense that OF and CF are cleared as these only make sense when we're doing arithmetic. It is weird that AF is undefined, but also weird that it exists at all...

TEST 0, 0: 246
TEST 1, 0: 246
TEST 1, 1: 202
TEST 0, 1: 246
TEST ffffffff, ffffffff: 286
TEST ffffffff, 0: 246

This is pretty boring. We start with a base of 0x202, we can set 0x40 (ZF) if it's zero, 0x04 (PF) if we have an even number of bits, and 0x80 (SF) if the high bit is set. Zero always gives us 0x246 (ZF and PF are set but not SF), and it's clear that we can't have SF and ZF set at the same time, or that PF has to be set when ZF is set.

Happy hacking everyone!