Documenting the undocumented

[mras0]

Some of the tests cover undefined behavior which is great! Some of the flags are still a mystery to me (in particular AAM in the case of an exception), but I've also narrowed down a few that might be worth documenting:

SHLD/SHRD

Undefined results if ShiftAmt >= OperandSize, but actual behavior is:
The value of the bits flowing into the destination are rotated (ROL/ROR style) from the second operand ("inBits"). CF is set according to last bit going out of the destination.

SHL/SHR r/m8,CL

The original i386 PRM is not super clear, but later manuals say OF/CF is undefined for ShiftAmt >= 8.
Actual behavior when ShiftAmt > 8:
SHL: OF and CF is set to 1 if ((ShiftAmt = 16 OR ShiftAmt = 24) AND (SrcValue & 1) otherwise 0.
SHR: OF set to 0, CF set if ((ShiftAmt = 16 OR ShiftAmt = 24) AND (SrcValue & 80h) otherwise 0.

BT/BTR/BTC/BTS

Manual states most flags except CF are undefined. Actual behavior: Everything except CF/OF is left alone.
Rotate the "source" value right according to "bit index". Set OF equal to XOR of the top two bits of the rotated value.

[barotto]

I also have this, anybody can confirm?

POPAD

With 16bit stack address size:
Pentium+ manuals state ESP is not popped from the stack and SP value is incremented by 4 instead.
i386/i486 manuals state a Pop() for ESP is performed, but the value is discarded.
Actual behaviour: Pop() is performed and the dword value is loaded into ESP, then the LSW is overwritten with the value of SP while the MSW is left untouched.

[mras0]

Actual behaviour: Pop() is performed and the dword value is loaded into ESP, then the LSW is overwritten with the value of SP while the MSW is left untouched.

Yes, it looks like ESP is actually popped and only the LSW "corrected". Unfortunately it doesn't look there's any tests where an exception happens after the pop to see what the MSW of ESP is in that case.

[tremalrik]

One thing I'd personally like to see some tests for on the 386 is the behavior of the LOOPE and LOOPNE instructions (opcodes E1 and E0) when used with the F2h/F3h repeat-prefixes. On both the i486 and the Pentium, using repeat-prefixes with these instructions changes the operation of these instructions in goofy ways:

• on the i486, REP LOOPE acts like LOOP - see https://groups.google.com/g/comp.sys.intel/c/6buCTbj3-PY/m/mI_LH1V6ihIJ which is an old Cyrix compatibility report from 1992 where issue IS-13 covers this behavior.
• on the Pentium, REPE LOOPNE behaves like LOOPE and REPNE LOOPE behaves like LOOPNE - see Intel XED's datafiles such as https://github.com/intelxed/xed/blob/722cd2387273c0e584c100da0f234baf027077f9/datafiles/xed-isa.txt#L8591 . (This behavior apparently went away in Pentium Pro, but AMD seems to have copied it at some point and then never removed it - I've been able to reproduce the Pentium behavior on a Zen 3)

.. which overall makes me curious whether i486 and Pentium inherited these oddities from the i386.