Given these two minimal examples:
shrd_of:
xor ecx, ecx
mov eax, 0DEADh
mov edx, 0BEEFh
mov r8, 0DEADh
shrd r8, r8, 1
cmovo rax, rdx
ret
shld_of:
xor ecx, ecx
mov eax, 0DEADh
mov edx, 0BEEFh
mov r8, 8000000000000001h
shld r8, r8, 1
cmovo rax, rdx
ret
At MMAT_GENERATED, the SHRD instruction is lowered into register operations without updating any of the affected flags:
mov r8.8, rt0.8 ; u=r8.8 d=rt0.8
mov r8.8, rt0^8.8 ; u=r8.8 d=rt0^8.8
mov #1.1, t2.1 ; u= d=t2.1
or (rt0.8 >>l t2.1), (rt0^8.8 <<l (#0x40.1-t2.1)), rt0.8 ; u=rt0.16,t2.1 d=rt0.8
shr rt0^8.8, t2.1, rt0^8.8 ; u=rt0^8.8,t2.1 d=rt0^8.8
mov rt0.8, r8.8 ; u=rt0.8 d=r8.8
lnot of.1, tt.1 ; u=of.1 d=tt.1
The same happens for SHLD:
or (rt0^8.8 <<l t2.1), (rt0.8 >>l (#0x40.1-t2.1)), rt0^8.8 ; u=rt0.16,t2.1 d=rt0^8.8
shl rt0.8, t2.1, rt0.8 ; u=rt0.8,t2.1 d=rt0.8
mov rt0^8.8, r8.8 ; u=rt0^8.8 d=r8.8
lnot of.1, tt.1 ; u=of.1 d=tt.1
In both cases, of.1 retains the zero assigned by the earlier xor ecx, ecx.
At MMAT_LVARS, both functions reduce to:
mov #0xDEAD.8, result.8 ; u= d=rax.8
And the resulting pseudocode:
return 0xDEAD;
Both instructions should produce OF = 1, since the sign bit changes with a shift count of one. The correct return value in each case is 0xBEEF.
This also affects CF, SF, ZF and PF, which are defined for these shift counts but are likewise missing from the generated microcode. For example, the parity test (cmovp) produces:
if ( __SETP__(1, 0) )
return 0xBEEF;