#291 Fix trap flag emulation after POPF in dynamic core

[koolkdev]

Until now in the dynamic core, the trap flag was checked and handled only after IRET. But if a program turns on the trap flag using POPFD, the dynamic core ignores it and a trap interrupt won't be executed until the next IRET (which may be in another context that doesn't except a trap interrupt, which will crash the program).

I made the changes to both core_dyn_x86 and core_dynrec.

I did notice that the normal core checks for the trap flag after jmps/calls too. But I am not sure if that is necessary? In which cases the core won't be already in trap mode? Right now I implemented the check only after POPF.

[Qbix]

Hi,

Is there a specific game or application that is affected by this ? (for easy testing)

[koolkdev]

Yes, but that game is Windows 98 game (Hugo: The Evil Mirror), so I know that this is out of the scope of this project. (I submitted a fix to dosbox-x fork too with a bit more info on debugging this bug, but since it is a bug in the cpu emulation, I thought it will be worth fixing here too)

So to help with testing on vanilla dosbox I took the Second Reality - Future Crew demo which uses the trap flag with IRET, and switched one of those IRETs to RETF POPF.
This is the patch:

printf '\x50\xB0\xCF\x2E\xA2\x11\x01\x2E\xA2\x22\x01\xB8\x5B\x5D\x2E\xA3\x0F\x01\x58\x5B\x5D\xCF' | dd of=second.exe bs=1 seek=25914 count=22 conv=notrunc 
printf '\xE4' | dd of=second.exe bs=1 seek=26026 count=1 conv=notrunc 
printf '\x2E\xC6\x06\x0E\x01\x40\xCC\x9D' | dd of=second.exe bs=1 seek=26132 count=8 conv=notrunc 
printf '\xE9\xF8\xFE' | dd of=second.exe bs=1 seek=26175 count=3 conv=notrunc 
printf '\x5D\xCB' | dd of=second.exe bs=1 seek=26193 count=2 conv=notrunc 

Without this fix, this program works only in normal core, not in dynamic core,