#698 Problems with MBF21 maps

[Michael Bäuerle]

I have tested some of the Cacoward 2024 winners that use MBF21 (many of them do).

1) Sleepwalking
https://www.doomworld.com/idgames/levels/doom2/Ports/s-u/sleepwalking

2) Eviternity II
https://eviternity.dfdoom.com/

For both WADs I got a Segmentation fault when entering the first map.

Tested with Doom Legacy 1.49 on Linux/ppc and NetBSD/amd64 .

[Wesley Johnson]

I now have three wads that are know to fail in the same way. They are changing the action function in states that are used for player weapon changes. Those are supposed to be acp2 functions with parameters "(player_t * pspdef_t * )". They are changing them to acp1 action functions with parameters "(mobj_t * )". This causes the acp2 function to be called with the wrong parameter stack, and chaos results.

The sleepwaking.wad is throwing over 2000 DEH errors.
Included are keywords that are not supported by DoomLegacy.
"Dropped" is something I saw in the DSDA source code, but it is not part of MBF21 (AFAIK).

DoomLegacy will generate a log file (not sure if have to be compiled with DEBUG=1 to get it though). The log file for sleepwalking.wad has 2194 DEH warnings,
The mapinfo loaded 4 maps with 0 errors.

This is the problem with wads that are developed on only one platform, by trial and error. They do things that are not standard, and it probably will not work on many other platforms.

There are some things that I see that I want to fix.
I also want to protect against this acp1 acp2 distinction being violated, because it crashes DoomLegacy by something the DEH did wrong.
This may take a while. I have several plans, none that I like. The most severe is to replace all the action function ptrs with indexes to a table of action functions. I could then test the index range and discover violations, and ensure that they are only called using the correct parameter stack.

[Wesley Johnson]

What debugging has revealed is at the point of segfault it has a player ptr, that has an mobj_t that has player = NULL. This cannot be.
Trying to trace this back does not make sense, yet.
I will have to guard all the action function calls to discover exactly where one usage is getting into the action states of the other. To do that I have to discover a way to identify an acp2 function from just the ptr.

[Wesley Johnson]

I fixed some dehacked missing stuff, like MBF21 flags in the weapons, and got it down to some 400 errrors.

I finally found the one that causes the segfault. It is because the wad DEH does not clear the states it is using, and just ASSUMES they are already 0 or something.
It is state 1424 that triggers the segfault, It is a Heretic state (S_GOLDWANDREADY), that has a acp2 function (A_WeaponReady) as its action function.

But the sleepwalkiing.wad DEH writes over this state the following:
[code]
Frame 1424
Next frame = 1425
Duration = 1
[/code]

Notice that they do not change the action field, so it is still A_WeaponReady. So first thing that mobj gets to that state, it tries to call A_WeaponReady with acp1(st->action.acp1). This results in chaos of strange calls, because the player ptr that A_WeaponReady gets is really an mobj, and the fields are garbage.

This is really the fault of the wad DEH not filling in all the fields and just ASSUMING they are 0.

I do not know what to do about this just yet.
Note: just clearing any state the DEH writes to will not do, because these DEH do modify the existing states. They do not advertise if they are going to be acp2 states or acp1 states, and many times the action field is NULL and you cannot tell anyway.

Fought with it for 9 hours today, going to think on it.

How long before the Heretic projects want to make use of MBF21, or the like.
Could I get away with detecting if Heretic mode is active, and if not, then clear to 0's and state that was Heretic. Or something like that. I do not trust being able to detect Heretic uses, unless the user is explicitly giving a game=heretic on the command line.
I have always worried about feature creep and the Heretic people wanting to make use of the Doom advanced DEH stuff and so I cannot blindly zero out Heretic state.
Do the Heretic DEH make use of any of the Doom states ?
Do any existing DoomLegacy specific Doom wads (like phobiata) have DEH that rely upon the Heretic states being there ??

[Wesley Johnson]

I have diagnosed several problems with the sleepwalking.wad.
It is not only MBF21.

It assumes that there are a large number of zeroed states. It writes incomplete state records, relying upon those zero values. When it hits the Heretic states in DoomLegacy, that causes the segfaults.
I was able to fix this by zeroing out the Heretic state info. Detecting when to employ this zeroing is still a challenge.

It relies upon DEHEXTRA. This is extra states, and the "dropped item" entry in a Thing.
This is a problem as DEHEXTRA expects you to bloat you port with a large number of states and things, and sounds, that are never used.
I have fixed this well enough to run this wad, but not quite to meet DEHEXTRA unofficial requirements (there are no official docs).

The dehacked lump is broken, as it is missing sprites being assigned for several states.
This is why the gun dissappears when it is fired. Same for shotgun and chaingun.
This I cannot fix, as the information is not there. I can see from the states before or after what the sprite should have been, but that would not be reaonable to be filling sprite fields just because the were left defaulted at 0.

It is using a decohack lump. From reading it, it seems to duplicate much of the weapons logic.
It is probably done right in the decohack, which is why it behaves better in ports that can read the decohack.

I tried the Eviternity wad for a while, but as it does not do anything for most of the wad, as the monsters just stand there and ignore the player. I am not sure if it is working or not.

You got any better choices for MBF21 wads ??

This effort is starting to snowball, and I need to get a replacement release out in a reasonable time due to the broken savegame in the current release.

[Michael Bäuerle]

More or less playable:
https://www.doomworld.com/idgames/levels/doom2/Ports/m-o/ne2
https://www.doomworld.com/idgames/levels/doom2/Ports/megawads/corruption
https://www.doomworld.com/idgames/levels/doom2/Ports/v-z/whatlies
https://www.doomworld.com/idgames/levels/doom2/Ports/d-f/duality
https://www.doomworld.com/idgames/levels/doom2/Ports/d-f/deadskin
https://doomwiki.org/wiki/Necromantic_Thirst

Broken Sky:
https://www.doomworld.com/idgames/levels/doom2/Ports/g-i/gaiaaura

Crash if weapon is fired:
https://doomwiki.org/wiki/AD_MORTEM

Crash while loading map:
https://www.doomworld.com/idgames/levels/doom2/Ports/megawads/simulacrum
https://www.doomworld.com/idgames/levels/doom2/Ports/megawads/pd2

[Wesley Johnson]

I have been using whatlies.wad for testing and it does seem to work, however cannot identify what ever it is using MBF21 upon, and I suppose that is a good thing.
Thank you for the testing.

Have found more problems, and gotten hints from doomworld.

The decohack lump is a leftover from an editor, it is not used by any port.

DSDA sets its zeroed states and things to a default value, that is not zero. And almost every wad developed upon DSDA is likely to depend upon that.

The fields added to states need to be initialized in the code, and the original behaviors need to be installed. The PatchInfoTables function where this is being done, is not being called in the right place, as it is executing after dehacked has modified those tables, (and thus undoing some of the dehacked).
There are also Heretic and Chexquest functions that do a similar thing. However, I am in a bind as they should be after all the launcher decisions, as they depend upon settings the launcher sets, but need to be before wad loading, which is in the launcher loop. Cannot have both. Got to think upon this.

[Michael Bäuerle]

With SVN r1739 on NetBSD/amd64 there are still problems, e.g. with the weapons in Sleepwalking and Ad Mortem.

Sleepwalking:
Trees are bobbing up and down.
Weapon sprites of the fire sequence are not displayed (weapon invisble until back in idle state),
All weapons stop working if one runs out of ammo.
Some of the custom enemies take no damage from the players weapon.

Ad Mortem crashes when the Shotgun is fired:

• At the start of Map1
• Console => "GIMME WEAPONS"
• Switch to Shotgun (not SSG)
• Fire Shotgun

[Wesley Johnson]

Patch to svn 1741 seems to fix most of these.
Trees bobbing (I thought it was SUPPOSED to be doing that) was caused by Heretic FLOATBOB flag being in those states (which had not been cleared).
The weapons were being drawn off screen due to parm1 and parm2 conflict with args, had to separate them back the way they were.
Never had all weapons stop working, and test shows they don't now.
Those immune guys are no longer immune, don't know exactly why.

I don't have Ad Mortem wad yet. The above fixes might have fixed that too.

[Wesley Johnson]

I cannot find Ad Mortem wad, can you give me a link to it.
I found some Ad Mortem deathmatch from 1997.

[Michael Bäuerle]

This one:
https://www.doomworld.com/vb/thread/125461

[Michael Bäuerle]

With Pirate Doom II (on Doom Legacy SVN r1741):
https://www.doomworld.com/idgames/levels/doom2/Ports/megawads/pd2
the crash does not happen if memory management is configured to PLAIN_MALLOC like this:

--- src/z_zone.c.orig<->2023-02-10 15:51:01.000000000 +0000
+++ src/z_zone.c
@@ -100,7 +100,7 @@
 // When the user writes out-of-bounds of malloced region it will do a sigsegv.
 // It does not use the tags, cannot recover PU_CACHE, PU_LEVEL, etc. memory.
 // Uses the most memory of all choices.
-//#define PLAIN_MALLOC
+#define PLAIN_MALLOC
.
 // [WDJ] Combination of malloc and tags.
 // Does malloc from heap, so will grow from heap as use increases.
@@ -112,13 +112,13 @@
 // allocations from it.
 // Has a command line option to set the zone block size ( -mb <int> ).
 // It also has some conditional experimental code.
-#define ZONE_ZALLOC
+//#define ZONE_ZALLOC
.
 // Grows the initial allocation block when it runs out of memory.
 // Runs in the smallest memory of all the choices.
 // Uses tags, and recovers PU_CACHE and PU_LEVEL memory first.
 // Applied as an option to ZONE_ZALLOC.
-#define GROW_ZONE
+//#define GROW_ZONE
.
 // Aggressively purges any PU_CACHE, clearing cache faster.
 // This stresses the memory system more, testing user code to not

Some other things:

• The title picture is broken
• Fist (replaced with a knife) crashes with Segmentation fault when fired. Same behaviour with or without berserk mode.

[Michael Bäuerle]

Tested Sleepwalking.wad again with r1741:

• Trees are looking OK
• All weapons seem to work, except the fist (modified with black gloves), see below.
• The plasma zombies take damage and can be killed.

Still not working:

• The green Cyberdemon cannot be killed. Rockets and plasma balls fly through.
• The IMPs in the sky (and the sky picture itself) are wrong. Woof shows a black sky with white stars (likely this is what the author intends).
• Picking up the berserk pack in the first level makes all weapons unusable.

[Wesley Johnson]

Found that the sleepwalking.wad sets incomplete new states and thus relies upon the DSDA behavior for such, which is different than our traditional.

If you run sleepwalking.wad with the switch -dsda, the fists work, and the there are no imps in the sky. The fists do not require picking up the berserk pack, just select fists.

It fails because the sleepwalking dehacked leaves the PUNCH state with a next field of 0, which is the default.

My problem is that we have many wads that already are dependent upon DoomLegacy dehacked behavior, and do I risk breaking some of them because DSDA defaulted differently and the sleepwalking.wad developers were too lazy to fill in the next field.
That is why that DSDA default behavior is enabled by " -dsda".

If I could figure a way to detect this need, I would make it a detected, and then auto remap could handle it.

Leaving the default at 0 has its own downside, which is how going to state 0 can freeze behavior. But that also made it obvious to the developer that they forgot to fill in the next field.
What do you think. Possibly know of any dehacked that used the 0 default for next ???

That sky is the replacement sky for Doom. Because we have freelook it is taller.
The sky controls are working for TNT, which gives options for modifiying the sky from the wad, but I could not get them to change anything for Doom2.
I will have to look in to that, but it probably has a reason.

[Michael Bäuerle]

If you run sleepwalking.wad with the switch -dsda, the fists work, and the there are no imps in the sky.

I can confirm that with "-dsda":

• Fists work
• No more Imps in the sky
• Snowflakes now fall down to the ground

Leaving the default at 0 has its own downside, which is how going to state 0 can freeze behavior. But that also made it obvious to the developer that they forgot to fill in the next field.

If the engine knows that something is wrong, I think it should be fixed and there should be an error message (instead of freeze, crash or other misbehaviour).

What do you think. Possibly know of any dehacked that used the 0 default for next ???

No, but I have proposed before to make the default behaviour compatible with the source ports that are popular today (for MBF this is Woof, for MBF21 this is DSDA).
Looking at the text files in WAD archives, I have never seen a comment that the WAD was tested with Doom Legacy.

No switch should be required to play current/modern WADs (>99% for my use case).
I always prefer switches for old or Legacy-specific behaviour), if autodetect is not possible.

[Wesley Johnson]

AdMortem is locked behind a "give me your email" Dropbox, and I do not have a throwaway email address to give it. I do not want to risk my primary email address.

[Michael Bäuerle]

AdMortem is locked behind a "give me your email" Dropbox, ...

Maybe a JavaScript problem. I have used wget:

$ wget -O AdMortem_1.5.6.zip "https://www.dropbox.com/scl/fi/3exnqm8nxg4gukhj10hcp/AdMortem_1.5.6.zip?rlkey=r5nwzu32pfh2dtefpbjy2p49b&dl=0"

[Wesley Johnson]

I have played pirate doom (using the latest SVN 1744, "w110_109 deh pirate", through at least 10 levels.
I have not had a memory crash.
The problems were due to sloppy dehacked lump. I could fix some of that, but some of it violates the unofficial dehacked expectations, like having a blank line after a Thing or Frame because everyone uses that to terminate the block.
It seems that the dehacked bugs do not affect it that much.
The sword works now.

The Evilternity wad has a huge dehacked that exceeds the number of states that DoomLegacy has. I tried compiing with 9999 states and it still was not enough to load Evilternity. I watched the remap messages just go on and on for minutes. Deutex could not unpack it.
I used some tricks and found the dehacked lump to be huge but cannot remember the actual number (think dehacked was 1.09 GB or something like that).

[Wesley Johnson]

The problem with the sky is because sleepwalking.wad does not follow the convention for SKY patch names.
It has a texture.txt lump that lists "SKY RSKY1", which clearly has only one patch, named RSKY1.
This is the same as every other wad because RSKY1 is the standard name for the sky patch.
But sleepwalking.wad does not have an RSKY1 patch, so when the search looks for it, it only finds the RSKY1 in legacy.wad (the one used to replace short doom2 skies).
The sky patch in the wad is named skynd03a, and how that weird name gets known to DSDA is probably a new DSDA feature that we do not know about yet.

[Wesley Johnson]

Got fix for AdMoretum, and as usual it is dehacked defaulting. They just could have filled in the args. The MBF21 defaulting had the wrong index used, my fault.
The shotgun has a trooper for the smoke, and no missile comes out. This is fixed by using -dsda again, so it is likely another incomplete Frame in the dehacked.

Will think about how to deal with this, but will not be turning DoomLegacy into a DSDA look-alike. I only spend this many hours on DoomLegacy because it plays the way I want it too. I got 5 other projects to work on too.

[Michael Bäuerle]

Maybe a problem with MBF #271 sky transfer?

I found this test WAD from ukiro (works e.g. with Woof):
https://www.doomworld.com/forum/topic/102222-sky-rendering-inconsistencies-gzdoom-get-your-shit-together/
Again for potential Dropbox problems:

$  wget "https://www.dropbox.com/s/y9wsjoti6scukbr/skyalign.wad"

But it crashes Doom Legacy r1744.

[Wesley Johnson]

I am burned out due to all those MBF21 fixes. Got other projects to work on too. This MBF21 emergency patching burned up 4 months for me.
What I need now is to know what reported bugs got fixed and what remains.

What I remember:

Sleepwalking: sky imps, snow, bobbing trees, weapons, cyberdemon, monsters are fixed.
Secret area is inaccessible, cause unknown.

Evilternity II: Massive, too large for any port except one with dynamic state allocation.
Only for DSDA and its clones.

AdMoretum : fixed state defalting

skyalign.wad: did not get to work on it that I can remember. Cannot even remember if I downloaded it and it just got put somewhere.

I expect you will always be able to find wads written for something else that will crash DoomLegacy. That is an open-ended source of work.

Can we close this bug report, due to its specific bugs being resolved well enough ?

[Michael Bäuerle]

Tested some WADs on NetBSD/amd64:
Sleepwalking and AdMortem work much better now,
Most important the weapons work and the monsters can be killed.
I can confirm that sky imps, snow and bobbing trees are fixed for Sleepwaling.

Problems with the sky and secret areas have low priority for me.
No problem that Evilternity II is not suitable for Doom Legacy.

Looks like that the MBF21 support is at least usable now.
Many thanks for your work on MBF21 support!

This ticket can be closed.

[Wesley Johnson]

Please open a separate bug report for any other wads that give MBF21 problems.
That allows me to track what needs work, and close some as done.
I expect there will be many more MBF21 problems, and even more DSDA compatibility issues.
Thank you for your testing.