PsyDoom 1.1.1 - PSX Doom port (reverse engineered) for PC

@Erick194 if you are reading this and want to collaborate on some of this reverse engineering stuff, let me know! As I understand it you've already done quite a bit of work on the PSX code and your objectives are similar (though you are targeting the original hardware), so perhaps we can help each other out. I would be very interested to know what you have deciphered so far! :)

This is amazing on two levels:

• 1: You attempt a port of the PSX stuff to PC.
• 2: Recently i have been discussing with a few PSXDoom prominents how to name their GZDoom port properly.

So in that case, i am going to tag

@Gerardo194,

@Dark Pulse.

EDIT: Well you tagged Erick already :P

If we're tagging people, @Kaiser and @Quasar should be there too.

15 minutes ago, Gez said:

If we're tagging people, @Kaiser and @Quasar should be there too.

Obviously you know this, but for everyone else:

• Prior work by the OP includes Phoenix Doom, a source port of the 3DO codebase to PC and extended. Whereas most ports use Linux Doom as a base, OP used 3DO Doom and then extended it for the PC. Impressive stuff indeed. 

Holy shit man, this is SO AWESOME <3 .

It looks like solid progress has been made so far, after all there's even modding possibilities in place (more or less proof-of-concept at the moment, but it works). It also seems to be running pretty well.

Which is what I was about to get to: I hope performance at high resolutions will be good, as well as mouse support be at least on par with PD. I seem to recall reading that PD did not run very well at high resolutions and the performance issues were more apparent there. I ran it in 320x200 so it wasn't a problem, but still.

Also, apart from the wrong speed, do demos stay in sync til the end at the moment, or do they desync? What are your objectives by the way? Is it aiming for accuracy + lots of quality-of-life improvements (a la PrBoom, for example), or is it going to be more like a remaster, similar to how PD was, with tons of bugs fixed (which would make demo compatibility impossible)?

Overall, I am SO looking forward to playing this. I noticed music playback is wrong in some parts - the end level music sounded pretty different from how it should have at the end of E1M1.

A lot of the game sim code will be virtually identical to Jag Doom so be sure to check out Calico, or the vanilla Jag source. Should help enormously to verify your results when transforming it to structured programming. There are also some things shared with Doom 64 so that might also be likewise useful.

I take it the name StationDoom is a placeholder? Because Phoenix Doom sounded awesome, but StationDoom, whilst a good reference, does sound a bit generic in my ears.

Something like Eclipse Doom sounds cool.

Also, during searching, i came across a recently released homebrew Doom title for PSX for which i have made a seperate thread: Doom 2D for PSX

1 hour ago, Quasar said:

Calico

Speaking of which, any progress on mouse support :) ?

@intacowetrust I'm interested in this project.

@intacowetrust 

Hello, first of all, good job with rendering and I viewed your code from github and I see that much of it is in MIPS code.
I am really interested, I wanted to compile it but it fail, it must be that I use VS2015.

I currently have the complete code in C, and I am recompiling it for psyq I am still on my way to finish and it is really interesting to fuction the codes.

Everything that has to do with the Williams Entertainment Sound System (WESS) is complete and works 100% in recompiled psx.

Here is an example of compilation for PSXDoom with rotating enemies.

@Redneckerz

By the way for the Gzdoom [GEC] ME, it will change its name which will be Dzdoom (DarkZdoom).

1 hour ago, Erick194 said:

 

@Redneckerz

By the way for the Gzdoom [GEC] ME, it will change its name which will be Dzdoom (DarkZdoom). 

DZDoom is a perfect name and would well with the QZ/GZ/LZ naming scheme that is currently at the ZDoom page.

10 hours ago, seed said:

Which is what I was about to get to: I hope performance at high resolutions will be good, as well as mouse support be at least on par with PD. I seem to recall reading that PD did not run very well at high resolutions and the performance issues were more apparent there. I ran it in 320x200 so it wasn't a problem, but still.

Yeah Phoenix Doom is using a software renderer at the minute, which means performance at higher resolutions can be an issue. I might revisit that decision at some point in the future, and and re-implement with Vulkan or OpenGL.

For this project though I'll probably keep it at the original resolution and do higher resolution support via a fork/offshoot project instead; so something like Chocolate vs Crispy Doom basically... I figure since we don't have the original source at hand there's value in maintaining a codebase that is as 'vanilla' as possible for historical reference and also to serve as a starting point for new modifications. Plus higher resolution support might not look great without rewriting a lot of the way the renderer works due to the 'shakiness' of the PSX renderer and some of the imprecision involved - that stuff would really get exposed at higher resolution.

Quote

Also, apart from the wrong speed, do demos stay in sync til the end at the moment, or do they desync?

Playback speed aside, both demos included with the game play correctly at the moment and the aim is to keep it that way. I wish there were more of them to be honest because they are quite valuable tools for verifying work!

Quote

What are your objectives by the way? Is it aiming for accuracy + lots of quality-of-life improvements (a la PrBoom, for example), or is it going to be more like a remaster, similar to how PD was, with tons of bugs fixed (which would make demo compatibility impossible)?

Primarily to reverse and convert the game code entirely to C++, remove dependencies on the PSX bios and the original .EXE, and where possible to strip away and simplify as many of the layers of emulation as possible. At the minute I use the 'Avocado' emulator to handle emulation of the PSX GPU, SPU, as well as some bios calls.

I'm aiming for this project to be a 'Chocolate Doom' for PSX DOOM basically, which will serve for as a basis for more advanced ports later. I'm considering doing limit removal stuff in this port however (so we can do stuff like PSX SIGIL), but if the changes for that are too big then perhaps we'd be better off doing that in a fork instead. Still TBD on that one.

As far as bugs go, I'm aiming to leave most of them in except for bugs that would cause undefined behavior and/or crashes. Those bugs are not of much value and will only cause annoyance on a modern operating system (which has less tolerance for such things) so I will probably be fixing things like the lost soul out of bounds issue mentioned in this thread: 

Quote

 I noticed music playback is wrong in some parts - the end level music sounded pretty different from how it should have at the end of E1M1.

Yeah there's issues with sound emulation sometimes being advanced too much at the moment. I'm in this tricky spot where I have to advance the emulation sometimes in order for certain hardware events to occur that native C++ code is waiting on (which would be stuck forever otherwise), so sometimes the sound does get out of whack in places. Eventually this issue should be resolved once dependencies on the PSX hardware (via emulation) are removed and the rate of sound advancement is tied to actual real time much more closely. That will be a bit down the road however...

9 hours ago, Quasar said:

A lot of the game sim code will be virtually identical to Jag Doom so be sure to check out Calico, or the vanilla Jag source. Should help enormously to verify your results when transforming it to structured programming. There are also some things shared with Doom 64 so that might also be likewise useful.

Indeed, you are very much correct. From what I've seen so far while studying the disassembly and cross referencing with other DOOM sources, it definitely bears the closest resemblance to Jag Doom. Having that said however, there are places where DOOM II stuff has been pulled in (for obvious reasons) or where PSX DOOM just does its own unique thing.

Excellent point on checking out Doom 64 though! I've been mostly focusing on studying the ancestors of PSX DOOM but had not considered studying it's descendants also. Perhaps Doom64 EX might yield some important clues about certain systems...

9 hours ago, Redneckerz said:

I take it the name StationDoom is a placeholder? Because Phoenix Doom sounded awesome, but StationDoom, whilst a good reference, does sound a bit generic in my ears. Something like Eclipse Doom sounds cool.

Yeah the name is by no means final - if anyone has any better ideas then I'm definitely open to suggestions! That goes for logo ideas too... (Phoenix DOOM needs a logo too). I was basically trying to think of a name that conveyed that it is a port of PlayStation Doom, but without invoking the word 'PlayStation' or any other trademarked stuff that might cause possible issues - hence the 'Station' abbreviation.

Quote

Also, during searching, i came across a recently released homebrew Doom title for PSX for which i have made a seperate thread: Doom 2D for PSX

Interesting find, I was not aware of this! :)

On 12/29/2019 at 11:56 AM, Erick194 said:

@intacowetrust 

Hello, first of all, good job with rendering and I viewed your code from github and I see that much of it is in MIPS code.

Thanks @Erick194! You are quite right, there is much work to be done on this project to produce code that is more than just assembly-like C++. What you see so far is mostly derived from the original machine code... So far I have only converted over only a very small portion of the game and mostly focused on setup/loading related code. That being said however the entire program structure (in terms of functions) is already there as well as some helpful auto-generated comments documenting where loads and stores are going to. The entire conversion process will take a lot of time and effort but I'm confident that it can be done eventually given enough persistence.

The hardest part was getting to this point and TBH I wasn't entirely sure if the strange mix of running most of the code natively and jumping into the emulator whenever something hardware specific was required would work. Somehow it all does however, occasional sound sync issues aside... I was partially inspired by this project to take that approach:

https://github.com/ughman/c2c

The nice thing about this setup as well is that it is very easy to do a small bit of reversing, test how it works with all of the existing code and then move on. If need be I can also run the original function through the emulator if there is a problem, to compare behavior. This can be quite a useful tool indeed!

Quote

I am really interested, I wanted to compile it but it fail, it must be that I use VS2015.

Yeah sorry, there might be occasional use of C++ 17 features in there that would require at least VS2017. Not that I intend to do object oriented DOOM or anything but some of the quality of life things in C++ 17 can be useful. I have also tested against Xcode 11 if you have that either... XC 10 might work too.

Quote

I currently have the complete code in C, and I am recompiling it for psyq I am still on my way to finish and it is really interesting to fuction the codes.

Yeah it's strangely alluring, gradually uncovering some of the mysteries inside of this game... For example I was puzzled for a long time why there two 'malloc' functions until I reversed the extra one and discovered that it was trying to allocate at the end of the heap rather than the start. I don't recall seeing this in other versions of DOOM. See the function I'm calling 'Z_EndMalloc' for that particular code:
https://github.com/BodbDearg/PsyDoom/blob/master/game/Doom/Base/z_zone.cpp

 

Quote

Everything that has to do with the Williams Entertainment Sound System (WESS) is complete and works 100% in recompiled psx.

 

Here is an example of compilation for PSXDoom with rotating enemies.

That's awesome man, nice work! :) Definitely cool to see it running in the original environment too. If you'd like to contribute to this project as well, I'd definitely be interested in seeing some of that deciphered WESS code and incorporating your findings. That might free me up to look at other stuff which hopefully might be of use to your own project.

1 hour ago, intacowetrust said:

Yeah sorry, there might be occasional use of C++ 17 features in there that would require at least VS2017. Not that I intend to do object oriented DOOM or anything but some of the quality of life things in C++ 17 can be useful. I have also tested against Xcode 11 if you have that either... XC 10 might work too.

I will have to try that Xcode 11. From what I see I can't, it's for mac :P

1 hour ago, intacowetrust said:

Yeah it's strangely alluring, gradually uncovering some of the mysteries inside of this game... For example I was puzzled for a long time why there two 'malloc' functions until I reversed the extra one and discovered that it was trying to allocate at the end of the heap rather than the start. I don't recall seeing this in other versions of DOOM. See the function I'm calling 'Z_EndMalloc' for that particular code:

It is correct both PsxDoom and Doom64 have that function, but it is called Z_Alloc, here the decompilation of z_zone of both games.

PSXDOOM z_zone.c

DOOM64 z_zone.c

8 hours ago, intacowetrust said:

Yeah Phoenix Doom is using a software renderer at the minute, which means performance at higher resolutions can be an issue. I might revisit that decision at some point in the future, and and re-implement with Vulkan or OpenGL.

NOICE, it would be an absolute blast to have such renderers in place, although I seem to recall the reason why PD runs worse at higher resolutions was due to some fancy feature that was added I think that was added, which wasn't an issue at lower res but became one once it was cranked up.

So there's not going to be much else apart from QOL improvements, that's fine. I agree that initially it would be best to have the whole game reverse-engineered, the code cleaned up, and in C++ since there original code was never released. This will probably make it easier for forking as there won't be any worries about removing unwanted features, and also perfect for historicity and preservation, making sure it never gets lost again.

8 hours ago, intacowetrust said:

Playback speed aside, both demos included with the game play correctly at the moment and the aim is to keep it that way. I wish there were more of them to be honest because they are quite valuable tools for verifying work!

Yea, shame there's only 2-3 internal demos, but hey, that's better than nothing, at least those can be used for reference in the beginning, and later on, when the game stabilizes we could start making demos and playing them back on the real hardware to see if they desync. That is, unless I'm dreaming - is it even possible to do this?

8 hours ago, intacowetrust said:

As far as bugs go, I'm aiming to leave most of them in except for bugs that would cause undefined behavior and/or crashes. Those bugs are not of much value and will only cause annoyance on a modern operating system (which has less tolerance for such things) so I will probably be fixing things like the lost soul out of bounds issue mentioned in this thread

Falls under QOL improvements, which is a good thing, no one wants crashes and bugs :) .

7 hours ago, intacowetrust said:

That goes for logo ideas too... (Phoenix DOOM needs a logo too).

That would be neat, pretty ugly to have default exe icons on the desktop (well at least they're not as ugly as they are in older versions of Windows :p ).

9 hours ago, intacowetrust said:

Primarily to reverse and convert the game code entirely to C++, remove dependencies on the PSX bios and the original .EXE, and where possible to strip away and simplify as many of the layers of emulation as possible. At the minute I use the 'Avocado' emulator to handle emulation of the PSX GPU, SPU, as well as some bios calls. 

Ideally, everything could be turned into portable code with no need for integrating an emulator anymore.

8 hours ago, intacowetrust said:

Yeah the name is by no means final - if anyone has any better ideas then I'm definitely open to suggestions! That goes for logo ideas too... (Phoenix DOOM needs a logo too). I was basically trying to think of a name that conveyed that it is a port of PlayStation Doom, but without invoking the word 'PlayStation' or any other trademarked stuff that might cause possible issues - hence the 'Station' abbreviation. 

I think "Station Doom" is perfectly cromulent. I could suggest something like Psych Doom (PSX -> Psych). Or "Loco Doom" since this port allows the game to leave the station. Or, same idea, "Dynamite Doom", since station is static, and the opposite of static is dynamic.

On 12/29/2019 at 11:44 AM, Redneckerz said:

 

Also, during searching, i came across a recently released homebrew Doom title for PSX for which i have made a seperate thread: Doom 2D for PSX

15 hours ago, intacowetrust said:

Interesting find, I was not aware of this! :)

Me neither, I think it looks interesting!!

18 hours ago, intacowetrust said:

 

Yeah the name is by no means final - if anyone has any better ideas then I'm definitely open to suggestions! That goes for logo ideas too... (Phoenix DOOM needs a logo too). I was basically trying to think of a name that conveyed that it is a port of PlayStation Doom, but without invoking the word 'PlayStation' or any other trademarked stuff that might cause possible issues - hence the 'Station' abbreviation.

 

 

Interesting find, I was not aware of this! :)

What do you think of the names put forward so far?

2 hours ago, Gerardo194 said:

 

Me neither, I think it looks interesting!!

Its not ultra polished, but hey, its a Doom platformer on PSX.

16 hours ago, Erick194 said:

It is correct both PsxDoom and Doom64 have that function, but it is called Z_Alloc, here the decompilation of z_zone of both games.

 

PSXDOOM z_zone.c

DOOM64 z_zone.c

Interesting, thanks for sharing!

12 hours ago, seed said:

 

Yea, shame there's only 2-3 internal demos, but hey, that's better than nothing, at least those can be used for reference in the beginning, and later on, when the game stabilizes we could start making demos and playing them back on the real hardware to see if they desync. That is, unless I'm dreaming - is it even possible to do this?

That's actually a really neat idea - I might end up doing that! (though perhaps in reverse, playing back real demos in the new build and verifying that way)

I've definitely seen code in there relating to demo recording and storing inputs in a 16 KiB demo buffer so it seems doable. It might be difficult to record on the real hardware, but perhaps it's possible using a PC, an old serial connection and a cheat cartridge; I have not had such a setup since the 90s though... Basically I would need to patch the game code or data somehow to activate demo recording, then grab the data from that buffer and save to a file. With a modified emulator that might be a lot easier since I could just set breakpoints and patch where required, and save the required memory out to a file on the host machine.

10 hours ago, Gez said:

I think "Station Doom" is perfectly cromulent. I could suggest something like Psych Doom (PSX -> Psych). Or "Loco Doom" since this port allows the game to leave the station. Or, same idea, "Dynamite Doom", since station is static, and the opposite of static is dynamic.

I think you might be onto something with 'Psy'... It could be named that in reference to PlayStation 1 SDK (PsyQ) that the game is built with and in honor of the legendary studio that worked on it, Psygnosis. It also has a nice ring to I think... Hmm, 'PsyDoom' - what do you guys think?

1 hour ago, Redneckerz said:

What do you think of the names put forward so far?

I think we're making some progress on this!

@intacowetrust

You know, there is little left to finish recompiling PsxDoom in Native Psx, once it is finished and it works correctly, your project will advance exponentially, I only ask for a little time.

As you can see in the picture all that is already converted into C/C++

This is really impressive. This was something that I've always wanted to see happen but never had the resources/knowledge to pull it off at the time. Hopefully, the code can be cleaned up and documented as the project advances, especially with things like the existence of Z_Alloc and why it needs to allocate at the end of the heap. Definitely looking forward to seeing further development on this.

4 hours ago, Erick194 said:

@intacowetrust

 

You know, there is little left to finish recompiling PsxDoom in Native Psx, once it is finished and it works correctly, your project will advance exponentially, I only ask for a little time.

 

As you can see in the picture all that is already converted into C/C++

Wow that's awesome @Erick194 - amazing work! I did not know you were already this far along! Having access to this code would be enormously helpful towards getting the PC client off the ground and would allow me to focus more on removing the dependency on the hardware. Do you have an idea of how long it will take to finish off the rest?

In the meantime as well I am more than happy to help out if you want. If there's a particular function or piece of code you want me to investigate just let me know. Or if you want to verify some existing work within this project I can do that also, just let me know.

43 minutes ago, Kaiser said:

This is really impressive. This was something that I've always wanted to see happen but never had the resources/knowledge to pull it off at the time. Hopefully, the code can be cleaned up and documented as the project advances, especially with things like the existence of Z_Alloc and why it needs to allocate at the end of the heap.

Indeed, I'm looking forward to aspect of this as well. I fully intend for it to be as clean and as well documented as possible, and also to serve as a nice basis for future modifications.

As far as the 'Z_Alloc' stuff goes my initial hunch would be that it is perhaps a strategy for reducing heap fragmentation, maybe through placing larger or more static allocations near the end of the heap? I have not yet studied all the places where it is used though, so that's just a guess right now.

Another function that goes hand in hand with the z_zone is the P_LoadBlocks, this reads and inserts the memory blocks of the textures and sprites back to the memory of the z_zone.

enum compressFlags {Decode,NoDecode};
typedef struct psxblock_s
{
	int		size;               //*     including the header and possibly tiny fragments */
	void    **user;             //*4    NULL if a free block */
	short   tag;                //*8    purgelevel */
	short   id;                 //*10   should be ZONEID */
	short   lump;               //*12
	short   flags;              //*14   0 = Decode || 1 = NoDecode || >2 = Error
	struct psxblock_s   *next;  //*16
	struct psxblock_s	*prev;  //*20
} psxblock_t;

void P_LoadBlocks(char *filename)//L80023698()
{
	psxblock_t	header, *base, *block, *next, *prev;
	int i, file_num, data_size, size;
	boolean error;
	byte *ptr;

	i = 0;
	while (true)
    {
        if (i >= 4)
			I_Error("P_LoadBlocks: Data Failure");
        i++;

        error = false;

        file_num = OpenFile(filename);
        data_size = SeekFile(file_num, 0, PSXCD_SEEK_END);

        ptr = (byte *)Z_Malloc(data_size - sizeof(psxblock_t), PU_STATIC, 0);

        base = (psxblock_t *)((byte *)ptr - sizeof(psxblock_t));
        header.size = base->size;
        header.user = base->user;
        header.tag = base->tag;
        header.id = base->id;
        header.lump = base->lump;
        header.flags = base->flags;
        header.next = base->next;
        header.prev = base->prev;

        prev = base->prev;
        next = base->next;

        SeekFile(file_num, 0, PSXCD_SEEK_SET);
		ReadFile(file_num, (byte *)base, data_size);
		CloseFile(file_num);
        block = base;
        size = data_size;

        do
        {
            if ((((block->id != ZONEID) || (block->lump >= numlumps)) || (block->flags > NoDecode)) ||
                (block->flags == Decode &&
                (decodedsize((byte *)block + sizeof(psxblock_t)) != lumpinfo[block->lump].size)))
			{
            error_:
				error = true;
				break;
			}

            size -= block->size;
            if (size < 0) goto error_;

            (byte *)block = (byte *)block + block->size;
        } while (size != 0);

        if (!error)
        {
            base->prev = prev;
            do
            {
                if (lumpcache[base->lump].cache == NULL)
                {
                    *(void **)&base->user = (void *)((byte *)lumpcache + base->lump);
                    *(void **)&lumpcache[base->lump].cache = (void *)((byte *)base + sizeof(memblock_t));
                    lumpencode[base->lump] = base->flags;
                }
                else
                {
                    base->user = NULL;
                    base->tag = 0;
                    base->id = 0;
                }

                data_size -= base->size;
                if (data_size == 0)
                {
                    if (next)
                        *(psxblock_t **)&base->size = (psxblock_t *)((int)next - (int)base);

                    base->next = next;
                }
                else
                {
                    base->next = (psxblock_t *)((int)&base->size + base->size);
                }

                if (base->next)
                    base->next->prev = base;

                base = base->next;

            } while (data_size != 0);
            Z_CheckHeap(mainzone);
            return;
        }

        base->size = header.size;
        base->user = header.user;
        base->tag = header.tag;
        base->id = header.id;
        base->lump = header.lump;
        base->flags = header.flags;
        base->next = header.next;
        base->prev = header.prev;
        Z_Free((byte *)ptr);
    }
}

52 minutes ago, intacowetrust said:

Do you have an idea of how long it will take to finish off the rest?

If everything goes well, it can last between 1 month or less, because I only need (p_pspr.c | p_setup.c | st_main.c) and review other files in the hope of not forgetting anything, which I really have to test It is the distribution of the leafs and their rendering.

On 12/30/2019 at 9:50 PM, Erick194 said:

Another function that goes hand in hand with the z_zone is the P_LoadBlocks, this reads and inserts the memory blocks of the textures and sprites back to the memory of the z_zone.

Yeah that was a fun one to decipher. Was drawn to it mainly because I was wondering 'what the hell is that'? Here was my own interpretation of it if you're curious: https://github.com/BodbDearg/PsyDoom/blob/a8eafbff51b9cdc4a9d74d268d8b8bc28bd7c695/game/Doom/Game/p_setup.cpp#L1676

On 12/30/2019 at 10:09 PM, Erick194 said:

If everything goes well, it can last between 1 month or less, because I only need (p_pspr.c | p_setup.c | st_main.c) and review other files in the hope of not forgetting anything, which I really have to test It is the distribution of the leafs and their rendering.

Nice - that's pretty close then!

13 hours ago, intacowetrust said:

 

I think you might be onto something with 'Psy'... It could be named that in reference to PlayStation 1 SDK (PsyQ) that the game is built with and in honor of the legendary studio that worked on it, Psygnosis. It also has a nice ring to I think... Hmm, 'PsyDoom' - what do you guys think?

 

 

I think we're making some progress on this!

The very similarly named PsiDoom already exists and is an source port for Amiga. It just has no Wiki entry yet.

PsyQDoom could be something? It differentiates enough from PsiDoom and QZDoom in my eyes and its both a reference to PS1 SDK and Psygnosis (Stupid of me that i didn't think of this first up when looking up codenames...)

6 hours ago, Erick194 said:

 

If everything goes well, it can last between 1 month or less, because I only need (p_pspr.c | p_setup.c | st_main.c) and review other files in the hope of not forgetting anything, which I really have to test It is the distribution of the leafs and their rendering.

So, as it stands then, we would have two PSX source port projects:

• DZDoom (DarkZDoom) - A GZDoom based build with all kinds of features from the PSX version added in to work with all the features that GZDoom brings.
• PsyQDoom (I am just holding this on for now as placeholder) - A port of the PSX Doom code to PC, without additional enhancements like Phoenix Doom does, and which strives for a vanilla port of the PSX framework. (I assume that for more modern stuff like high res/OpenGL support, you could fork this stuff).

Sounds about right, no?