Beyond: Two Souls - Reversing | Porting (THAT'S ALL FOLKS?)

[HugoPeters]

Well that would be the best thing ever probably. But how? I mean the renderer is obviously pretty advanced and custom tailored to the SPUs right? How would you make that running on PC? Or do you plan to write an emulator specifically for Beyond

I'm not going to use their renderer, I'm only using the main executable as reference (the disassembled output is very useful).
If I can remake the way the game parses content I can make a new renderer using Mogre.

[HugoPeters]

Update! Been looking into Lua, and been porting / understanding Beyond's kernel code, and I can almost safely say the communicators are just Lua metatables of the inidivudal communicator tables. These tables have their own functions, variables, etc. Those functions look like they're just wrappers of C++ engine/kernel level methods.

Aaaanyhooow, here's some ported shit:



In the region "game manager test" is around the first 400 lines of the actual Game Manager, using that as temp until I build the parser.

Here's the debug output...



Not that much to see, but everything works just as expected and seems to work exactly like the game's (decompiled) source does.

[Hamers]

Well that would be the best thing ever probably. But how? I mean the renderer is obviously pretty advanced and custom tailored to the SPUs right? How would you make that running on PC? Or do you plan to write an emulator specifically for Beyond

I'm not going to use their renderer, I'm only using the main executable as reference (the disassembled output is very useful).
If I can remake the way the game parses content I can make a new renderer using Mogre.

Wow Well, if you can really do this, that probably would be indeed the best thing ever. Good luck

[HugoPeters]

Coming Soon... a brand new tool for viewing Beyond's files. Much more user friendly and intuitive.

I decided to do a complete rewrite as I know so much more now that I can make a much, much better and more stable core, a richer feature set and of course that more intuitive and user friendly layout.

To give a small example of the improved core: the total amount of lines used for reading the IDM files (encrypted to SDAT) used to be 273 in InfraTool.
In this new tool, there are just 50 lines spend on that!
That's a reducement of 82%!

Here's a screenshot from the debugger of a tiny part of the parser that doesn't really say anything at all but does kinda give a little clue of how I want to "order" the files in this new tool.




And yeah, of course I made a logo.

[Mr.Mouse]

Looking good.

[HugoPeters]

PLOT TWIST #1.

=====> http://youtu.be/aVO_eiPiRuY

I have (partially) reversed the sequence / movie timeline format, demo at around 2/5 of the video.

And also: new tool is pretty neat.

[JayK]

Glad to see an update from you, the progress you've made on your tool is really amazing, it looks very well constructed. Keep it up and good luck.

[HugoPeters]

Glad to see an update from you, the progress you've made on your tool is really amazing, it looks very well constructed. Keep it up and good luck.

Thanks. The "movies" in Beyond are basically cutscenes like the Prologue, but are also the things that contain choices, button-press-events, etc. As you can see in the video, I can get the the Prologue's audio to play together with all dialog. It's all separate pieces of dialog, with time indices into the dialog audio. I also found the indices of the subtitles (which is an array of start - end floats) and the localization text gives indices into these floats, so not everything is used.
I can also already parse the camera shots, so I know what camera shot starts when and what properties it has (thanks QD for leaving the property names in there ).
If we can get the textures out, it should be just a matter of time before we can get whole sequences and parts of the game to play.

If anyone knows anything about JPEG / DXT / DDS / compression, please check out: viewtopic.php?f=21&t=11607

[HugoPeters]

-
-
-
Just in before the weekly update, this should clear things up (or not)
-
-
-

edit: you may have noticed the audio is a bit off and the dialog is a bit choppy, this has been corrected now.

[HugoPeters]

Spoilers!!!
See here, what's possible right now: https://www.youtube.com/watch?v=4ikhbv4xkGk
Dialog is still off here and there, will try to fix today. Also the loop points are incorrect so is the volume of some parts.

[HugoPeters]

Here's a nice extended demo video of the parsed timeline: https://www.youtube.com/watch?v=tW9b_UE9Pd4

chrrox and I are working on models as we speak, and I can confirm it's exactly as how I would expect them to do it.

<catalog>
skeleton
_____particles
_____nodes
_____mesh table
__________<link to compressed mesh files>
__________meshdata
_______________shaders
____________________textures
_______________edge data

So catalogs only have one skeleton which links to meshes, etc. All models have 1 skeleton only so it serves as the root entity (thanks debug strings ). Area's have more then one skeleton (for all static objects) so the dev would have to specify the root entity (again, thanks debug strings!)

Soon more? But no textures unless we manage to figure out the texture format/compression. Anyone who knows PPC/elf assembly?

[HugoPeters]

PLOT TWIST #2

Surprise.

[RunaWhite]

OMG so are the models possible?? This is awesome. I hope it'll be possible to get them also with bones and weights... and that it's going to be the same format as Heavy Rain (so I read). I've been waiting for those models for ages.

Thank you very much for all the hard work you guys do on this stuff!

[HeliosAI]

Wow thanks so much for all the work on this This is amazing ;_;

[HugoPeters]

Models Update!

Big thanks to chrrox and howfie for helping me getting this working!!

I ported chrrox's PSA edge index decompression over to C#, and when I finally got my hand on a working bitreader and getting the normals correct (really big thanks to chrrox and howfie for that!) I succeeded in getting the models to show up!

I will be using Mogre as my graphics rendering engine (and it's fucking awesome).



(the arm missing is because there's cloth physics there)





And here's a glimpse into the future...