Page 1 of 3
Heavy Rain Tool
Posted: Sun Sep 03, 2017 2:42 pm
by MushroomLT
The beginnings of the ultimate Heavy Rain extractor tool, or even PC port in the future - Heavy Rain Tool
After finding @HugoPeters work on Beyond, I became really interested in the concept of reverse-engineering a game engine, so I started working on my favourite game - Heavy Rain.
Currently I managed to:
- Read the BigFile index (the sdat encrypted file)
- Find and display a list of entries grouped by type
- Unpack all of the segs archived entries
- Read the inner structure of the entries
- Parse some of the blocks of the entries to extract data from them:
- Textures (some of them are misaligned)
- Scripts that are not compiled
- BIK Video files
- Some of the audio files + Multi-Channel audio exporting (Mostly soundtrack)
What I'm working on:
- Linking the entries together by parsing the storyboard
- Properly aligning textures
(Last updated: 2017-09-12, PreAlpha 3)
Tool usage:
* Run the tool to generate "engine.json" configuration file.
* Open the configuration file and set "qdGameDirectory" to the game file directory location
* Set vlcExeLocation to the location of your VLC Media Player installation to be able to play sounds directly from the tool
* Set loadBigFile to the BigFile index file you want to load.
The tool can extract any entry into a separate binary file by selecting an entry and clicking the Export button.
To view the inner blocks of the entry double click it in the tree. Selecting a block will automatically adjust the offset in the hex view.
To export video from an entry double click the entry, and double click the VIDEDATA block.
To export sounds from an entry double click the entry, and double click the PARTITIO block.
To play the exported sound using VLC player, double click the entry inside of the PARTITIO block.
To view scripts, double click the SCRIPT block, and select the inner script entry, it will be displayed under the Contents tab.
Some progress
Posted: Fri Sep 08, 2017 5:17 pm
by MushroomLT
I managed to perform a RAM dump of RPCS3 running the game by modifying it's source, and from the dump I found the LUA initialisation code for probably all of the data types. Now the tool is capable of displaying what data each entry has:
Code: Select all
CommunicatorBlock Contents:
DCINFO Entries:
Entry 0: SOUND_OFFSET; 64112
Entry 1: SOUND_DATA; 64108
Entry 2: SCRIPT_SOUND_ENTITY; 15116
Entry 3: SOUND_OFFSET; 64116
Entry 4: SOUND_DATA; 64112
Entry 5: SCRIPT_SOUND_ENTITY; 15118
Entry 6: SOUND_OFFSET; 66806
Entry 7: SOUND_DATA; 66803
Entry 8: SCRIPT_SOUND_ENTITY; 15117
Entry 9: SOUND_OFFSET; 46980
Entry 10: SOUND_DATA; 46976
Entry 11: SCRIPT_SOUND_ENTITY; 10944
Entry 12: SOUND_OFFSET; 64114
Entry 13: SOUND_DATA; 64110
Entry 14: SCRIPT_SOUND_ENTITY; 15120
Entry 15: SOUND_OFFSET; 62123
Entry 16: SOUND_DATA; 62119
Entry 17: SCRIPT_SOUND_ENTITY; 10947
Entry 18: SCRIPT_SOUND_ENTITY; 10945
Entry 19: SCRIPT_SOUND_ENTITY; 15119
Entry 20: LOADING_ZONE; 5681
Entry 21: LOADING_ZONE; 5756
Entry 22: LOADING_ZONE; 6602
Entry 23: LOADING_ZONE; 6630
Entry 24: LOADING_ZONE; 6676
Entry 25: LOADING_ZONE; 6718
Entry 26: LOADING_ZONE; 6767
Entry 27: LOADING_ZONE; 6779
Entry 28: LOADING_ZONE; 22721
Entry 29: LOADING_ZONE; 22724
Entry 30: LOADING_ZONE; 22727
Entry 31: LOADING_ZONE; 22729
Entry 32: LOADING_ZONE; 22754
Entry 33: LOADING_ZONE; 38606
Entry 34: STORYBOARD; 46
Entry 35: LOADING_ZONE; 5674
Entry 36: LOADING_ZONE; 5675
Entry 37: LOADING_ZONE; 5677
Entry 38: LOADING_ZONE; 5678
Entry 39: LOADING_ZONE; 5679
Entry 40: LOADING_ZONE; 5689
Entry 41: LOADING_ZONE; 5690
Entry 42: LOADING_ZONE; 5691
Entry 43: LOADING_ZONE; 5704
Entry 44: LOADING_ZONE; 5739
Entry 45: LOADING_ZONE; 5740
Entry 46: LOADING_ZONE; 5741
Entry 47: LOADING_ZONE; 5754
Entry 48: LOADING_ZONE; 6498
Entry 49: LOADING_ZONE; 6569
Entry 50: LOADING_ZONE; 6659
Entry 51: LOADING_ZONE; 6662
Entry 52: LOADING_ZONE; 6669
Entry 53: LOADING_ZONE; 6670
Entry 54: LOADING_ZONE; 6679
Entry 55: LOADING_ZONE; 6686
Entry 56: LOADING_ZONE; 6689
Entry 57: LOADING_ZONE; 6696
Entry 58: LOADING_ZONE; 6697
Entry 59: LOADING_ZONE; 6698
Entry 60: LOADING_ZONE; 6701
Entry 61: LOADING_ZONE; 6703
Entry 62: LOADING_ZONE; 6705
Entry 63: LOADING_ZONE; 6706
Entry 64: LOADING_ZONE; 6707
Entry 65: LOADING_ZONE; 6709
Entry 66: LOADING_ZONE; 6711
Entry 67: LOADING_ZONE; 6714
Entry 68: LOADING_ZONE; 6716
Entry 69: LOADING_ZONE; 6719
Entry 70: LOADING_ZONE; 6720
Entry 71: LOADING_ZONE; 6723
Entry 72: LOADING_ZONE; 6724
Entry 73: LOADING_ZONE; 6725
Entry 74: LOADING_ZONE; 6728
Entry 75: LOADING_ZONE; 6730
Entry 76: LOADING_ZONE; 6732
Entry 77: LOADING_ZONE; 6733
Entry 78: LOADING_ZONE; 6734
Entry 79: LOADING_ZONE; 6738
Entry 80: LOADING_ZONE; 6740
Entry 81: LOADING_ZONE; 6742
Entry 82: LOADING_ZONE; 6745
Entry 83: LOADING_ZONE; 6748
Entry 84: LOADING_ZONE; 6753
Entry 85: LOADING_ZONE; 6755
Entry 86: LOADING_ZONE; 6758
Entry 87: LOADING_ZONE; 6760
Entry 88: LOADING_ZONE; 6771
Entry 89: LOADING_ZONE; 6773
Entry 90: LOADING_ZONE; 6774
Entry 91: LOADING_ZONE; 6776
Entry 92: LOADING_ZONE; 6777
Entry 93: LOADING_ZONE; 6785
Entry 94: LOADING_ZONE; 6788
Entry 95: LOADING_ZONE; 6789
Entry 96: LOADING_ZONE; 6792
Entry 97: LOADING_ZONE; 6795
Entry 98: LOADING_ZONE; 6797
Entry 99: LOADING_ZONE; 6798
Entry 100: LOADING_ZONE; 6799
Entry 101: LOADING_ZONE; 6800
Entry 102: LOADING_ZONE; 6801
Entry 103: LOADING_ZONE; 6805
Entry 104: LOADING_ZONE; 6807
Entry 105: LOADING_ZONE; 6808
Entry 106: LOADING_ZONE; 6810
Entry 107: LOADING_ZONE; 6811
Entry 108: LOADING_ZONE; 6814
Entry 109: LOADING_ZONE; 6816
Entry 110: LOADING_ZONE; 6817
Entry 111: LOADING_ZONE; 6818
Entry 112: LOADING_ZONE; 6825
Entry 113: LOADING_ZONE; 6828
Entry 114: LOADING_ZONE; 6830
Entry 115: LOADING_ZONE; 6832
Entry 116: LOADING_ZONE; 6834
Entry 117: LOADING_ZONE; 6835
Entry 118: LOADING_ZONE; 6837
Entry 119: LOADING_ZONE; 7700
Entry 120: LOADING_ZONE; 10309
Entry 121: LOADING_ZONE; 10419
Entry 122: LOADING_ZONE; 17933
Entry 123: LOADING_ZONE; 18595
Entry 124: LOADING_ZONE; 20832
Entry 125: LOADING_ZONE; 20834
Entry 126: LOADING_ZONE; 20836
Entry 127: LOADING_ZONE; 20850
Entry 128: LOADING_ZONE; 20852
Entry 129: LOADING_ZONE; 22359
Entry 130: LOADING_ZONE; 23752
Entry 131: LOADING_ZONE; 23756
Entry 132: LOADING_ZONE; 23761
Entry 133: LOADING_ZONE; 23763
Entry 134: LOADING_ZONE; 23969
Entry 135: LOADING_ZONE; 25561
Entry 136: LOADING_ZONE; 28431
Entry 137: LOADING_ZONE; 29901
Entry 138: LOADING_ZONE; 29907
Entry 139: LOADING_ZONE; 29909
Entry 140: LOADING_ZONE; 29911
Entry 141: LOADING_ZONE; 29913
Entry 142: LOADING_ZONE; 29915
Entry 143: LOADING_ZONE; 29917
Entry 144: LOADING_ZONE; 29920
Entry 145: LOADING_ZONE; 29924
Entry 146: LOADING_ZONE; 29928
Entry 147: LOADING_ZONE; 29932
Entry 148: LOADING_ZONE; 29937
Entry 149: LOADING_ZONE; 29953
Entry 150: LOADING_ZONE; 29956
Entry 151: LOADING_ZONE; 33069
Entry 152: LOADING_ZONE; 33215
Entry 153: LOADING_ZONE; 33239
Entry 154: LOADING_ZONE; 33241
Entry 155: LOADING_ZONE; 34655
Entry 156: LOADING_ZONE; 35557
Entry 157: LOADING_ZONE; 35863
Entry 158: LOADING_ZONE; 38507
Entry 159: LOADING_ZONE; 39037
Entry 160: LOADING_ZONE; 39041
Entry 161: LOADING_ZONE; 39269
Entry 162: LOADING_ZONE; 39448
Entry 163: LOADING_ZONE; 39519
Entry 164: LOADING_ZONE; 39521
Entry 165: LOADING_ZONE; 39523
Entry 166: LOADING_ZONE; 40373
Entry 167: STORYBOARD; 35
Entry 168: ENGINE_TEXTURE_FILE; 49948
Entry 169: ENGINE_TEXTURE_FILE; 50553
Entry 170: ENGINE_TEXTURE_FILE; 54206
Entry 171: ENGINE_TEXTURE_FILE; 54207
Entry 172: ENGINE_TEXTURE_FILE; 54208
Entry 173: ENGINE_TEXTURE_FILE; 54209
Entry 174: ENGINE_TEXTURE_FILE; 54210
Entry 175: ENGINE_TEXTURE_FILE; 54211
Entry 176: ENGINE_TEXTURE_FILE; 54212
Entry 177: ENGINE_TEXTURE_FILE; 54213
Entry 178: ENGINE_TEXTURE_FILE; 54214
Entry 179: ENGINE_TEXTURE_FILE; 62181
Entry 180: ENGINE_TEXTURE_FILE; 62182
Entry 181: ENGINE_TEXTURE_FILE; 62183
Entry 182: ENGINE_TEXTURE_FILE; 62184
Entry 183: ENGINE_TEXTURE_FILE; 62185
Entry 184: ENGINE_TEXTURE_FILE; 62186
Entry 185: ENGINE_TEXTURE_FILE; 62187
Entry 186: ENGINE_TEXTURE_FILE; 62188
Entry 187: ENGINE_TEXTURE_FILE; 62921
Entry 188: ENGINE_TEXTURE_FILE; 64307
Entry 189: ENGINE_TEXTURE_FILE; 64308
Entry 190: ENGINE_TEXTURE_FILE; 64309
Entry 191: ENGINE_TEXTURE_FILE; 69866
Entry 192: ENGINE_TEXTURE_FILE; 75541
Entry 193: ENGINE_TEXTURE_FILE; 75542
Entry 194: ENGINE_TEXTURE_FILE; 75771
Entry 195: ENGINE_TEXTURE_FILE; 76140
Entry 196: ENGINE_TEXTURE_FILE; 78367
Entry 197: ENGINE_TEXTURE_FILE; 78368
Entry 198: ENGINE_TEXTURE_FILE; 78369
Entry 199: ENGINE_TEXTURE_FILE; 78370
Entry 200: ENGINE_TEXTURE_FILE; 78371
Entry 201: ENGINE_TEXTURE_FILE; 78372
Entry 202: ENGINE_TEXTURE_FILE; 78373
Entry 203: ENGINE_TEXTURE_FILE; 78374
Entry 204: ENGINE_TEXTURE_FILE; 78375
Entry 205: ENGINE_TEXTURE_FILE; 78376
Entry 206: ENGINE_TEXTURE_FILE; 78377
Entry 207: ENGINE_TEXTURE_FILE; 78378
Entry 208: ENGINE_TEXTURE_FILE; 79345
Entry 209: ENGINE_TEXTURE_FILE; 62189
Entry 210: ENGINE_TEXTURE_FILE; 69848
Entry 211: MENUS_MASTER_FONT_DATA; 9
Entry 212: MENUS_MASTER_DATA; 28
Entry 213: CHARACTER; 3
Entry 214: CHARACTER; 345
Entry 215: CHARACTER; 7
Entry 216: CHARACTER; 6
DCDATA Size: 5336713; Entries: 7
Entry 0: 0
Entry 1: 48
Entry 2: 0
Entry 3: 0
Entry 4: 256
Entry 5: 0
Entry 6: 1
Textures reversed
Posted: Fri Sep 08, 2017 9:28 pm
by MushroomLT
Textures reversed!
Re: Heavy Rain Tool
Posted: Sat Sep 09, 2017 10:18 pm
by WRS
Awesome keep it up!
Re: Heavy Rain Tool
Posted: Tue Sep 12, 2017 1:54 pm
by MushroomLT
DataContainer structure reversed - working on the storyboard now, which links all the scenes.
Re: Heavy Rain Tool
Posted: Sat Sep 16, 2017 11:25 pm
by HugoPeters
Great to see my stuff inspired a follow-up!
Re: Heavy Rain Tool
Posted: Sun Sep 17, 2017 11:55 am
by Hamers
Awesome work dude! Also these are the scenes that were cut from the game in Beta status, fully playable but unpolished:
0501 Sleepless Night Underwater
0701 Going to School Wakeup
1002 Swimming Again
Since they have almost been completed and use mostly existing assets, everything you could find in regards to them would be quite amazing. I doubt you can fully restore them, but maybe they have a lot of traces left
Re: Heavy Rain Tool
Posted: Sun Sep 17, 2017 8:42 pm
by michalss
well i hope u finnish it since Hugo had to drop it..
Re: Heavy Rain Tool
Posted: Sun Sep 24, 2017 11:38 pm
by oyunceviri
Any news?
Re: Heavy Rain Tool
Posted: Sun Oct 01, 2017 1:23 pm
by daemon1
Quantic Dream game formats were reversed all the way, and tools released to public. But it was forbidden to talk about this engine here on xentax. Any changes?
Re: Heavy Rain Tool
Posted: Mon Oct 02, 2017 4:06 pm
by HugoPeters
daemon1 wrote:Quantic Dream game formats were reversed all the way, and tools released to public. But it was forbidden to talk about this engine here on xentax. Any changes?
<snip>
No they were not, there was only a model extractor which just brute-force scans files for model data.
Re: Heavy Rain Tool
Posted: Mon Oct 02, 2017 4:18 pm
by daemon1
HugoPeters wrote:daemon1 wrote:Quantic Dream game formats were reversed all the way, and tools released to public. But it was forbidden to talk about this engine here on xentax. Any changes?
<snip>
No they were not, there was only a model extractor which just brute-force scans files for model data.
Yes, they were. You probably just don't know about that. And a proper tool was released which is able to extract all models and levels with textures, skeletons and fully weighted including cloth, from both Heavy Rain & Beyond Two Souls. I posted images with models extracted with that tool.
My question is: can I now talk about this on xentax?
Re: Heavy Rain Tool
Posted: Tue Oct 03, 2017 6:31 am
by Modman69
I think the tool was on the Cg in Games forum (QDtools)
but it seems it cannot be downloaded any longer
It's to bad since I sold my copy of Heavy Rain for PS3 a while back because I figured there was never going to be a way to get the models
I would probably go buy another copy if I knew the tools were available
(actually I bet a lot of people might buy a copy)
Re: Heavy Rain Tool
Posted: Tue Oct 03, 2017 4:16 pm
by daemon1
Modman69 wrote:I think the tool was on the Cg in Games forum (QDtools)
Tools can still be downloaded from CG and also Facepunch, and maybe other sites.
Re: Heavy Rain Tool
Posted: Wed Oct 04, 2017 10:50 am
by HugoPeters
daemon1 wrote:HugoPeters wrote:daemon1 wrote:Quantic Dream game formats were reversed all the way, and tools released to public. But it was forbidden to talk about this engine here on xentax. Any changes?
<snip>
No they were not, there was only a model extractor which just brute-force scans files for model data.
Yes, they were. You probably just don't know about that. And a proper tool was released which is able to extract all models and levels with textures, skeletons and fully weighted including cloth, from both Heavy Rain & Beyond Two Souls. I posted images with models extracted with that tool.
My question is: can I now talk about this on xentax?
Again, no they were not. Apparently you think models are the only thing what makes a game
The tool you're talking about, like I said, brute-force scans files for model data. I didn't say it doesn't produce proper models.
Give me a tool that can also interpret animation data and all the rest that comes with it and can produce a scene like I showcase on my portfolio (
http://hugo.fyi/projects/reverse-engineering-b2s) - then we'll talk