Page 1 of 3

Bloodstained: Curse of the Moon 1 & 2 (and other Inti Creates) stuff

Posted: Sun May 02, 2021 6:48 am
by xttl
Hello everyone, I managed to find out very little information about these games from the interwebs, so maybe someone will find this useful.

Here's a tool for decrypting/encrypting (if you can really call it encryption, it's not using AES or any other real encryption algorithm) datafiles from COTM1 and COTM2. Not all files encrypted, and I think some are compressed in addition to being encrypted. It's a very simple single file C program, should compile and work fine with gcc, clang, msvc, pretty much anything.

It may also work for some other Inti Creates games, but I haven't tried.

I have no idea where the game code reads or calculates the keys for the "encryption algorithm", but it seems there are four different possible keys used with the same decryption code, one per data type ("obj", "bft", "set" or "scroll").

The same keys seem to work for both COTM1 and COTM2 on all platforms.

Also, in the PC versions of both games, the filenames have been obfuscated changing each file's name to the MD5 hash of the actual filename. It is possible to recover the original filenames by logging them as the game accesses them (by eg. hacking the binary to report them via OutputDebugStringA), or using file listings from console versions which don't have obfuscated filenames. Note that for some filenames, you'll need to add a platform specific "Win/" directory prefix before hashing the filename (on Switch, these files would be in an actual "NX/" subdirectory inside the romfs, for some reason there is no equivalent subdirectory inside the PS4 version's package).

Using these methods, I have recovered original filenames for all but two of the files in the the DataHash directory on PC for COTM2. (I played through the game multiple times taking different routes, but the logger didn't catch anything which would match those two when MD5 hashed, unfortunately it's not really feasible to brute force the remaining ones, at least not with only a single albeit reasonably powerful GPU, the filenames could be up to 20-30 characters or even more, and may contain at least lowercase and capital letters, numbers, underscores and a single dot.)

For COTM1, I have recovered 261 of 296 filenames based on file listings from console versions. I'll need to play through it a few times with a logger patched one day...

Next up is going to be figuring out the actual map and background graphics formats. I want to write a program which can show the maps. :-)

Btw. if you want to look at them, use the "set" key for map*.stb files and "scroll" key for map*.scb files. *.osb files use the "obj" key.

The zip includes the encdec.c program and MD5 filename hash lists for COTM1 and COTM2.

(removed, see viewtopic.php?p=176034#p176034)

edit: oh yeah, if you rename map00.stb to map01.stb (look at the hash lists), you'll get a debug map (in both games) when you start the game! nothing that interesting or useful there, though.

edit2: forgot the line for ARGV0 in the args enum...

edit3: "bft" key is used for fonts, and the keys for all four data types are generated like this:

Code: Select all

#define WTFSTRING "90210"
#define BASEKEY 0xA1B34F58CAD705B2LL

uint64_t type2key (const char *type)
{
  uint64_t key;
  int i, l;
  char buf[12];
  
  strcpy(buf,type);
  strcat(buf,WTFSTRING);
  
  l = strlen(buf);
  
  key = BASEKEY;
  
  for (i=0; i<l; i++)
  {
    key += buf[i];
    key *= 141;
  }
  
  return key;
}
(this gives correct 64-bit keys with input "obj", "bft", "scroll" or "set")

The scroll, font and obj data have been zlib compressed before obfuscation, set data seems uncompresed. The first four bytes are a header which tells the required length for decompression buffer.

Here are raw RGBA views of BMPFont.bfb and Win/map01.scb after deobfuscation and zlib decompression (imgur links because it seems only one attachment per post is permitted and I'll only make a 2nd post when/if I can do something interesting like editing the stage layout, or someone else posts which just happened):

pic1
pic2

I stll don't know the actual header (after decompression), stage layout, etc., etc. formats.

I changed the name of the program to inti_encdec since it also works for other Inti Creates games. The version in the zip can now decode/encode larger files which are found in games other than COTM1/COTM2. It can also now decode/encode the system and game save data files in COTM1 and COTM2! (but probably not any of the other games, the save keys are different between COTM1 and COTM2 so probably the other games all use their own keys too).

I have also added file lists for other games to the zip (thanks RandomTBush) plus added the missing files to the COTM1/COTM2 lists.


I have tested decoding & re-encoding the map01 files with it and the game still loads it fine. :) Next up will be figuring out the actual map formats...

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Mon May 03, 2021 6:08 am
by RandomTBush
Ah, if only I had this sorta thing earlier. Excellent! :)

I actually went through the effort of matching the MD5s to the filenames myself a while back, here's the remaining hash/filename pairs you missed from Curse of the Moon 1:

Code: Select all

b507cdff033c27d984488a2bc3f3d2ba MapList.json
c9b695137c3dbd390150e4348f074456 Patch/BMPFont.bfb
1bfe6e6cf774a0870c018295951bf12b Patch/Calibration.ttb
84e0bdd4f11b909afd106b92b7fa2eec Patch/Calibration00.osb
71252704d813442e58c71c759d4d4ad1 Patch/EndingText00.osb
f9cf939192ce50e965fa11c4a0de01b2 Patch/GraphicText00.osb
98fbc7acc38ac990b17a5378670d554c Patch/GraphicText04.osb
be2e9161a3a6bb7ef65ca32561ac77e4 Patch/IconClear00.osb
8689aef4b3ef37c2971f133a5f0a459e Patch/map01.scb
0fd633200c773c7e84f748b628328cc2 Patch/map01.stb
e60dfd11647f3c2e1177d03c2eb8fa0c Patch/map02.scb
32cc4ef02249169f6e0a7fc79054a1a7 Patch/map02.stb
c4c6b279e307d8b2f7f46fc63430e8b5 Patch/map03.scb
b004be0f1bee105e0833d8b6456b6b75 Patch/map03.stb
5c035e8886b9d1ce24515a858f7efdb2 Patch/map04.scb
1ea347e5b1a0403acf227c16e11eee22 Patch/map04.stb
2bf50e8a81ac3922e77961d4964c4bbe Patch/map05.scb
5af88824a4446fe0d2f1aaa909038549 Patch/map05.stb
88d5983aa1760c94ccc8dfe103b21382 Patch/map06.scb
02d49b6034dac68b8bb19fa07edf4fcd Patch/map06.stb
4c14f860a52c402993341f172fd74f4e Patch/map07.stb
ae64a0c89814868a34acb4b62200548c Patch/map08.scb
9f8740b5f4013f455c4092d25d7441ec Patch/map08.stb
268d9be0086f14534c599281eec71c0f Patch/map09.scb
a53f870e5c4c376ec19450a8e8ba3d7f Patch/map09.stb
4112b562760811b246db33484dce725b Patch/MapDemo00.osb
88023ef468a04af2f2901a6a3683cf11 Patch/MapDemo03.osb
48fb4636e9cd3dacf77a6a6bf98e5703 Patch/Openingext00_en.osb
ad3cd8a3bda596b1547359e2479e0625 Patch/Option.ttb
39b9333c89ce132badcc0c2de3a58912 Patch/OptionKeyboard.ttb
2f493b619a9cf9f79e8291655dbc5e36 Patch/PauseMenu.ttb
7ce96b6103c39567090f03eb5dff9e56 Patch/Side01.osb
635acf9a9f63b05d9b1ce9382633a8a6 Patch/Staffroll_Picture00.osb
ccda2e2e84f49dfd56530e35616a3b44 Patch/Title00.osb
8e2a602aadb120226e2393c6afb24dcf Patch/Window00.osb
And one from Curse of the Moon 2:

Code: Select all

7f51424d446ad5e968e5251366c617ce Win/Window00.osb
That should just leave one file from Curse of the Moon 2 (db5f673dd61144c9c37e7b388e74e42e), which apparently doesn't belong to CotM2 at all, as it's the Demonpillar (zk04_im0.osb) from Dragon Marked for Death.

(EDIT: I can also confirm that your program also works just fine with files from both Blaster Master Zero 2 and Mighty Gunvolt Burst! I assume Dragon Marked for Death will work too, if the leftover file in CotM2 is any indication.)
(EDIT the 2nd: Yep, Dragon Marked for Death also works, barring a handful of files.)

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Mon May 03, 2021 7:32 am
by xttl
Oh yeah, I got my file list for COTM1 from an unpatched Switch version, that explains why all the Patch/* files were missing. MapList.json was missed because I didn't think to check if any of the hashes from COTM2 match COTM1. The proper filename for the file from the wrong game could probably be found from the console versions of that game.

Btw. which RGBA viewer are you using? I can't figure out how to get the colors right in BinxelView. Wrong R/G/B/A byte ordering I guess, actually it seems the program may not have proper support for 32bpp RGBA at all, only 24bpp RGB.

I thought I could avoid having to write any graphics code until later (if/when I have enough of the formats figured out for drawing the actual stage maps), but if there's not a better raw RGBA data viewer I guess it can't be helped...

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Mon May 03, 2021 8:12 am
by RandomTBush
xttl wrote: Mon May 03, 2021 7:32 am Oh yeah, I got my file list for COTM1 from an unpatched Switch version, that explains why all the Patch/* files were missing. MapList.json was missed because I didn't think to check if any of the hashes from COTM2 match COTM1. The proper filename for the file from the wrong game could probably be found from the console versions of that game.

Btw. which RGBA viewer are you using? I can't figure out how to get the colors right in BinxelView. Wrong R/G/B/A byte ordering I guess, actually it seems the program may not have proper support for 32bpp RGBA at all, only 24bpp RGB.

I thought I could avoid having to write any graphics code until later (if/when I have enough of the formats figured out for drawing the actual stage maps), but if there's not a better raw RGBA data viewer I guess it can't be helped...
Yeah, that would explain it then, heh.

Anyway, I use TiledGGD for previewing the files, with the following adjustments: Image > Format > 32 Bit per pixel, Palette > Colour order > RGB, and Palette > Alpha Settings... > Uncheck "Ignore Alpha value", then you can just adjust the width and height with the arrow keys, and shift the image with Page Up/Page Down. Some files in CotM2 specifically (including the unused file) will look like an outline / silhouette in the program, that's because they use palettes located above the image (which is then applied to the image's red channel).

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Tue May 04, 2021 4:47 am
by xttl
Updated the zip in first post:
  • Turns out, the key is NOT supposed to be reset every 384kB, large files (which some Inti games other than COTM1/COTM2 have) can now be unpacked and repacked successfully. Thanks to RandomTBush for testing this.
  • Added system and game save data decryption/encryption for COTM1 and COTM2. The same code (actually it's just the same obfuscation applied twice with different keys) could also work for other Inti games, but game specific keys/keystrings are required.
  • Added three new possible asset types/keys to the list "txt20170401", "snd90210", "json180601", cannot be tested with any files from COTM1/COTM2. (unless with something from inside one of the large archive files..?)
(the save data may well be checksummed in addition to the obfuscation, I haven't tried to edit it yet)

I may need to update the tool to use memorymapped files, compress/decompress in chunks and/or encode/decode inside the same buffer to reduce memory use a bit if there are compatible games with even larger files out there...

edit: another little update:
- added key for tb2 files from Blaster Master Zero 2 & Dragon Marked for Death
- the program is using memory mapped files now, and decodes/encodes inside a single buffer (should require less memory for large files)

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Sat Jun 26, 2021 5:43 am
by DMFDLancer
Hey, uh, I was Googling around and found this topic. Big Inti fan, been searching this kind of stuff as it came up.

It sounds like you guys are using this tool to mod the games? Texture swaps, potential save edits, stage swaps... On the Steam versions? That's really hype, if true. I'm pretty not so good at this kind of thing but I wanna learn and probe the games, regardless.

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Sun Jun 27, 2021 3:06 am
by xttl
Sorry for not posting any updates in a long time! I haven't forgotten about this, just have had other things to do (and some difficulties doing much of anything at all...)
DMFDLancer wrote: Sat Jun 26, 2021 5:43 am It sounds like you guys are using this tool to mod the games? Texture swaps, potential save edits, stage swaps... On the Steam versions? That's really hype, if true. I'm pretty not so good at this kind of thing but I wanna learn and probe the games, regardless.
Stage swapping was always and is possible without any tools or knowledge about the file formats and obfuscation if you just know the correct MD5 hashed filenames. The main stage definition files are named "map##.stb" (or, in patched COTM 1 for some stages, "Patch/map##.stb").

So if you wanted to eg. swap the first real stage (map01) in COTM 2 with the debug stage (map00), you need to calculate the MD5 hashes for "map00.stb" and "map01.stb". On pretty much any Linux/Unix command line (including WSL), commands "echo" and "md5sum" (or in some cases just "md5") can be used for this:

Code: Select all

$ echo -n "map00.stb" | md5sum
8404a4d77634c07774590adce322c008  -
$ echo -n "map01.stb" | md5sum
2d463c71d6c85a02eb19a07b04a550d8  -
which means: just rename 8404a4d77634c07774590adce322c008 to 2d463c71d6c85a02eb19a07b04a550d8 in the DataHash directory/folder to replace the first stage with the debug stage.

In COTM 1, you may need to replace "Patch/map01.stb" instead:

Code: Select all

$ echo -n "Patch/map01.stb" | md5sum
0fd633200c773c7e84f748b628328cc2  -
so rename 8404a4d77634c07774590adce322c008 to 0fd633200c773c7e84f748b628328cc2.

If you insist on using Windows without WSL (or at least cygwin, msys or something), I'm sure there are good tools to calculate MD5 hashes from strings for Windows too, or you could look up some javascript thing from the webs which runs on your browser. Windows since Vista (or 7?) has included a command line tool called certutil which can MD5 (and SHA1, SHA256, etc.) hash files but I don't know if you can somehow pipe strings to it... you'd also need an echo command that can omit newline characters anyway. Powershell could probably be used for hashing strings somehow, but I know next to nothing about it.

(or: you could just use the filename hash list text files from the encdec tool zip and be done with it)

Graphics modifications are already possible with the tool from this thread. There are huge limitations though: since not much is known about the actual format the level and sprite graphics data is, you'll have to edit everything as unstructured raw graphics data. You won't be able to make an existing sprite frame larger for example, you can just draw over what's already there. You also have no control over how tiles are reused to build other frames.

Graphics ripping is very possible, and I think so far the only thing somebody has used this tool & info for. That does also need some more manual work than it would require if the sprite/background formats were fully understood.

Everything else (editing stage layouts or creating wholly new stages, editing saves, etc.) needs further reverse engineering of the game executable and/or extensive guessing by looking at and carefully editing/corrupting the descrambled and decompressed data. You can de/rescramble and de/recompress all level and save data (among other things) with the tool in this thread, but without knowing the actual data formats it isn't possible to do anything interesting. So far, by editing the level "set" (.stb) or "scroll" (.scb) files, I have only managed to either make the game crash, make items, enemies and triggers from certain rooms or all rooms on a stage disappear, or make a couple of misplaced tiles or invisible barriers appear here and there.

The scrambling and compression methods, and probably most if not all of the actual file formats are the same between the PC/Steam and console versions, so if and when this stuff is figured out, it will also be possible to edit the console versions (assuming you have access to a hacked or developer console, of course).

Personally, I'll be happy if I can just figure out enough of the map format to write a map viewer/editor for COTM 1 and 2. (or someone else figures it out and posts the code/info here ;)

oh yeah, in theory you could also already translate the game to another language since you can de/rescramble and de/recompress both the font graphics files and the ttb files which contain all or most of the text, but you'd have to live with the length of the original text unless you figure out the ttb format (it probably isn't very complex)

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Wed Jun 30, 2021 12:23 pm
by xttl
Well, here is a tool (source included) which can dump and rebuild TTB files in COTM1 and COTM2 (possibly other Inti games too) which contain all the in-game text.
inti_textconv.zip
Refer to the file lists included to see which data files (under Data/ or DataHash/) are the TTB files.

First, convert the TTB file to a plain text file like this:

Code: Select all

textconv d <ttbfile> <textfile>
eg. to convert PauseMenu.ttb:

Code: Select all

textconv d 208b1e0c2edfd32ad57b9ff7e03d64b7 PauseMenu.txt
then edit the file with any text editor which supports UTF-8, and convert it back to a TTB like this

Code: Select all

textconv e PauseMenu.txt 208b1e0c2edfd32ad57b9ff7e03d64b7
Do not try to change the TXT file structure (don't touch any of the numbers and don't try to split the text for a TTB record to multiple lines, don't add empty lines, etc.).

With this tool you could, for example, translate the games to any language. The text length CAN be changed! (though you should probably stay within a reasonable range of the original text to avoid any possible problems with text positioning and dialog box size, also pay attention to the "\n" newlines and line lengths)

If you need characters which are not included in the original font, then you'll need to edit the BMPFont/BMPFont2 files (as raw graphics since there is no special tool for this yet...). I could help with this is required.

The game uses UTF-8 encoding internally for both the English and Japanese text. It seems SOME (kana) but not all (kanji) Japanese characters work even if the game is set to English mode. Also, all English/latin alphabet characters seem to still work when the game is in Japanese mode. If you are willing to break Japanese support, this can be used to add any diacritics required by your target language.

There are other unused characters in the font too (eg. PlayStation and Switch button graphics, on PC it always uses the Xbox graphics) that may be possible to use for this purpose, but I don't have the character values used for these.

Note that parts of the original English text use the double-width "ideographic space" unicode character (U+3000) instead of a regular space (U+0020), especially text that is displayed using the smaller font (in eg. PauseMenu.ttb). Do not try to change these to regular spaces (or you'll get no space at all when the text is displayed).

If the original text uses C-style formatting specifiers (if you see a percent sign somewhere), do not change the amount or order of those formatting specifiers. See MainGame.ttb from COTM1 or COTM2 for examples of this.

Any more questions, ask here.

edit: seems some versions of Windows notepad may add garbage bytes (BOM) to the beginning when a text file is (re)saved as UTF-8 which confused the "parser" in textconv, updated the zip with a fixed version

edit2: whoops, encoding was not scrambling the last four bytes of the output ttb file, fixed

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Thu Jul 01, 2021 9:48 pm
by DMFDLancer
Sorry, I'm just really really dumb with this kind of stuff lol

Gonna see later if I can modify Dragon Marked For Death's JSON files. They're readily readable (at least they should be after I use your app, if the Switch files are any indication) and there's data corresponding to damage mechanics and stat growth.

I don't know how much this info could help, but Inti staff mentioned in a stream Q&A that every Inti Creates game from Gunbike to Megaman to Gunvolt to CotM all run on the same Inti Creates Engine, or more recently (Burst, CotM2, Gunvolt 3 IIRC) it's successor. I imagine it's a nested doll situation?

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Fri Jul 02, 2021 12:03 pm
by xttl
IIRC RandomTBush tried it, and the JSON key found from COTM2's executable unfortunately does not work for the JSON files from DMFD. None of the others found so far worked either, so it needs more research, probably just new keystrings. I have only purchased copies of COTM1 and COTM2 myself so it is kind of difficult for me to look at the other games, but I'll see what I can do.

(actually, I also have the Mega Man legacy collections on PC and MM9 on PS3, but the MM9/MM10 version of the engine is much older and probably different enough that I'm not going to bother with it for now)

Btw. COTM1 and COTM2 also have one JSON file each which is completely unprotected (not compressed, not scrambled), but it also seems completely unused by the game. It's "MapList.json" = Data/b507cdff033c27d984488a2bc3f3d2ba or DataHash/b507cdff033c27d984488a2bc3f3d2ba

I wonder if the other games really use the JSON files either? (possibly, since Inti bothered to protect them on PC after all...)

Also IIRC console versions of DMFD have the JSON files completely unprotected (but that does not really help if you want to modify them on PC, because almost certainly the PC version will refuse to read unprotected data files if you copied one over from a console version).

edit: ok, I've figured it out, I'll update inti_encdec to decode/encode JSON files from DMFD PC soon
for anyone curious, the trick is to zlib decompress first, then descramble after with a key generated from password/keystring "xN5sUeRo"
I did not expect there would be filetypes which are scrambled before they are compressed (zlib does not even really decompress the scrambled files, on the contrary) so I have to restructure the program a bit.

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Fri Jul 02, 2021 11:17 pm
by RandomTBush
Oh yeah, and I found out why the BISAR/BIGRP files from CotM and MGB wouldn't decrypt either, xttl -- you need to change "COMP_YES" to "COMP_NO" for the "snd" type, as they're not compressed.

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Sat Jul 03, 2021 12:52 am
by xttl
Yeah, I fixed that some time ago already (set snd filetype to not compressed) but didn't get around to upload a new version of the program...

Btw. snd filetype is NOT for bigrp files, only for bisar. The bigrp files don't look like compressed or scrambled at all (at least the main header at the start of file isn't, and not all of the contents are either because plain MIDI MThd/MTrk headers among other things can be found inside the bigrp files from COTM1 & COTM2).

Anyway, here's a new version of program which handles snd/bisar files correctly and also can decode & encode JSON files from DMFD PC (use "json2" filetype):
inti_encdec_v2d.zip
Note that I changed the order of command line arguments, filetype comes right after the command now, eg.:

Code: Select all

inti_encdec d json2 62c7a9ef056d3d4e35c976486a0994e8 hud_stage_chat.json
inti_encdec e scroll Map01.scb ab255c3c30bf31caf83d9819abf32b55
edit: damn it I knew I'd forget something again, fixed json2 output (was not writing header properly)

edit2: fixed one incorrect printed status message, also added support for DMFD PC save data as filetype "save3" (look for "SystemData.bin" wherever the game puts save files, probably in game dir, somewhere under your Steam directory or Windows user home dir, not really tested because I don't have the game myself, I'll remove the filetype if it doesn't work) -- redownload if you got "v2" without "b"

edit3: removed save3 filetype since it isn't working as expected

edit4: restored save3 filetype, you'll need to input your SteamID to descramble/scramble savegames from DMFD PC

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Sat Jul 03, 2021 5:41 am
by DMFDLancer
Welp, HitConstParam.json was gutted on Steam. It's just this cResHudWorkCtrl thing... It used to be stuff like mpPlayerStatusGrowRate. I'm gonna see if I can impact anything with the others, as those seem to be hud related, but it's not looking great.

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Sat Jul 03, 2021 5:43 am
by xttl
DMFDLancer wrote: Sat Jul 03, 2021 5:41 amWelp, HitConstParam.json was gutted on Steam. It's just this cResHudWorkCtrl thing... It used to be stuff like mpPlayerStatusGrowRate. I'm gonna see if I can impact anything with the others, as those seem to be hud related, but it's not looking great.
I wonder what happens if you copy the file from a console version, modify it and and encode it?

...probably they just moved that stuff into the executable or another (binary) datafile...

I guess it could even be just a leftover in the console versions too, like MapList.json in COTM / COTM2? Unless someone with a hacked console and the game has already tested that modifying it does really effect the game...

Re: Bloodstained: Curse of the Moon 1 & 2 stuff

Posted: Sat Jul 03, 2021 6:56 am
by DMFDLancer
Tried removing HitConstParam, and cramming in the console one, nada.

I did mess with hud_stage_chat, and I stretched the chat wheel out to oblivion by boosting every coordinate I could find. That's technically a mod, I guess. (: The file was identical to the old Switch version, but with some stuff trimmed. (No declared names or something?)

I'm really slow at this, but I'll keep plugging away and see if I can do something. Maybe I can "downpatch" to whatever ancient Switch version I have? Either that or I'll break something in a weird way like I did when I was doing file swaps between GV2 and iX.

EDIT: Tried the save data decoder for Dragon. Uhhhhh... I think it worked? Either way it's totally unreadable. There's a bit at the start (#Ÿ®h ‘w and some NUL characters, according to Notepad++) that match but the rest is totally different. I'll mention that there's a config.json in the same folder, but that's just a display selector.