Noesis g1m/g1t/g1a importer

[Allanoon]

Yeah Allanoon sent me a few archives and I made a quick and dirty quickbms script (they changed their archive layouts a bit so Ceth outputs corrupted files).
We're not 100% sure that it works correctly so he was going to do more tests before I'd release it but if you're willing to try it out too in the meantime and send crashing g1m files if you find any, here it is:
fdata.zip
Usage is simply

Code: Select all

quickbms_4gb_files.exe fdata.bms inputFolderWithTheFdataFiles [optionalOutputFolder]

I'll suggest to remove the 1kb files as quite a few of them will make quickbms extraction process stop; Also do that with .rdb .rdx and .lang files.
It's better to extract just an handfull of files (like 1000-2000 together) as if the process hang there's the risk you won't extract everything OR you'll have to extract them again just to be sure.

[xieronis]

Yeah Allanoon sent me a few archives and I made a quick and dirty quickbms script (they changed their archive layouts a bit so Ceth outputs corrupted files).
We're not 100% sure that it works correctly so he was going to do more tests before I'd release it but if you're willing to try it out too in the meantime and send crashing g1m files if you find any, here it is:
fdata.zip
Usage is simply

Code: Select all

quickbms_4gb_files.exe fdata.bms inputFolderWithTheFdataFiles [optionalOutputFolder]

I'll suggest to remove the 1kb files as quite a few of them will make quickbms extraction process stop; Also do that with .rdb .rdx and .lang files.
It's better to extract just an handfull of files (like 1000-2000 together) as if the process hang there's the risk you won't extract everything OR you'll have to extract them again just to be sure.

Right now it's only wanting to extract 6 total files (even after deleting the 1kb files). The files come out with odd extensions (ex: .82945A44) It did extract 16 g1t files.

Edit: Apparently I had to just keep rerunning it.

[Joschka]

Update: I checked a few of these 1kb files and it looks like they're just empty so it makes the qbms script crash. Not sure what Koei is doing with these useless things, you can add a size check to prevent the script from crashing.

Also I took a look at some samples and some crash for two reasons:
-Some files have the g1m hash type in the archives but are in reality morph shape containers that they were using in Atelier games before.
-NUNO5 update (their latest cloth/hair meshes util). I'll probably won't be as annoying to crack from scratch this time contrary to SoP (it's pretty much reserved fields that started to get used) but will still be very tedious and I don't have the necessary interest/motivation right now to go through this.

So yeah don't expect an update very soon on my end.

[benjamin77]

For Fire emblem warriors three hopes some g1m's don't have working g1t's like the wolves or asura. I understand this is because the material list is from the demo not the fully game? I am wondering how or if anyone knows how to get those last remaining g1t's?

I've found a way, far from ideal but it works.

First, open CharacterEditor.kidssingletondb (in root/kidsobjdb) using Cethleann's NyotenguDatabase. You can use command prompt (if on Windows) to write the result to a .txt file, like so:
*path to NyotenguDatabase.exe* *path to CharacterEditor.kidssingletondb* > *desired output path.txt*
You can drag and drop the first two into command prompt, probably the quickest way. If you've done this right, you'll end up with a .txt file (I would upload this here, but it's too big.)

Now that you've got this file, open it in a text editor like wordpad and search it using Ctrl+F for the hash of the model you need textures for. It should appear as part of a list. From the same list, grab the hash under 'TexturesRenderStateObjectHash' and search for this instead.
This should take you to another list. The value under 'TextureBindTableCSVFileResourceHash' is the ktid hash you need, use it with Nyotengu.KTID.exe as usual and you should get the g1t file you want.

I've been able to get textures for the ashen wolves and asura models with this, but I don't know for sure if it works with all models.

I've parsed and rebuilt the correct g1t and outputted it in the same folder as ktid
But when I browse g1m in Noesis, I don't see the textrue display.
May I ask how to get the AoCThingy table so that g1m and g1t can correspond?

I refer to your method, but I don't get the hash value content of txt
May I ask where I did wrong?

[fullinu]

Yeah Allanoon sent me a few archives and I made a quick and dirty quickbms script (they changed their archive layouts a bit so Ceth outputs corrupted files).

Thanks, this script works like a charm.

Wo Long definitely seems to be a hard to crack. First, PS4 and PC version both have archives, and script works on both - this is a good news.
Bad news, is that PS4 version no longer have loose files with filenames, they probably all are in .rdb headers

Nioh2 tool by id-daemon works on models, and extract smd skeleton and ascii model. Meshes with new weights format are all borked

[Joschka]

Yeah Allanoon sent me a few archives and I made a quick and dirty quickbms script (they changed their archive layouts a bit so Ceth outputs corrupted files).

Thanks, this script works like a charm.

Wo Long definitely seems to be a hard to crack. First, PS4 and PC version both have archives, and script works on both - this is a good news.
Bad news, is that PS4 version no longer have loose files with filenames, they probably all are in .rdb headers

Nioh2 tool by id-daemon works on models, and extract smd skeleton and ascii model. Meshes with new weights format are all borked

Yeah that's expected, as Daemon's current util just outright skips the new NUNO5 chunks. So all the associated meshes (capes and other simulated parts) will render with their default positions, without the vertex shader transform applied to give them their correct shape.
Meanwhile Noesis crashes as KT updated the aforementioned chunks, since they're tightly packed it leads to a crash.

I got some samples from WH and grabbed WoLong's demo but I'm busy with other reverse engineering projects currently so not sure when I'll look into this.

[benjamin77]

Yeah Allanoon sent me a few archives and I made a quick and dirty quickbms script (they changed their archive layouts a bit so Ceth outputs corrupted files).

Thanks, this script works like a charm.

Wo Long definitely seems to be a hard to crack. First, PS4 and PC version both have archives, and script works on both - this is a good news.
Bad news, is that PS4 version no longer have loose files with filenames, they probably all are in .rdb headers

Nioh2 tool by id-daemon works on models, and extract smd skeleton and ascii model. Meshes with new weights format are all borked

Yeah that's expected, as Daemon's current util just outright skips the new NUNO5 chunks. So all the associated meshes (capes and other simulated parts) will render with their default positions, without the vertex shader transform applied to give them their correct shape.
Meanwhile Noesis crashes as KT updated the aforementioned chunks, since they're tightly packed it leads to a crash.

I got some samples from WH and grabbed WoLong's demo but I'm busy with other reverse engineering projects currently so not sure when I'll look into this.

Hello Joschka, I have successfully extracted in SAMURAI WARRIORS 5. And use Nyotengu.KTID.exe to successfully merge the g1t file into the ktid directory. But I open the model with g1m, still can't see the texture. I also used Nyotengu.Database.exe to analyze CharacterEditor.kidssingletondb to get a txt hash list, but I don't know how to match g1m and g1t, can you help me?

And there is no fmt_g1m.py file in my plugin folder? How to get this?

[Joschka]

Hello Joschka, I have successfully extracted in SAMURAI WARRIORS 5. And use Nyotengu.KTID.exe to successfully merge the g1t file into the ktid directory. But I open the model with g1m, still can't see the texture. I also used Nyotengu.Database.exe to analyze CharacterEditor.kidssingletondb to get a txt hash list, but I don't know how to match g1m and g1t, can you help me?

And there is no fmt_g1m.py file in my plugin folder? How to get this?

Hello,
After you generated the g1t file, put the g1t and the g1m in the same folder, then in Noesis go to Tools->Plugin Options->Project G1M and select "Merge All". After that simply load the g1m and the g1t next to it will be loaded automatically.

WoLong/Wild Hearts update: I started to look into these more seriously and got most of the samples loading. I'll do a few more tests/fixes later and will release a testing build here in the next days.

[Joschka]

Version 1.7.4 is now available
This update adds support for the new NUNO update that TN/OF have been using for their recent titles.



I also updated the quickbms script to properly discriminate between the g1m files and the g1h containers, you should do a new proper extraction to have everything set up accordingly. It will also ignore the useless little files.

fdata.zip

[csh0748]

Version 1.7.4 is now available
This update adds support for the new NUNO update that TN/OF have been using for their recent titles.



I also updated the quickbms script to properly discriminate between the g1m files and the g1h containers, you should do a new proper extraction to have everything set up accordingly. It will also ignore the useless little files.
fdata.zip

Hello, thank you very much for your hard work on this tool. What I want to know now is that the g1m and g1t files obtained after using quickbms to unpack Wolong’s assets seem to be a bunch of random file names. How can I Know which file number corresponds to the corresponding model in the game? Or is there a way to unpack other assets to get a list file table for mapping the corresponding model in the game? Thank you again

[benjamin77]

Hello,
After you generated the g1t file, put the g1t and the g1m in the same folder, then in Noesis go to Tools->Plugin Options->Project G1M and select "Merge All". After that simply load the g1m and the g1t next to it will be loaded automatically.

Thanks Joschka

[iceS]

Version 1.7.4 is now available
This update adds support for the new NUNO update that TN/OF have been using for their recent titles.



I also updated the quickbms script to properly discriminate between the g1m files and the g1h containers, you should do a new proper extraction to have everything set up accordingly. It will also ignore the useless little files.
fdata.zip

Script error while unpacking RRPreview.rdb

Code: Select all

 0000000000000000 3200832    srsa\0x93f658ed.srsa
325519424
325519480
1397904193
  0000000000000000 4644528    srsa\0x58883b0c.srsa
328782896
328782952

Error: offset in GetVarChr (0x0000000000000000) is bigger than the var (0x00000000)

Last script line before the error or that produced the error:
  153 GetVarChr MAGIC MEMORY_FILE 0 long
  coverage file 0    21%   328711999  1523384985 . offset 000000001398d472

[Joschka]

Hello, thank you very much for your hard work on this tool. What I want to know now is that the g1m and g1t files obtained after using quickbms to unpack Wolong’s assets seem to be a bunch of random file names. How can I Know which file number corresponds to the corresponding model in the game? Or is there a way to unpack other assets to get a list file table for mapping the corresponding model in the game? Thank you again

Use these to make the g1t file associated to a given g1m. As for filenames they are stripped so you will only have the hashes.

WHWOThingy.zip

G1m hash on the left as usual, then

Code: Select all

Nyotengu.KTID.exe ..\kidsobjdb\0xb290631c.kidsobjdb ..\g1t ..\ktid\valueOnRightColumn.ktid

Only the non field props are covered by these (ie characters, mobs, animatable models etc), I'm not dling several dozens of gigs to make exhaustive file lists for games I'm not personally interested in.

Many thanks to Allanoon for testing and reporting bugs for this update.

[benjamin77]

Hello, thank you very much for your hard work on this tool. What I want to know now is that the g1m and g1t files obtained after using quickbms to unpack Wolong’s assets seem to be a bunch of random file names. How can I Know which file number corresponds to the corresponding model in the game? Or is there a way to unpack other assets to get a list file table for mapping the corresponding model in the game? Thank you again

Use these to make the g1t file associated to a given g1m. As for filenames they are stripped so you will only have the hashes.
WHWOThingy.zip

G1m hash on the left as usual, then

Code: Select all

Nyotengu.KTID.exe ..\kidsobjdb\0xb290631c.kidsobjdb ..\g1t ..\ktid\valueOnRightColumn.ktid

Only the non field props are covered by these (ie characters, mobs, animatable models etc), I'm not dling several dozens of gigs to make exhaustive file lists for games I'm not personally interested in.

Many thanks to Allanoon for testing and reporting bugs for this update.

good job

[fullinu]

Version 1.7.4 is now available
This update adds support for the new NUNO update that TN/OF have been using for their recent titles.

I also updated the quickbms script to properly discriminate between the g1m files and the g1h containers, you should do a new proper extraction to have everything set up accordingly. It will also ignore the useless little files.

Thank you, this is really helpful