Rage 2 .arc .tab Archives

[blenux]

Hi all any chance of getting some Rage 2 scripts, files are .arc and .tab pretty sure they are different to the Rage 1 formats.

File and Last files uploaded here in each folder.

[kilik]

it's look

https://forum.xentax.com/viewtopic.php?f=10&t=19143

[infogram]

https://github.com/gibbed/Gibbed.JustCause4 can apparently work with compressed JC4 arc/tab files perfectly, so it should be pretty easy to update it for Rage 2's files.. Maybe gibbed will push an update soon to support them, otherwise I'll try taking a look later.

EDIT: had a quick look through, seems they changed the header & file entry formats between JC4 and R2, header had a file entry count added to it (shifting some of the fields around), but that's easy to work around in gibbed's code.

Harder part is the file entry changes, seems they added 4 bytes to the start of the entry format but I'm not sure what those 4 bytes are for though, could be they extended the name-hash from 32 to 64bit, or maybe the 4 bytes is used as an XOR key or something (edit:nope)... still need to try some things out.

EDIT: with those changes files seem to extract fine! Haven't got any file-names associated with the extracted files yet though, so it's all files with names like 0D116543.unknown

Hoping they haven't changed the name hashing algo too much, since I don't really want to go reversing the EXE itself... I'll try looking into it tomorrow, hopefully be able to post a fixed up tool soon.

EDIT: For anyone interested here's a 010 template for reading JC4 & RAGE2 .tab files, you can see the difference in the file entries here (TAB_ENTRY_V2 vs TAB_ENTRY_V3)

Code: Select all

//------------------------------------------------
//--- 010 Editor v9.0.1 Binary Template
//
//      File: AvalancheTabFile.bt
//   Authors: infogram
//   Version: 1.0
// File Mask: *.tab
//  ID Bytes: 54 41 42 00
//
// Structs based on Gibbed.JustCause4, thanks to gibbed!
//
// TODO: other avalanche games (JC2/JC3/MadMax?)
//------------------------------------------------

// Split magic/version portion of header to its own struct
// That way we can decide what other part of the header to read based on version #...
typedef struct TAB_MAGIC
{
  DWORD Magic;
  WORD MajorVersion;
  WORD MinorVersion;
};

typedef struct TAB_HEADER_V2
{
  DWORD Alignment;
  DWORD Unknown0C;
  DWORD CompressedBlockSize;
  DWORD UncompressedBlockSize;
  DWORD CompressedBlockCount;
};

typedef struct TAB_ENTRY_V2 
{
  DWORD NameHash;
  DWORD Offset;
  DWORD CompressedSize;
  DWORD UncompressedSize;
  WORD CompressedBlockIdx;
  BYTE CompressionType; // 1 = Zlib, 4 = Oodle
  BYTE CompressionFlags;
};

typedef struct TAB_HEADER_V3
{
  DWORD Alignment;
  DWORD FileEntryCount;
  DWORD CompressedBlockCount;
  DWORD Unknown14;
  DWORD CompressedBlockSize;
  DWORD UncompressedBlockSize;
};

typedef struct TAB_ENTRY_V3
{
  DWORD NameHash; // Is this still the hash? or maybe Unknown04? or maybe both fields are a 64-bit hash?
  DWORD Unknown04;
  DWORD Offset;
  DWORD CompressedSize;
  DWORD UncompressedSize;
  WORD CompressedBlockIdx;
  BYTE CompressionType; // 1 = Zlib, 4 = Oodle
  BYTE CompressionFlags;
};

typedef struct TAB_COMPRESSED_BLOCK
{
  DWORD CompressedSize;
  DWORD UncompressedSize;
};

TAB_MAGIC Magic;
if(Magic.MajorVersion == 2 && Magic.MinorVersion == 1)
{
  // V2.1 == Just Cause 4
  
  TAB_HEADER_V2 Header;
  TAB_COMPRESSED_BLOCK CompressedBlocks[Header.CompressedBlockCount];
  
  // No file count in V2, so calc how many entries based on filesize
  local int numEntries = (FileSize() - FTell()) / 0x14;
  TAB_ENTRY_V2 FileEntries[numEntries];
}
else if(Magic.MajorVersion == 3 && Magic.MinorVersion == 1)
{
  // V3.1 == RAGE 2
  
  TAB_HEADER_V3 Header;
  TAB_COMPRESSED_BLOCK CompressedBlocks[Header.CompressedBlockCount];
  TAB_ENTRY_V3 FileEntries[Header.FileEntryCount];
}
else
{
  Printf("Unknown TAB version v%d.%d!", Magic.MajorVersion, Magic.MinorVersion);
}

[OrangeC]

This could be really useful as the OGG sound files in the game15.arch and game16.arc archives don't have names.

[rengareng]

https://github.com/gibbed/Gibbed.JustCause4 can apparently work with compressed JC4 arc/tab files perfectly, so it should be pretty easy to update it for Rage 2's files.. Maybe gibbed will push an update soon to support them, otherwise I'll try taking a look later.

EDIT: had a quick look through, seems they changed the header & file entry formats between JC4 and R2, header had a file entry count added to it (shifting some of the fields around), but that's easy to work around in gibbed's code.

Harder part is the file entry changes, seems they added 4 bytes to the start of the entry format but I'm not sure what those 4 bytes are for though, could be they extended the name-hash from 32 to 64bit, or maybe the 4 bytes is used as an XOR key or something (edit:nope)... still need to try some things out.

EDIT: with those changes files seem to extract fine! Haven't got any file-names associated with the extracted files yet though, so it's all files with names like 0D116543.unknown

Hoping they haven't changed the name hashing algo too much, since I don't really want to go reversing the EXE itself... I'll try looking into it tomorrow, hopefully be able to post a fixed up tool soon.

EDIT: For anyone interested here's a 010 template for reading JC4 & RAGE2 .tab files, you can see the difference in the file entries here (TAB_ENTRY_V2 vs TAB_ENTRY_V3)

Code: Select all

//------------------------------------------------
//--- 010 Editor v9.0.1 Binary Template
//
//      File: AvalancheTabFile.bt
//   Authors: infogram
//   Version: 1.0
// File Mask: *.tab
//  ID Bytes: 54 41 42 00
//
// Structs based on Gibbed.JustCause4, thanks to gibbed!
//
// TODO: other avalanche games (JC2/JC3/MadMax?)
//------------------------------------------------

// Split magic/version portion of header to its own struct
// That way we can decide what other part of the header to read based on version #...
typedef struct TAB_MAGIC
{
  DWORD Magic;
  WORD MajorVersion;
  WORD MinorVersion;
};

typedef struct TAB_HEADER_V2
{
  DWORD Alignment;
  DWORD Unknown0C;
  DWORD CompressedBlockSize;
  DWORD UncompressedBlockSize;
  DWORD CompressedBlockCount;
};

typedef struct TAB_ENTRY_V2 
{
  DWORD NameHash;
  DWORD Offset;
  DWORD CompressedSize;
  DWORD UncompressedSize;
  WORD CompressedBlockIdx;
  BYTE CompressionType; // 1 = Zlib, 4 = Oodle
  BYTE CompressionFlags;
};

typedef struct TAB_HEADER_V3
{
  DWORD Alignment;
  DWORD FileEntryCount;
  DWORD CompressedBlockCount;
  DWORD Unknown14;
  DWORD CompressedBlockSize;
  DWORD UncompressedBlockSize;
};

typedef struct TAB_ENTRY_V3
{
  DWORD NameHash; // Is this still the hash? or maybe Unknown04? or maybe both fields are a 64-bit hash?
  DWORD Unknown04;
  DWORD Offset;
  DWORD CompressedSize;
  DWORD UncompressedSize;
  WORD CompressedBlockIdx;
  BYTE CompressionType; // 1 = Zlib, 4 = Oodle
  BYTE CompressionFlags;
};

typedef struct TAB_COMPRESSED_BLOCK
{
  DWORD CompressedSize;
  DWORD UncompressedSize;
};

TAB_MAGIC Magic;
if(Magic.MajorVersion == 2 && Magic.MinorVersion == 1)
{
  // V2.1 == Just Cause 4
  
  TAB_HEADER_V2 Header;
  TAB_COMPRESSED_BLOCK CompressedBlocks[Header.CompressedBlockCount];
  
  // No file count in V2, so calc how many entries based on filesize
  local int numEntries = (FileSize() - FTell()) / 0x14;
  TAB_ENTRY_V2 FileEntries[numEntries];
}
else if(Magic.MajorVersion == 3 && Magic.MinorVersion == 1)
{
  // V3.1 == RAGE 2
  
  TAB_HEADER_V3 Header;
  TAB_COMPRESSED_BLOCK CompressedBlocks[Header.CompressedBlockCount];
  TAB_ENTRY_V3 FileEntries[Header.FileEntryCount];
}
else
{
  Printf("Unknown TAB version v%d.%d!", Magic.MajorVersion, Magic.MinorVersion);
}

The game uses 64bit hash for names, and entries are sorted by the hash.
Hash of

Code: Select all

text/master_eng.stringlookup

should be

Code: Select all

0x8453EE3581F31F39

.
I found the hash algorithm. It's MurmurHash3 128bit x64 version. The first 64bit is taken from the resulting 128bit.
I tried it here

[infogram]

The game uses 64bit hash for names, and entries are sorted by the hash.
Hash of

Code: Select all

text/master_eng.stringlookup

should be

Code: Select all

0x8453EE3581F31F39

.
I found the hash algorithm. It's MurmurHash3 128bit x64 version. The first 64bit is taken from the resulting 128bit.
I tried it here

Holy crap, thanks a ton for finding that out! I was trying a bunch of things, like weird 64-bit versions of lookup3, searching the exe for hash algo patterns.. really wasn't having any luck with it though, huge thanks for figuring it out

(wonder what made them change the hash algo, they do still use lookup3 in some string-related stuff, guess Murmur must be faster to calculate or something?)

Anyway I've updated gibbed's JustCause4 tool with the changed file format + Murmur3 algo, and it all seems to work great! Not really sure if we can repack at all though, maybe the game can load loose files if we're lucky.

Now I just need to test it against all of the archives and build a file-list to go with the hashes now (my friend Grinderkiller is working on that part), after that I'll post it up here (along with the changed source code of course), should be ready by tomorrow hopefully

(edit: just heard from Grinderkiller that the game will load in loose files fine! It worked for the videos at least, haven't tested any other files yet afaik, but sounds good so far!)

Rage2Unpack Release

Rage2Unpack Download: http://bit.ly/2HO6PRl

Read the readme to learn how to use it, and download the latest filelist from https://forum.xentax.com/viewtopic.php?p=153102#p153102 to extract more files!

Rage2Hook Release

Also made a DLL hook for dumping filenames from the game, it'll dump the filenames of any content that the game loads in.

If people could play through the game with this DLL enabled and post the rage2hook.log from it here, hopefully we can combine them all and cover more of the archives that way, like how people have been doing for RE2 Remake.

Rage2Hook Download: http://bit.ly/2VZChpx
Just extract the dll file next to your RAGE2.exe, and now whenever you run the game all content filenames will be logged to rage2hook.log, thanks in advance to anyone that submits logs for us

(this hook is made for the patch 1 Steam release, but might work fine with others, please let me know if you have any issues with your copy!)

[OrangeC]

Thanks for tool and the hook. with the hook enabled, does the names get dumped into its own log file or does it put inot a .txt file?

[infogram]

Got the the filelists up to 16% (49920/302871), just copied some translation/audio filenames and edited them for each of the different languages, and a few other small adds.
Seems it's mostly audio related stuff that's missing (probably story related stuff, since the filelist is mostly from early-game afaik), hopefully people can help fill those gaps with their logs

Download filelists update 1: http://bit.ly/2VIEksZ
Just delete the projects directory from Rage2Unpack and extract the one from that zip in its place.

Thanks for tool and the hook. with the hook enabled, does the names get dumped into its own log file or does it put inot a .txt file?

They should get put in the rage2hook.log file next to RAGE2.exe, as long as the RAGE2 dir is writable at least (if it doesn't show up maybe run the RAGE2.exe as administrator, that should let the log be written no matter what)

(edit: the log is appended to the last log too, so don't worry about overwriting any older logs, it should always save to the end of any previous one)

After you've finished logging you can copy that rage2hook.log into "projects\RAGE 2\files\archives_win64", rename it as "new.filelist" and then run RebuildFileLists.exe, that should then check your new filelist for any missing files and add them to the right .filelist file for it, then you can check "projects\RAGE 2\files\status.txt" to see the new count of named files.
(if you want to submit your log you don't need to do this though, just post your .log file here and I'll manage the rest )

[rengareng]

The game uses 64bit hash for names, and entries are sorted by the hash.
Hash of

Code: Select all

text/master_eng.stringlookup

should be

Code: Select all

0x8453EE3581F31F39

.
I found the hash algorithm. It's MurmurHash3 128bit x64 version. The first 64bit is taken from the resulting 128bit.
I tried it here

Holy crap, thanks a ton for finding that out! I was trying a bunch of things, like weird 64-bit versions of lookup3, searching the exe for hash algo patterns.. really wasn't having any luck with it though, huge thanks for figuring it out

(wonder what made them change the hash algo, they do still use lookup3 in some string-related stuff, guess Murmur must be faster to calculate or something?)

Anyway I've updated gibbed's JustCause4 tool with the changed file format + Murmur3 algo, and it all seems to work great! Not really sure if we can repack at all though, maybe the game can load loose files if we're lucky.

Now I just need to test it against all of the archives and build a file-list to go with the hashes now (my friend Grinderkiller is working on that part), after that I'll post it up here (along with the changed source code of course), should be ready by tomorrow hopefully

(edit: just heard from Grinderkiller that the game will load in loose files fine! It worked for the videos at least, haven't tested any other files yet afaik, but sounds good so far!)

Rage2Unpack Release

So the tool seems to be working fine, and Grinderkiller got back to me about the filelists.
He collected them from the start of a new game, but so far only 5% of files are named (17901/302871).
It's a start at least, but seems there's still a lot more work to be done to get everything named...
(edit: latest filelist has 16% named, check my post below for updated filelists!)

What this means is that extracting the .tab files will mostly result in files being called stuff like "0C3247F95F5F301A.unknown" instead of being named something like "climate\zone_0\diffuse_textures_mid_0.ddsc", afaik it's mostly sounds that are unnamed, but there are some other filetypes too.

Anyway I don't want to keep people waiting anymore, so here's the tool:

Rage2Unpack Download: http://bit.ly/2HO6PRl
(edit: check my post below for updated filelists to use with rage2unpack!)

Rage2Hook Release

Also made a DLL hook for dumping filenames from the game, it'll dump the filenames of any content that the game loads in.

If people could play through the game with this DLL enabled and post the rage2hook.log from it here, hopefully we can combine them all and cover more of the archives that way, like how people have been doing for RE2 Remake.

Rage2Hook Download: http://bit.ly/2VZChpx

This hook is made for the patch 1 Steam release, but might work fine with others, let me know if you have any issues with your copy.

Can you share the source code for the hook? I used dinput8.dll for other games, it didn't work this game.
I also found the hash function from the executable:

Code: Select all

// 0x107ABF0 is the relative virtual address (rva) for RAGE2.exe (update 1)
uint64_t MurmurHash3(const void * key, int64_t key_length, int32_t seed);

[infogram]

Can you share the source code for the hook? I used dinput8.dll for other games, it didn't work this game.
I also found the hash function from the executable:

Code: Select all

// 0x107ABF0 is the relative virtual address (rva) for RAGE2.exe (update 1)
uint64_t MurmurHash3(const void * key, int64_t key_length, int32_t seed);

Yep that's the same function I hook, I think 0x107ABF0 is for the earlier update though, it moved to 0x107EF10 in the latest, unless I maybe got the updates confused... (I found it at those addresses in two different EXEs at least), but 0x107EF10 is the one Rage2Hook checks & hooks.

You can have the code for it sure, it's a little messy since I put it together pretty quick though: https://pastebin.com/sfF0VFHg
Once you build it rename the dll to xinput9_1_0.dll to load it into the game, proxy.cpp handles all the exports for that.

There's a few extra things in that source that isn't in the released Rage2Hook, eg. if you have the filelists in your Rage2 folder (SteamApps\RAGE 2\projects\RAGE 2\files\archives_win64\initial\...) it'll load them all and make sure rage2hook.log won't contain any already known filenames. It also now checks for extension in the hashed string to determine if it's a path that we should log (previously it just checked for "/" character).
I'll probably release a built DLL with all that tomorrow after I test some more, and some more filelist adds too, bit late here atm

[rengareng]

Can you share the source code for the hook? I used dinput8.dll for other games, it didn't work this game.
I also found the hash function from the executable:

Code: Select all

// 0x107ABF0 is the relative virtual address (rva) for RAGE2.exe (update 1)
uint64_t MurmurHash3(const void * key, int64_t key_length, int32_t seed);

Yep that's the same function I hook, I think 0x107ABF0 is for the earlier update though, it moved to 0x107EF10 in the latest, unless I maybe got the updates confused... (I found it at those addresses in two different EXEs at least), but 0x107EF10 is the one Rage2Hook checks & hooks.

You can have the code for it sure, it's a little messy since I put it together pretty quick though: https://pastebin.com/sfF0VFHg
Once you build it rename the dll to xinput9_1_0.dll to load it into the game, proxy.cpp handles all the exports for that.

There's a few extra things in that source that isn't in the released Rage2Hook, eg. if you have the filelists in your Rage2 folder (SteamApps\RAGE 2\projects\RAGE 2\files\archives_win64\initial\...) it'll load them all and make sure rage2hook.log won't contain any already known filenames. It also now checks for extension in the hashed string to determine if it's a path that we should log (previously it just checked for "/" character).
I'll probably release a built DLL with all that tomorrow after I test some more, and some more filelist adds too, bit late here atm

It is unfortunate that they use this function for many different purposes. Only thing I can suggest for performance improvement is that using unordered_set instead of vector to lookup for a hash quickly.

[kilik]

Wow amazing genious !!

audio look fsb5 header
so we got probably the file uses encryption,

fsbext say insert the needed keyword:
type ? for viewing the hex dump of the first 176 bytes of the file because
it's possible to see part of the plain-text password in the encrypted file!

[Ciprianno]

That is awesome ,thanks for sharing with us infogram

[OrangeC]

Tried to rebuildfilenamelists tool but got install path not detected.

[infogram]

Hey all, just a small update for you guys... filelist is now 60% (182342/302871) complete

With a lot of searching and some tricks I managed to get it up to there, there's still that 40% to go though, and now I really don't have any more tricks up my sleeve to pull... so I think the only way we'll get any further is from people posting Rage2Hook logs here, really hope you guys can help out

Download filelist update 2: http://bit.ly/2QkyHAk
(delete the "projects" directory from Rage2Unpack and extract the one from this zip in its place.)

Small breakdown of the filelist: we have 89% (31712/35328) of the main non-translation-related files, and 56% (150630/267543) of the translation/voice related files.
I'm pretty sure the missing files are mostly quest/story related now though, hopefully in time we'll get those filled.

Enjoy

It is unfortunate that they use this function for many different purposes. Only thing I can suggest for performance improvement is that using unordered_set instead of vector to lookup for a hash quickly.

Yeah it's too bad the game uses it so much, was thinking of debugging it to try and narrow down which caller is actually responsible for paths, but I figured it's probably better to log the main Murmur func instead of limiting it to just one caller.

Thanks for the tip about unordered_set too, will add that into the next Rage2Hook release

Also it really sucks they order by hash in the .tab file, in some other games that use hashes they sometimes leave it sorted alphabetical, which can give us clues about some missing files, no such luck here though

edit: oh wow, turns out while .tab is ordered by hash, the data in the .arc is actually ordered alphabetically, so you can just sort the tab entries by the data offset to get an alphabetical listing of them! (as in, sorted alphabetically without needing to know the filename )

That makes working out missing items a lot easier.. for example see this pic: https://i.imgur.com/68htpqc.png
The pink ones are missing filenames, but now that it's sorted alphabetically next to the others, you can make a good guess for what the filename might be

audio look fsb5 header
so we got probably the file uses encryption,

Hmm, have you checked if there's any JC3/JC4 tools for them? If there's anything close maybe we can tweak it like we did with gibbed's tool.

That is awesome ,thanks for sharing with us infogram

No problem, hope people find it useful!

Tried to rebuildfilenamelists tool but got install path not detected.

I had the same problem too, it's an issue with the original tool afaik.
Easy to fix though, open regedit, go to "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\", make a new key called "Steam App 548570", then make a string entry inside it called InstallLocation, and set to your RAGE 2 dir.

Here's a .reg file that will do all that, make sure to edit the install path and then save it into a .reg file, then run it and allow it to import etc: https://pastebin.com/raw/KdyXYmLM