RaiderZ filesystem
Posted: Mon Jan 24, 2011 3:45 pm
The second closed beta test of RaiderZ revealed a new filesystem. I wanted to take a quick look at what's new!
Note: This isn't a request or a solution, but definitely falls under the 'compression methods' category!
Previously
Previously, the MRS format was used to store files.
This same format (and version) was used in another game by this developer (MAIET) - GunZ - and uses a modified ZIP format to compress the headers. See: viewtopic.php?f=21&t=4871
MRS files were loaded from hard-coded names in the RaiderZ executable.. not anymore!
A File List!
In the installation directory is the new filesystem's file list - meet fileindex.msf!
This MSF file is the very first file read as it contains everything about the filesystem that's needed to load all the right files, and, naturally, there is some protection..
I should first say that 'fileindex.msf' is slightly different from other MSF files in that the uncompressed size is added to the MSF data like so:
When reading the file list, RaiderZ first reads the data into it's own file so uSize doesn't interfere with the MSF reading (see below).
MSF files
Each MSF has a corresponding uncompressed size value from the file list (or, with the case of filelist.msf, the value preceeding the data - see above).
The MSF data is fed through an XOR table function (pseudo-code):
The XOR table is 64 KB, which I've dumped for reference purposes. It could be dynamically calculated at runtime, but I've not followed this up!
See attachment
The next step is the decompression. Now I'm not interested in following all that as it's sure to change before release! For the purposes of exploring the file list, I just dumped the result.
A File List! Decrypted!
Now decrypted, the file list takes the following format:
Files can now be read from the filesystem using this table structure - the Pos and PackedSize values point to the size and position within the MRF file (see MRF file notes below).
MRF files
MRS files are literally just MSF files which have been joined together
The MSF files can be read off using the decrypted file list!
Once the MSF is read off the MRF package, it's processed like I've already described! That's everything!
Edits Formatting!
Note: This isn't a request or a solution, but definitely falls under the 'compression methods' category!
Previously
Previously, the MRS format was used to store files.
This same format (and version) was used in another game by this developer (MAIET) - GunZ - and uses a modified ZIP format to compress the headers. See: viewtopic.php?f=21&t=4871
MRS files were loaded from hard-coded names in the RaiderZ executable.. not anymore!
A File List!
In the installation directory is the new filesystem's file list - meet fileindex.msf!
This MSF file is the very first file read as it contains everything about the filesystem that's needed to load all the right files, and, naturally, there is some protection..
I should first say that 'fileindex.msf' is slightly different from other MSF files in that the uncompressed size is added to the MSF data like so:
Code: Select all
uint32 uSize;
uint8 Data[ .. ]
MSF files
Each MSF has a corresponding uncompressed size value from the file list (or, with the case of filelist.msf, the value preceeding the data - see above).
The MSF data is fed through an XOR table function (pseudo-code):
Code: Select all
// data != null and filesize > 0
EAX = FileSize -1;
while(EAX)
{
CL = data[EAX -1]
CL >>= 1;
EBX = EAX & 0xFFFF;
CL ^= table[EBX]
data[EAX] ^= CL
EAX--;
}
AL = data[FileSize -1]
AL >>= 1;
AL ^= table[0]
AL ^= data[0] // AL is now 0x00
data[0] = AL
See attachment
The next step is the decompression. Now I'm not interested in following all that as it's sure to change before release! For the purposes of exploring the file list, I just dumped the result.
A File List! Decrypted!
Now decrypted, the file list takes the following format:
Code: Select all
while (!EoF())
{
uint32 uSize
uint32 Pos // relative to the MRF file
uint32 PackedSize
uint16 MRFNameLen
uint16 FileNameLen
uint32 UNKNOWN
char MRFName[ MRFNameLen ]
char FileName[ FileNameLen ]
}
MRF files
MRS files are literally just MSF files which have been joined together
The MSF files can be read off using the decrypted file list!Once the MSF is read off the MRF package, it's processed like I've already described! That's everything!
Edits Formatting!
.gif "Smile"){kind=small}