watch dogs 2 .dat .fat archive?

[redcomet]

any progress?
Can't wait to mod wd2

[disastorm]

I don't know anything about how to do this, but I'll post my thoughts on the fat file, maybe people who know stuff will see it and automatically know the answer.

It looks like the fat file may be best viewed at 20 bytes per line.

The first 8 bytes are always the same, probably the name and version or something like that. 35544146 0B000000

The next 4 bytes seem to have 2 variants: either 01064600 or 00004600

files with 01064600 :
common.fat
videos.fat
installpackage.fat
san_francisco_english.fat
san_francisco_hires.fat
san_francisco_preload.fat
san_francisco_cache.fat
san_francisco_cache_patch.fat
installpackage_english.fat
patch.fat
shadersobj.fat
san_francisco.fat

files with 00004600:
sound_english.fat
sound.fat
san_francisco_sound.fat
san_francisco_sound_english.fat


The 8 bytes after that are always all Fs in the initial header: FFFFFFFF FFFFFFFF
And the 4 bytes after that initial header thing is always 00000000.


Here are some fat images in case anyone has any ideas:
The fourth column in all of them seems to be constantly increasing I believe, but it doesnt start at zero and they seem too high to be offsets.

common.fat ( the second column seems to have some repeated values) :


videos.fat ( you'll notice the second column is mostly zeroes ):


sound.fat ( second column is also mostly zeroes, also note this is one of the files with 00004600 for bytes 9-12 ):



Here are some fats for 0 byte dat files:

[iambosh]

i suppose we could modify the quick bms script for fat files? i really wise we knew how to modify gibbs tools

[daemon1]

i know how to modify gibbed tools, but i doubt it will help. Do you think the format is similar to Watch dogs 1?

[redcomet]

i have found rick/gibbed 's twitter,maybe someone can ask him about the disrupt tool
https://twitter.com/gibbed/with_replies

[Ekey]

Header:

Code: Select all

    dwID: uint32_t; //5TAF
    dwVersion: uint32_t; //11
    dwUnknown1: uint32_t;
    dwUnknown2: uint32_t; //-1
    dwUnknown3: uint32_t; //-1
    dwUnknown4: uint32_t; //0
    dwTotalFiles: uint32_t;

Entry:

Code: Select all

    dwHash: uint64_t; //NameHash
    dwSize: uint32_t;
    dwOffset: uint32_t;
    dwZSize: uint32_t;

for gibbed source's

Code: Select all

           var a = TFileStream.ReadInt32();
           var b = TFileStream.ReadInt32();
           var c = TFileStream.ReadInt32();
           var d = TFileStream.ReadInt32();
           var e = TFileStream.ReadInt32();

           NameHash = a;
           NameHash |= ((ulong)b) << 32;
           Offset = (long)d << 2;
           Offset |= ((c & 0xC0000000u) >> 30);
           UncompressedSize = (uint)(e & 0xFFFFFFFC) >> 2;
           CompressedSize = (uint)((c & 0x3FFFFFFF) >> 0);

Compression seems is LZ series

[Sir Kane]

Yeah, it uses LZMA and modified version of LZ4.

[Ekey]

Interesting.. there are 2 types of compression? Or One > LZMA + LZ4 ?

[Sir Kane]

Two types.

Code: Select all

#pragma pack(push, 4)
struct SFatFileEntry{
	/*uint32_t	field_0;
	uint32_t	field_4;
	uint32_t	field_8;
	uint32_t	field_c;
	uint32_t	field_10;*/
	uint64_t	Hash;
	uint64_t	CompressedSize : 30;
	uint64_t	Offset : 34;
	uint32_t	CompressionMethod : 2;
	uint32_t	UncompressedSize : 30;
};
#pragma pack(pop)

Type 2 is the modified LZ4, 0 is LZMA.

[Ekey]

Hash is Fnv1a x64

Code: Select all

static ulong gethash64(string value)
{
    ulong fnv64Prime = 0x00000100000001B3ul;
    ulong hash = 0xCBF29CE484222325ul;

    for (var i = 0; i < value.Length; i++)
    {
        hash *= fnv64Prime;
        hash = hash ^ value[i];
    }

    return hash & 0x1FFFFFFFFFFFFFFFul | 0xA000000000000000ul;
}

[disastorm]

Wow didn't expect people to continue this. I wish I could help more but not really sure what to do. It seems you guys figured out offset and file size? Is the last thing to just get the compression correct?

[Ekey]

Is the last thing to just get the compression correct?

Yup. Currently i have not found a piece code of compression in library. Maybe Sir Kane can give a some hint's

[disastorm]

If I'm not mistaken, even after getting the compression, would we still need to figure out what the various values in the xmls are? didn't gibbed need to make some kind of binary class mapping files for the watch dogs 1 gibbed?

[volfin]

about the compression, this is the same problem Aluigi came across when unpacking another Ubisoft title, Far Cry: Primal. It also uses LZMA and an unknown LZ4. So that's something to keep in mind, perhaps some clues can be found in one game to benefit the other...

http://www.zenhax.com/viewtopic.php?f=9&t=378#p11725

[Sir Kane]

0xE6C is good constant to look for to find the LZMA code (which turns out to be slightly modified, too).