Page 1 of 20
watch dogs 2 .dat .fat archive?
Posted: Sat Dec 03, 2016 6:27 am
by redcomet
any progress?
Can't wait to mod wd2
Re: watch dogs 2 .dat .fat archive?
Posted: Sat Dec 03, 2016 10:08 am
by disastorm
I don't know anything about how to do this, but I'll post my thoughts on the fat file, maybe people who know stuff will see it and automatically know the answer.
It looks like the fat file may be best viewed at 20 bytes per line.
The first 8 bytes are always the same, probably the name and version or something like that. 35544146 0B000000
The next 4 bytes seem to have 2 variants: either 01064600 or 00004600
files with 01064600 :
common.fat
videos.fat
installpackage.fat
san_francisco_english.fat
san_francisco_hires.fat
san_francisco_preload.fat
san_francisco_cache.fat
san_francisco_cache_patch.fat
installpackage_english.fat
patch.fat
shadersobj.fat
san_francisco.fat
files with 00004600:
sound_english.fat
sound.fat
san_francisco_sound.fat
san_francisco_sound_english.fat
The 8 bytes after that are always all Fs in the initial header: FFFFFFFF FFFFFFFF
And the 4 bytes after that initial header thing is always 00000000.
Here are some fat images in case anyone has any ideas:
The fourth column in all of them seems to be constantly increasing I believe, but it doesnt start at zero and they seem too high to be offsets.
common.fat ( the second column seems to have some repeated values) :
videos.fat ( you'll notice the second column is mostly zeroes ):
sound.fat ( second column is also mostly zeroes, also note this is one of the files with 00004600 for bytes 9-12 ):
Here are some fats for 0 byte dat files:
Re: watch dogs 2 .dat .fat archive?
Posted: Sat Dec 03, 2016 10:47 pm
by iambosh
i suppose we could modify the quick bms script for fat files? i really wise we knew how to modify gibbs tools
Re: watch dogs 2 .dat .fat archive?
Posted: Tue Dec 06, 2016 10:26 am
by daemon1
i know how to modify gibbed tools, but i doubt it will help. Do you think the format is similar to Watch dogs 1?
Re: watch dogs 2 .dat .fat archive?
Posted: Tue Dec 06, 2016 4:36 pm
by redcomet
i have found rick/gibbed 's twitter,maybe someone can ask him about the disrupt tool
https://twitter.com/gibbed/with_replies
Re: watch dogs 2 .dat .fat archive?
Posted: Tue Dec 06, 2016 4:40 pm
by Ekey
Header:
Code: Select all
dwID: uint32_t; //5TAF
dwVersion: uint32_t; //11
dwUnknown1: uint32_t;
dwUnknown2: uint32_t; //-1
dwUnknown3: uint32_t; //-1
dwUnknown4: uint32_t; //0
dwTotalFiles: uint32_t;
Entry:
Code: Select all
dwHash: uint64_t; //NameHash
dwSize: uint32_t;
dwOffset: uint32_t;
dwZSize: uint32_t;
for gibbed source's
Code: Select all
var a = TFileStream.ReadInt32();
var b = TFileStream.ReadInt32();
var c = TFileStream.ReadInt32();
var d = TFileStream.ReadInt32();
var e = TFileStream.ReadInt32();
NameHash = a;
NameHash |= ((ulong)b) << 32;
Offset = (long)d << 2;
Offset |= ((c & 0xC0000000u) >> 30);
UncompressedSize = (uint)(e & 0xFFFFFFFC) >> 2;
CompressedSize = (uint)((c & 0x3FFFFFFF) >> 0);
Compression seems is LZ series
Re: watch dogs 2 .dat .fat archive?
Posted: Tue Dec 06, 2016 11:44 pm
by Sir Kane
Yeah, it uses LZMA and modified version of LZ4.
Re: watch dogs 2 .dat .fat archive?
Posted: Tue Dec 06, 2016 11:46 pm
by Ekey
Interesting.. there are 2 types of compression? Or One > LZMA + LZ4 ?
Re: watch dogs 2 .dat .fat archive?
Posted: Wed Dec 07, 2016 12:35 am
by Sir Kane
Two types.
Code: Select all
#pragma pack(push, 4)
struct SFatFileEntry{
/*uint32_t field_0;
uint32_t field_4;
uint32_t field_8;
uint32_t field_c;
uint32_t field_10;*/
uint64_t Hash;
uint64_t CompressedSize : 30;
uint64_t Offset : 34;
uint32_t CompressionMethod : 2;
uint32_t UncompressedSize : 30;
};
#pragma pack(pop)
Type 2 is the modified LZ4, 0 is LZMA.
Re: watch dogs 2 .dat .fat archive?
Posted: Wed Dec 07, 2016 12:59 am
by Ekey
Hash is Fnv1a x64
Code: Select all
static ulong gethash64(string value)
{
ulong fnv64Prime = 0x00000100000001B3ul;
ulong hash = 0xCBF29CE484222325ul;
for (var i = 0; i < value.Length; i++)
{
hash *= fnv64Prime;
hash = hash ^ value[i];
}
return hash & 0x1FFFFFFFFFFFFFFFul | 0xA000000000000000ul;
}
Re: watch dogs 2 .dat .fat archive?
Posted: Wed Dec 07, 2016 12:37 pm
by disastorm
Wow didn't expect people to continue this. I wish I could help more but not really sure what to do. It seems you guys figured out offset and file size? Is the last thing to just get the compression correct?
Re: watch dogs 2 .dat .fat archive?
Posted: Wed Dec 07, 2016 1:54 pm
by Ekey
disastorm wrote:Is the last thing to just get the compression correct?
Yup. Currently i have not found a piece code of compression in library. Maybe Sir Kane can give a some hint's
.gif "Smile")
Re: watch dogs 2 .dat .fat archive?
Posted: Wed Dec 07, 2016 2:55 pm
by disastorm
If I'm not mistaken, even after getting the compression, would we still need to figure out what the various values in the xmls are? didn't gibbed need to make some kind of binary class mapping files for the watch dogs 1 gibbed?
Re: watch dogs 2 .dat .fat archive?
Posted: Wed Dec 07, 2016 6:16 pm
by volfin
about the compression, this is the same problem Aluigi came across when unpacking another Ubisoft title, Far Cry: Primal. It also uses LZMA and an unknown LZ4. So that's something to keep in mind, perhaps some clues can be found in one game to benefit the other...
http://www.zenhax.com/viewtopic.php?f=9&t=378#p11725
Re: watch dogs 2 .dat .fat archive?
Posted: Thu Dec 08, 2016 12:10 am
by Sir Kane
0xE6C is good constant to look for to find the LZMA code (which turns out to be slightly modified, too).