[Request] Guild Wars 2 DAT

[BobSaget]

Well now its slightly happening because of load but a few tries will get it working again though.

[Loumie]

Does anyone know if the client is always making a query to retrieve it at boot ?

If the build matches the latest query, it just launches the game and doesn't look at either manifest.

Yep, just saw that this morning after the update was finished. I'll have to rescan my DAT then ;D

I'm looking at the txtm file for now, to see how they map the strs files~

The strs files themselves are actually pretty interesting. Using the same format as the original game.

I didn't spend much time on the strs files, but I didn't have time to look at the ones that seems maybe encrypted of some sort, maybe that's what you're talking about because I didn't see anything special. Just a BOM, a size, a string, and other values that are here when it is not in "clear".

Cheers.

[Nimba]

Loumie, this might help you, but I was looking for where GW2 stored its settings and found a .DAT file in My Documents/Guild Wars 2. Then I noticed the Size, it was 1.2 megs. I deleted it, and for sure, I got my settings reset. But I was curious why it was so big. I backed up my old one before deleting which I think had some data from an older beta.

But anyways. I noticed in the hex editor, it had the same header as the GW2 Dat file. So I made the GW2DatBrowser point to it, and viola. A small DAT file with Manifests. Just manifests. At the very end of one, I saw a loclpref file, which i think had settings, but the new ones GW2 generated, didnt have this file. Which I am assuming is something left over from the old client.

But I do know somewhere in this local.dat it stores settings. There are quite a few files that GW2Dat doesnt even recognize and doesn't let me view them. But this could help you

Maybe your update GW2DatBrowser you mentioned on GameRevision can get you more data.

[Loumie]

Hey,

In fact, it seems that they are moving things around between betas, before this new beta I spotted the file you are talking about, the loclpref one, actually there are 4 files that seem special (apart from the ones that are 4097 / 4098 /etc..):

- Ids were: 2 / 3 / 12 / 13

There was:
- the loclpref that you talk about
- a file starting with dibw that seems to be like a CD Key or a serial
- the two remaining ones I don't remember

BUT, indeed now, a new file appeared the Local.dat one, I didn't look at it yet, but it seems interesting given what you found
I'll run my parser on it to check all the files inside.

But, what I discovered is pretty funny, actually by rewritting the pseudo code that rafi did into a more correct one, I'm beginning to think that it is a simple zlib compression, as the description of what the algorithm of zlib does is exactly what the decompression algorithm is doing to extract the files, the only difference could be the initial static data. I'm more focused on this right now, if I find anything I'll let you know.

Edit: So I parsed this Local.dat and it is indeed what I was looking for, the 198828 is the main manifest, then there are 42 other manifests for the 42 sub categories, then there is the gw2.exe(198831) number and the localpref(3)

[JimboJoe]

I'm willing to pay for a working dat extractor, which at least extracts text-information (most important) and graphics from the game. Feel free to send me a pm.

[Dolkar]

What about sniffing out the location of in-game files and downloading them uncompressed without using the official downloader? That might be much easier than unpacking them from the .dat file.

The first thing that is probably getting downloaded is the list of the files and their locations (possibly even names?). I'll try to sniff that one out when I have some time. Or has anyone already successfully done that?

[Gornoth]

Here is a dll that decompresses the guild wars 2 .dat.
It was created by Loumie at gamerevision.

Hey,

So, here is a dll, with the sources, which contain a simple function "inflate" (in gw2Inflate.h) that decompresses a file.
I'm using codeblocks/GCC because the C compiler of VS is not even compliant with C99 (unless I missed something).

It successfully decompresses the files of the last version with about 50~ fails out of 187919 files. These fails seems to happen at the end of the decoding, usually the last bytes in the files, I don't have time to look at it right now though.

Feel free to point at eventual mistakes, I'm more a C++ programmer than a pure C one ;D.

Cheers.

[Nimba]

What about sniffing out the location of in-game files and downloading them uncompressed without using the official downloader? That might be much easier than unpacking them from the .dat file.

The first thing that is probably getting downloaded is the list of the files and their locations (possibly even names?). I'll try to sniff that one out when I have some time. Or has anyone already successfully done that?

That is kinda what we have been doing all along. THere is no way atm to find the real file-name. At least we haven't found any correlation tables. But if we need specific files, we find their number, and download them.

Another thing is, GW2DatBrowser works, stuff can be extracted with that.

Here is a dll that decompresses the guild wars 2 .dat.
It was created by Loumie at gamerevision.

Hey,

So, here is a dll, with the sources, which contain a simple function "inflate" (in gw2Inflate.h) that decompresses a file.
I'm using codeblocks/GCC because the C compiler of VS is not even compliant with C99 (unless I missed something).

It successfully decompresses the files of the last version with about 50~ fails out of 187919 files. These fails seems to happen at the end of the decoding, usually the last bytes in the files, I don't have time to look at it right now though.

Feel free to point at eventual mistakes, I'm more a C++ programmer than a pure C one ;D.

Cheers.

Thank you for reposting this. I was trying to find it for a while after I missed it.

[Neoryudo]

been trying to use the GW2datbroswer, but am finding it crashes every time at 78%, anyone else finding this?

[Loumie]

Yeah, I edited out the link on gamerevision when I got fed up to only receive PMs from people wanting a full extractor program.
Anyway, I would have given it to you if you had asked me Nimba, you've shared plenty enough ;D

Cheers.

[Nimba]

Yeah, I edited out the link on gamerevision when I got fed up to only receive PMs from people wanting a full extractor program.
Anyway, I would have given it to you if you had asked me Nimba, you've shared plenty enough ;D

Cheers.

I got it from someone else that got it, but I did message you on GameRevision for the DLL (I am Daegalus there) but its cool No harm and I understand the reason. But the DLL works great.

[xtridence]

Thanks Loumie for providing the decompression algorithm. It works well with GWUnpacker (with few minor adjustments to make it C++ compatible).

Changelog:
v0: Nimba's modified GWUnpacker
v1: Ported Loumie's decompression algorithm to GWUnpacker
v2: Implemented ral's suggestions with regards to sorting certain files types.
v3: Some minor bugfixes related to ATEP files
v4: Added support for "newer" versions of dat files

[ral]

Thank you

And to sort the files a bit more and to have some less .raw files just add the following switch/case in the GWUnpacker.cpp from xtridence.
Hope the PF one is not a mess from programming style but I'm not a c++ developer and it works like that so I didn't care about it.

Add this one to case 'XETA':

Code: Select all

case 'XCD3':
	CreateDirectory("GWDat/ATEX/3DCX",NULL);
	printf(" 3DCX");
	sprintf(newname,"GWDat/ATEX/3DCX/%s.atex\0",Name);
	break;

And the rest in the switch (i):

Code: Select all

case 'UETA':
	CreateDirectory("GWDat/ATEU",NULL);
	printf(" ATEU");
	switch (k)
	{
	case '1TXD':
		CreateDirectory("GWDat/ATEU/DXT1",NULL);
		printf(" DXT1");
		sprintf(newname,"GWDat/ATEU/DXT1/%s.ateu\0",Name);
		break;
	case '2TXD':
		CreateDirectory("GWDat/ATEU/DXT2",NULL);
		printf(" DXT2");
		sprintf(newname,"GWDat/ATEU/DXT2/%s.ateu\0",Name);
		break;
	case '3TXD':
		CreateDirectory("GWDat/ATEU/DXT3",NULL);
		printf(" DXT3");
		sprintf(newname,"GWDat/ATEU/DXT3/%s.ateu\0",Name);
		break;
	break;
	case '5TXD':
		CreateDirectory("GWDat/ATEU/DXT5",NULL);
		printf(" DXT5");
		sprintf(newname,"GWDat/ATEU/DXT5/%s.ateu\0",Name);
		break;
	break;
	case 'LTXD':
		CreateDirectory("GWDat/ATEU/DXTL",NULL);
		printf(" DXTL");
		sprintf(newname,"GWDat/ATEU/DXTL/%s.ateu\0",Name);
		break;
	}
	break;
case 'PETA':
	CreateDirectory("GWDat/ATEP",NULL);
	printf(" ATEP");
	switch (k)
	{
	case '1TXD':
		CreateDirectory("GWDat/ATEP/DXT1",NULL);
		printf(" DXT1");
		sprintf(newname,"GWDat/ATEP/DXT1/%s.atep\0",Name);
		break;
	case '5TXD':
		CreateDirectory("GWDat/ATEP/DXT5",NULL);
		printf(" DXT5");
		sprintf(newname,"GWDat/ATEP/DXT5/%s.atep\0",Name);
		break;
	}
	break;
case 'ffba':
	CreateDirectory("GWDat/abff",NULL);
	printf(" abff");
	sprintf(newname,"GWDat/abff/%s.abff\0",Name);
	break;
	// FP
case 83536:
	CreateDirectory("GWDat/PF",NULL);
	char type[9];
	for (int z=8; z<16;z++)
		type[z-8] = Output[z];
	type[8] = '\0';
	printf(" PF %s", type);
	char dirname[18];
	sprintf(dirname, "GWDat/PF/%s\0", type);
	if (strcmp("ABNKBKCK", type) == 0)
	{
		CreateDirectory(dirname, NULL);
		sprintf(newname,"GWDat/PF/%s/%s.mp3\0",type, Name);
	}
	else
	{
		CreateDirectory(dirname, NULL);
		sprintf(newname,"GWDat/PF/%s/%s.pf\0",type, Name);
	}
	break;
case 'dnsa':
	CreateDirectory("GWDat/asnd",NULL);
	printf(" asnd");
	sprintf(newname,"GWDat/asnd/%s.asnd\0",Name);
	break;
case 'srts':
	CreateDirectory("GWDat/strs",NULL);
	printf(" strs");
	sprintf(newname,"GWDat/strs/%s.strs\0",Name);
	break;

Hope I didn't miss any files. And I didn't know any of the files (except the audio because of the Xing Header) so I just gave them the names I saw in the header.

EDIT:// Edit was because I forgot making ATEP and ATEU dir :/ Sorry

[xtridence]

I've updated the unpacker to sort certain file types. Thank you ral for the suggestion.

[ral]

Thanks for adding.
Oh and the PF - ASNDASND has 2 different kind of audio files I just noticed.
Some with Xing Header and some with Ogg header.
This should work to identify the different filetypes:

Code: Select all

if (strcmp("ASNDASND", type) == 0)
{
	char oggOrNot[5];
	for (int z=92; z<96;z++)
		oggOrNot[z-92] = Output[z];
	oggOrNot[4] = '\0';
	CreateDirectory(dirname, NULL);
	if (strcmp(oggOrNot, "OggS") == 0)
		sprintf(newname,"GWDat/PF/%s/%s.ogg\0",type, Name);
	else
		sprintf(newname,"GWDat/PF/%s/%s.mp3\0",type, Name);
}

And I'm sorry, in my code above I forgot the CreateDirectory for ATEP and ATEU so you won't get the files if the dir doesn't exist :/