XeNTaX

I don't suppose you remember the parameters for the atexreader for converting the atex files to be open-able in say photoshop or xnview, currently does nothing when I try feeding it the any of the following:
atexreader *.atex *
atexreader * *
atexreader *.atex *.png
atexreader *.atex *.tga
atexreader [filenumber].atex *

Tested on the DXT1,DXT5, and 3DCX directories.

On page 3 you find a dl by Nimba of the GWUnpacker. That one containts the ATEXReader.exe.
I don't have the time to test it all at the moment. I know it doesn't always work and I can't tell you why but when you have the atexreader in the same dir as the .atex files and you open cmd and go to the dir. Then it should work when you type:
atexreader 00000116.atex

atexreader *.atex works too to get all files

When you are let's say in cmd at c:\ and use a command like
c:\gw2\atexreader\atexreader c:\gw2\GWDat\ATEX\DXT1\00000116.atex
then it doesn't work.´

//
And .abff and .asnd are audio files too

I'm just wondering where does the game get IDs of files it needs to download. The only file it downloads prior to the actual game files contains these 5 numbers: However, the first file downloaded is 13361. I could only identify the largest number, which seems to be the size of the executable in bytes.
Those IDs are not hard-coded to the executable, either. It might be a hash, but generated from what? Any ideas?

Dolkar wrote:I'm just wondering where does the game get IDs of files it needs to download. The only file it downloads prior to the actual game files contains these 5 numbers:

Code: Select all

14120 200012 21623832 199991 2428

However, the first file downloaded is 13361. I could only identify the largest number, which seems to be the size of the executable in bytes.
Those IDs are not hard-coded to the executable, either. It might be a hash, but generated from what? Any ideas?

well thats build, exe, exesize, someManifest, manifestSize

then it takes teh build number and uses it against

/manifest/program/102/build# to get the file manifest for that build.

This was all on the 6th or 7th post in this thread.

Nimba wrote:So this is what I have found so far that could help:

/latest/102 : has the latest build # (ex. returns: 13525 2312313 12313123 ...) the first number is the build number.
/manifest/program/102/{build#} : manifest file for build number (ex. 13525, 13603)
/program/102/0/0/{fileNumber}/ : the actual data file uncompressed (ex. 844474 - MP3 File of a Roar)
/program/102/0/0/{fileNumber}/compressed : the actual data file compressed (have not investigated the compression used)

The Manifest File:

It is a list of UInt32 in Little Endian order. once converted to decimals, they are the fileNumber used in the above links. They do not have filenames, the number is the filename.
The Cookie information, and the CDN address is in the GW2Dev.exe file, should not be hard to find for those that are working on it.

Also, randomly (well seems random to me) there are numbers in the Manifest files that are not Files. They are the size in bytes of the previous file downloaded.

Example:
1814180 - 345208 bytes
2519389 - 345036 bytes
345036
2054826 - 72964 bytes
2948595 - 104384 bytes
104384

Oh, thanks! I must have missed that one...

It just bugs me why I couldn't see the manifest file being downloaded...

EDIT: The format of /manifest/program/101/14120 file seems to be different though:

unknown 4 bytes, probably CRC
after that, every 12 bytes: fileID (4 bytes), fileID (4 bytes), size(4 bytes)

Hey,

Soooo.. I thought that having a wiki to share the different file structures / encryptions / etc... would be a good idea, because it seems that a lot of people are just working on their own and everybody could benefit from shared experience. (The goal would be to centralize all infos on the DAT file, game data/sounds/textures/maps/etc...)
But with new Xentax "rules of conducts" and some encryption/file formats that are sensitive to release publicly w.r.t. ArenaNet, I think that a private wiki would be more appropriate than the Xentax one, unless Xentax could create a private one ? (It is ok for general files, but going into details on game data/disassembly is I think sensitive). I know that there is a private section on GameRevision, but they are a botting site, it's not really related to DAT RE.

So I created one where I started to fill out infos of what I currently know (this is on-going), so, everyone that thinks can contribute in any way and actually wants to share some infos with others can ask me for the URL/rights details!

Lastly, I'm sorry but people who can't program or can't come up with an idea to contribute I don't see the point of giving them access. But don't worry, with this kind of stuff I'm pretty sure that tools will be available a lot faster for you to play with ;D

Anyways,
Cheers !

mind pming me the info? ill def help out where i can

Updated unpacker so that it might be able to work with certain dat files.

Modified ATEXreader so that it can work with ATEP and ATEU files. Apparently they are the same as ATEX.

Note: ATEXreader does not support DXTA (yet).

Changelog:
v1: Support for ATEP files
v2: Support for ATEU files
v3: Support for 3DCX format

https://bitbucket.org/Daegalus/gw2re/overview

I have setup a bitbucket for all the devs working on RE to share their source through that. We are releasing open source packages as it is. It would be nice to have everything in one place so that we can increase the speed to better tools.

So xtridence, if you wish, submit your code to the repo. If you want, just the ATEX reader. I can add you directly as a contributer so you can commit directly, or you can do the pull request method. If you wish, you can add your changes to existing products.

If you don't want to interfere with anyone, make a copy of their project, and make changes to it under a modified folder name.

As for others, this is for developers and its only source code, you won't be able to download binaries.

Nimba wrote:https://bitbucket.org/Daegalus/gw2re/overview

I have setup a bitbucket for all the devs working on RE to share their source through that. We are releasing open source packages as it is. It would be nice to have everything in one place so that we can increase the speed to better tools.

So xtridence, if you wish, submit your code to the repo. If you want, just the ATEX reader. I can add you directly as a contributer so you can commit directly, or you can do the pull request method. If you wish, you can add your changes to existing products.

If you don't want to interfere with anyone, make a copy of their project, and make changes to it under a modified folder name.

As for others, this is for developers and its only source code, you won't be able to download binaries.

Thank you for your work on this. The Gw2DatUnpacker works great!

Knows anyone what is format that used for string files?

Gornoth wrote:Thank you for your work on this. The Gw2DatUnpacker works great!

Glad you like it!

Edit: I updated it yesterday btw, so if it crashed for you before, it should no longer do that. Instead it should just skip the files it could not read.

xtridence wrote:Modified ATEXreader so that it can work with ATEP and ATEU files. Apparently they are the same as ATEX.

In the exe, it checks for a couple more FourCC codes for textures. They're don't seem to be in use yet, but they're likely to be at some point. For instance, they changed most of the UI textures from ATEX to ATEU, between the build before the BWE and the BWE build.

In the current build they're in the function located at 0x701900. All of them are: ATEX, ATTX, ATEC, ATEP, ATEU, ATET.

Edit: Are you sure the channels in the 3DC files are correct and don't need to be moved around/inverted? It's *very* rare for [0,0.5,0.5] to be used for flat surfaces. Usually it's [0.5,0.5,1] or rarely [0.5,1,0.5].

Edit 2: I would guess the swapping of red/blue that happens after the switch should not happen for 3DC textures, and that their neutral state is [0.5,0.5,0]. It would make a whole lot more sense than [0,0.5,0.5] (unless they use the X-axis as the vertical axis and negative X means up). Personally, to get them usable to me, I changed it to not swap red and blue, and instead I swap red and green, and invert red and blue (after the swap). This makes it work like it should in the engine I'm currently trying out things in. For some textures I seem to have to invert green as well so it might be that all of them are inverted. I'll have to check with more textures to see which one is more usual.

Edit 3: You definitely need to add 'free(img)' in the if-case for FourCC and the if-case for compression format, before the returns. And also in the default case in the switch (together with free(output)). Doing ATexReader *.atex in a folder with 19k files made me run out of memory quite fast due to the DXTA files.

Thanks for spotting those bugs. I was wondering why the 3DC colors seems a little off (it's because the RGBA struct seemed to have swapped the red and blue for some reason).

I've implemented all of your suggestions (except for the part on compression because there was no buffer to free before the return statement).

(I've put this build in a separate post just in case I broke the tool when I did some adjustments.)

Changelog
v4: 3DC bugfix, plugged a few memory leaks
v5: Bugfixes

You now have another bug on your hands instead. You cannot do 'delete[]' on a variable that was allocated with 'malloc'. Malloc'ed chunks of memory need to be freed using free(). I'm looking at the 'delete[] img' you added.