[Request] Guild Wars 2 DAT

[Nimba]

Has anyone modified the gw unpacker to work with gw2stream.dat?

Yes and no, I did a bit of modifying, but there seems to be more than just modifying the header. The thing hits the first file, and bugs out, then errors out on the 2nd file. It seems that either the MFT Header has gotten more fields, the MFT Entry has more fields, or both. Since for some reason when it reads the first file, it wants to allocate 330 megs of space for the output file....

Here is my source of what I have so far. All I did was update the headers, a few of the checks, and so on, nothing special. I put it into its own folder. I use Visual Studio 2010 and it upgraded the SLN file from the original. The original is still there as sln.orig, so if you want to use that you can.

All the original work, sources, and everything is still present form the GW1 download.

I tried the same thing with GWDatBrowser, and it broke in the exact same place after I modified the headers. Maybe I am doing this wrong. Its the same code. GWDatBrowser pretty much directly uses GWUnpacker in its code. with a bit of modification to non-crucial areas.

[blinky57]

hello guys,

so i've been reading through this topic and while most of it is going over my head, i was wondering if its possible to extract 3d models from the .dat?

[Nimba]

hello guys,

so i've been reading through this topic and while most of it is going over my head, i was wondering if its possible to extract 3d models from the .dat?

Extracting anything isn't possible yet, unless someone somewhere has made progress on getting the files to extract. But you can download the files directly uncompressed. and I know i haev a lot of MODL files downloaded. So yes, you can get the models.

[blinky57]

ok part two. if you already have the files, any way i could bum the male sareph armor model off you?

thanks

[Nimba]

ok part two. if you already have the files, any way i could bum the male sareph armor model off you?

thanks

Sure, if i knew which file it was, or how to open these. I have no real clue what the file format is, nor are tehy named or soerted in any way to figure out the models. they are all just numbers all lumped together.

[Nimba]

I am curious if anyone has managed to figure out what the 2nd file in the LATEST query is. it goes build exeNum exeFileSize WEIRDFILE WEIRDFILESIZE

The file has map names, languages and this at the bottom:

Code: Select all

buildId.manifests.....\...............S...............J...............?...............5...............*...........baseId.fileId.size.flags.name.PackAssetManifestFileV0.PackAssetRootManifestV0.......v.......FOOT

[nicoli_s]

what file # is that?

[nicoli_s]

and can you post that file? it looks very similar to file #13, except mine ends with
buildId.records.....<...............3...............*...........................baseId.fileId.size.PackAssetManifestRecordV0.PackAssetManifestV0................FOOT

and has no plain text map names or anything

[Phi La Ke Terre]

For those who try to re-use the GWunpacker/GWDatBrowser : the decompression function used is different. Can can look at the function at GW2Demo.exe 0x7CD130, I think it is the good one, (but I don't have any clue on how I can use it). I can disassemble it, but since I don't understand assembler, I can't do much.

Code: Select all

.text:007CD130 ; ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦ S U B R O U T I N E ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦
.text:007CD130
.text:007CD130 ; Attributes: bp-based frame
.text:007CD130
.text:007CD130 sub_7CD130      proc near               ; CODE XREF: sub_7CC4C0+148p
.text:007CD130
.text:007CD130 var_1270        = dword ptr -1270h
.text:007CD130 var_94C         = dword ptr -94Ch
.text:007CD130 var_940         = dword ptr -940h
.text:007CD130 var_1C          = dword ptr -1Ch
.text:007CD130 var_18          = dword ptr -18h
.text:007CD130 var_14          = dword ptr -14h
.text:007CD130 var_10          = dword ptr -10h
.text:007CD130 var_C           = dword ptr -0Ch
.text:007CD130 var_8           = dword ptr -8
.text:007CD130 var_4           = dword ptr -4
.text:007CD130 arg_0           = dword ptr  8
.text:007CD130
.text:007CD130                 push    ebp
.text:007CD131                 mov     ebp, esp
.text:007CD133                 mov     eax, 1270h
.text:007CD138                 call    call_1
.text:007CD13D                 push    ebx
.text:007CD13E                 push    esi
.text:007CD13F                 mov     esi, [ebp+arg_0]
.text:007CD142                 mov     eax, [esi+0Ch]
.text:007CD145                 push    edi
.text:007CD146                 mov     [ebp+var_4], edx
.text:007CD149                 mov     edx, [esi+10h]
.text:007CD14C                 mov     ebx, eax
.text:007CD14E                 mov     edi, edx
.text:007CD150                 shl     eax, 4
.text:007CD153                 shr     edi, 1Ch
.text:007CD156                 or      edi, eax
.text:007CD158                 mov     eax, [esi+8]
.text:007CD15B                 shr     ebx, 1Ch
.text:007CD15E                 mov     [ebp+var_8], ecx
.text:007CD161                 mov     [esi+0Ch], edi
.text:007CD164                 cmp     eax, 4
.text:007CD167                 jb      short loc_1
.text:007CD169                 shl     edx, 4
.text:007CD16C                 mov     [esi+10h], edx
.text:007CD16F                 add     eax, 0FFFFFFFCh
.text:007CD172                 jmp     short loc_2
.text:007CD174 ; ---------------------------------------------------------------------------
.text:007CD174
.text:007CD174 loc_1:                             ; CODE XREF: sub_7CD130+37j
.text:007CD174                 mov     edx, [esi]
.text:007CD176                 cmp     edx, [esi+4]
.text:007CD179                 jz      short loc_7CD1A3
.text:007CD17B                 mov     edi, [edx]
.text:007CD17D                 add     edx, 4
.text:007CD180                 lea     ecx, [eax+1Ch]
.text:007CD183                 mov     [esi], edx
.text:007CD185                 mov     edx, edi
.text:007CD187                 shr     edx, cl
.text:007CD189                 mov     [ebp+arg_0], ecx
.text:007CD18C                 mov     ecx, 4
.text:007CD191                 sub     ecx, eax
.text:007CD193                 or      [esi+0Ch], edx
.text:007CD196                 mov     eax, [ebp+arg_0]
.text:007CD199                 shl     edi, cl
.text:007CD19B                 mov     ecx, [ebp+var_8]
.text:007CD19E                 mov     [esi+10h], edi
.text:007CD1A1                 jmp     short loc_2
.text:007CD1A3 ; ---------------------------------------------------------------------------
.text:007CD1A3
.text:007CD1A3 loc_7CD1A3:                             ; CODE XREF: sub_7CD130+49j
.text:007CD1A3                 xor     eax, eax
.text:007CD1A5                 mov     [esi+10h], eax
.text:007CD1A8
.text:007CD1A8 loc_2:                             ; CODE XREF: sub_7CD130+42j
.text:007CD1A8                                         ; sub_7CD130+71j
.text:007CD1A8                 mov     edx, [ebp+var_4]
.text:007CD1AB                 inc     ebx
.text:007CD1AC                 mov     [esi+8], eax
.text:007CD1AF                 lea     eax, [ecx+edx]
.text:007CD1B2                 mov     [ebp+var_10], ebx
.text:007CD1B5                 mov     ebx, ecx
.text:007CD1B7                 mov     [ebp+var_4], eax
.text:007CD1BA                 cmp     ecx, eax
.text:007CD1BC                 jz      loc_7CD2F1
.text:007CD1C2
.text:007CD1C2 loc_7CD1C2:                             ; CODE XREF: sub_7CD130+1BBj
.text:007CD1C2                 push    esi
.text:007CD1C3                 lea     ecx, [ebp+var_1270]
.text:007CD1C9                 call    sub_7D4010
.text:007CD1CE                 push    esi
.text:007CD1CF                 lea     ecx, [ebp+var_940]
.text:007CD1D5                 call    sub_7D4010
.text:007CD1DA                 mov     eax, [esi+0Ch]
.text:007CD1DD                 mov     ecx, [esi+10h]
.text:007CD1E0                 mov     edi, eax
.text:007CD1E2                 mov     edx, ecx
.text:007CD1E4                 shr     edx, 1Ch
.text:007CD1E7                 shl     eax, 4
.text:007CD1EA                 or      edx, eax
.text:007CD1EC                 mov     [esi+0Ch], edx
.text:007CD1EF                 mov     edx, [esi+8]
.text:007CD1F2                 shr     edi, 1Ch
.text:007CD1F5                 mov     [ebp+var_C], edi
.text:007CD1F8                 cmp     edx, 4
.text:007CD1FB                 jb      short loc_7CD20B
.text:007CD1FD                 shl     ecx, 4
.text:007CD200                 add     edx, 0FFFFFFFCh
.text:007CD203                 mov     [esi+10h], ecx
.text:007CD206                 mov     [esi+8], edx
.text:007CD209                 jmp     short loc_7CD242
.text:007CD20B ; ---------------------------------------------------------------------------
.text:007CD20B
.text:007CD20B loc_7CD20B:                             ; CODE XREF: sub_7CD130+CBj
.text:007CD20B                 mov     ecx, [esi]
.text:007CD20D                 cmp     ecx, [esi+4]
.text:007CD210                 jz      short loc_7CD23A
.text:007CD212                 mov     eax, [ecx]
.text:007CD214                 add     ecx, 4
.text:007CD217                 mov     [esi], ecx
.text:007CD219                 lea     ecx, [edx+1Ch]
.text:007CD21C                 mov     edi, eax
.text:007CD21E                 shr     edi, cl
.text:007CD220                 mov     [ebp+arg_0], ecx
.text:007CD223                 mov     ecx, 4
.text:007CD228                 sub     ecx, edx
.text:007CD22A                 or      [esi+0Ch], edi
.text:007CD22D                 mov     edi, [ebp+var_C]
.text:007CD230                 shl     eax, cl
.text:007CD232                 mov     [esi+10h], eax
.text:007CD235                 mov     eax, [ebp+arg_0]
.text:007CD238                 jmp     short loc_7CD23F
.text:007CD23A ; ---------------------------------------------------------------------------
.text:007CD23A
.text:007CD23A loc_7CD23A:                             ; CODE XREF: sub_7CD130+E0j
.text:007CD23A                 xor     eax, eax
.text:007CD23C                 mov     [esi+10h], eax
.text:007CD23F
.text:007CD23F loc_7CD23F:                             ; CODE XREF: sub_7CD130+108j
.text:007CD23F                 mov     [esi+8], eax
.text:007CD242
.text:007CD242 loc_7CD242:                             ; CODE XREF: sub_7CD130+D9j
.text:007CD242                 inc     edi
.text:007CD243                 shl     edi, 0Ch
.text:007CD246                 test    edi, edi
.text:007CD248                 jz      short loc_7CD2C2
.text:007CD24A                 lea     ebx, [ebx+0]
.text:007CD250
.text:007CD250 loc_7CD250:                             ; CODE XREF: sub_7CD130+190j
.text:007CD250                 dec     edi
.text:007CD251                 mov     [ebp+arg_0], edi
.text:007CD254                 cmp     ebx, [ebp+var_4]
.text:007CD257                 jz      short loc_7CD2C2
.text:007CD259                 push    esi
.text:007CD25A                 lea     ecx, [ebp+var_1270]
.text:007CD260                 call    sub_7CCA10
.text:007CD265                 cmp     eax, 100h
.text:007CD26A                 jb      loc_7CD2FF
.text:007CD270                 lea     ecx, [eax-100h]
.text:007CD276                 call    sub_7CCDB0
.text:007CD27B                 mov     edi, eax
.text:007CD27D                 add     edi, [ebp+var_10]
.text:007CD280                 push    esi
.text:007CD281                 lea     ecx, [ebp+var_940]
.text:007CD287                 call    sub_7CCA10
.text:007CD28C                 mov     ecx, eax
.text:007CD28E                 call    sub_7CCC10
.text:007CD293                 lea     ecx, [edi+ebx]
.text:007CD296                 cmp     ecx, [ebp+var_4]
.text:007CD299                 ja      short loc_7CD304
.text:007CD29B                 mov     edx, ebx
.text:007CD29D                 sub     edx, [ebp+var_8]
.text:007CD2A0                 cmp     eax, edx
.text:007CD2A2                 jge     short loc_7CD304
.text:007CD2A4                 mov     ecx, ebx
.text:007CD2A6                 sub     ecx, eax
.text:007CD2A8                 dec     ecx
.text:007CD2A9                 test    edi, edi
.text:007CD2AB                 jz      short loc_7CD2BB
.text:007CD2AD                 lea     ecx, [ecx+0]
.text:007CD2B0
.text:007CD2B0 loc_7CD2B0:                             ; CODE XREF: sub_7CD130+189j
.text:007CD2B0                 mov     al, [ecx]
.text:007CD2B2                 dec     edi
.text:007CD2B3                 mov     [ebx], al
.text:007CD2B5                 inc     ebx
.text:007CD2B6                 inc     ecx
.text:007CD2B7                 test    edi, edi
.text:007CD2B9                 jnz     short loc_7CD2B0
.text:007CD2BB
.text:007CD2BB loc_7CD2BB:                             ; CODE XREF: sub_7CD130+17Bj
.text:007CD2BB                 mov     edi, [ebp+arg_0]
.text:007CD2BE
.text:007CD2BE loc_7CD2BE:                             ; CODE XREF: sub_7CD130+1D2j
.text:007CD2BE                 test    edi, edi
.text:007CD2C0                 jnz     short loc_7CD250
.text:007CD2C2
.text:007CD2C2 loc_7CD2C2:                             ; CODE XREF: sub_7CD130+118j
.text:007CD2C2                                         ; sub_7CD130+127j
.text:007CD2C2                 mov     ecx, [ebp+var_1C]
.text:007CD2C5                 test    ecx, ecx
.text:007CD2C7                 jz      short loc_7CD2CE
.text:007CD2C9                 call    sub_5E4D80
.text:007CD2CE
.text:007CD2CE loc_7CD2CE:                             ; CODE XREF: sub_7CD130+197j
.text:007CD2CE                 mov     ecx, [ebp+var_94C]
.text:007CD2D4                 xor     eax, eax
.text:007CD2D6                 mov     [ebp+var_1C], eax
.text:007CD2D9                 mov     [ebp+var_14], eax
.text:007CD2DC                 mov     [ebp+var_18], eax
.text:007CD2DF                 cmp     ecx, eax
.text:007CD2E1                 jz      short loc_7CD2E8
.text:007CD2E3                 call    sub_5E4D80
.text:007CD2E8
.text:007CD2E8 loc_7CD2E8:                             ; CODE XREF: sub_7CD130+1B1j
.text:007CD2E8                 cmp     ebx, [ebp+var_4]
.text:007CD2EB                 jnz     loc_7CD1C2
.text:007CD2F1
.text:007CD2F1 loc_7CD2F1:                             ; CODE XREF: sub_7CD130+8Cj
.text:007CD2F1                 pop     edi
.text:007CD2F2                 pop     esi
.text:007CD2F3                 mov     eax, 1
.text:007CD2F8                 pop     ebx
.text:007CD2F9                 mov     esp, ebp
.text:007CD2FB                 pop     ebp
.text:007CD2FC                 retn    4
.text:007CD2FF ; ---------------------------------------------------------------------------
.text:007CD2FF
.text:007CD2FF loc_7CD2FF:                             ; CODE XREF: sub_7CD130+13Aj
.text:007CD2FF                 mov     [ebx], al
.text:007CD301                 inc     ebx
.text:007CD302                 jmp     short loc_7CD2BE
.text:007CD304 ; ---------------------------------------------------------------------------
.text:007CD304
.text:007CD304 loc_7CD304:                             ; CODE XREF: sub_7CD130+169j
.text:007CD304                                         ; sub_7CD130+172j
.text:007CD304                 mov     ecx, [ebp+var_1C]
.text:007CD307                 xor     esi, esi
.text:007CD309                 cmp     ecx, esi
.text:007CD30B                 jz      short loc_7CD312
.text:007CD30D                 call    sub_5E4D80
.text:007CD312
.text:007CD312 loc_7CD312:                             ; CODE XREF: sub_7CD130+1DBj
.text:007CD312                 mov     ecx, [ebp+var_94C]
.text:007CD318                 mov     [ebp+var_1C], esi
.text:007CD31B                 mov     [ebp+var_14], esi
.text:007CD31E                 mov     [ebp+var_18], esi
.text:007CD321                 cmp     ecx, esi
.text:007CD323                 jz      short loc_7CD32A
.text:007CD325                 call    sub_5E4D80
.text:007CD32A
.text:007CD32A loc_7CD32A:                             ; CODE XREF: sub_7CD130+1F3j
.text:007CD32A                 pop     edi
.text:007CD32B                 pop     esi
.text:007CD32C                 xor     eax, eax
.text:007CD32E                 pop     ebx
.text:007CD32F                 mov     esp, ebp
.text:007CD331                 pop     ebp
.text:007CD332                 retn    4
.text:007CD332 sub_7CD130      endp

Sorry for the big code, but it seems I'm not allowed to attach the file.

[Nimba]

and can you post that file? it looks very similar to file #13, except mine ends with
buildId.records.....<...............3...............*...........................baseId.fileId.size.PackAssetManifestRecordV0.PackAssetManifestV0................FOOT

and has no plain text map names or anything

Its in latest, but I'll pull the most recent ones:

Code: Select all

101: Build=13785 Patcher=137873  PatcherFileSize=21119536 NewFile=137870 NewFileSize=2016
102: Build=13793 Patcher=3055547 PatcherFileSize=21993320 NewFile=3055544 NewFileSize=6808

I have attached the 2 files in a zip. They are labeled accordingly.

[nicoli_s]

thx for posting those nimba. ive been working on gw2 since i got a hold of the last beta test client, i basically went the same route as other people suggested, and i downloaded all the files uncompressed, and i have an app that does some categorization and displays them, so i can see textures, strings etc. Seems they changed very little from gw1, as i was able to reuse the ATEXReader posted on this forum, and was able to figure out the string data pretty quickly. One thing that is weird is that im not seeing that file you posted inside the .dat files, unless im missing something. But my format guesser finds others that have a similar header to this file, so either its an error on my part, or that file is only used in the actual patching, and not stored in the dat

[Nimba]

Ya, I was surprised the textures just needed ATEX, figured out it works a day or 2 ago. We havent gotten around to doing any processing in the text files, but they are pretty easy to read even if you don't process them.

I guess our focus needs to be on analyzing more of the uncompressed files, and figuring out the DAT structure. The assembly that was posted above could lead to something. It is just past my expertise at that point. Not good enough to make use of it.

As for that file, it might be used as a template in memory for the Patcher to properly insert data into the DAT file or preferences for the patcher of some sort.

[nicoli_s]

also im almost done figuring out the basic structure of that file, so ill post it once im done

[Nimba]

also im almost done figuring out the basic structure of that file, so ill post it once im done

Very nice, how similar is it to GW1?

[Nimba]

Just an FYI to everyone. They have invalidated the cookie for the Dev branch, and branch 103 and 104. It now only works for the beta client.

So unless we get a new copy of the dev client to get the new cookie, we can't download 102,103,or104. For now, we are stuck with what we have, and the beta client.