Page 11 of 19
Re: [Request] Guild Wars 2 DAT
Posted: Sat May 19, 2012 5:05 pm
by jeckerson
kthackeray wrote:jeckerson wrote:If there are some way to brute-force that keys or read packets while playing?
You have to get them from analyzing server packets, yes. The GW1 algorithm has already been cracked and so far there is no evidence to show that they are using something different here. I'm not sure why you would want to waste the time to brute force them just for some text. Imagine how long it would take to do so for ~198,000 strings each with a unique key.
Thank you for answer. I just need to know all info in strs files, which have information about items, skills, quests as I think so. But damn, what kind of structure data they have there which read and show in da game...
Re: [Request] Guild Wars 2 DAT
Posted: Sat May 19, 2012 5:09 pm
by Ekey
Cruelbob wrote:I have scanned Gw2.exe for cryptographic primitives.
Very old and not actually scanner
.gif "Smile")
here better
Signsrch 0.1.7
Re: [Request] Guild Wars 2 DAT
Posted: Sat May 19, 2012 5:24 pm
by Cruelbob
I think that i have found SendPacket function:
Code: Select all
CPU Disasm
Address Hex dump Command Comments
009FDDA0 /$ 55 PUSH EBP ; Gw2.009FDDA0(guessed Arg1)
009FDDA1 |. 8BEC MOV EBP,ESP
009FDDA3 |. 83EC 28 SUB ESP,28
009FDDA6 |. 53 PUSH EBX
009FDDA7 |. 56 PUSH ESI
009FDDA8 |. 57 PUSH EDI
009FDDA9 |. 8BF9 MOV EDI,ECX
009FDDAB |. 8BF2 MOV ESI,EDX
009FDDAD |. 8975 F8 MOV DWORD PTR SS:[LOCAL.2],ESI
009FDDB0 |. 85FF TEST EDI,EDI
009FDDB2 |. 75 14 JNE SHORT 009FDDC8
009FDDB4 |. 68 6F0E0000 PUSH 0E6F ; /Arg1 = 0E6F
009FDDB9 |. BA D4D32F01 MOV EDX,OFFSET 012FD3D4 ; |ASCII "..\..\..\Services\Msg\MsgConn.cpp"
009FDDBE |. B9 64D92F01 MOV ECX,OFFSET 012FD964 ; |ASCII "mc"
009FDDC3 |. E8 1842C1FF CALL 00611FE0 ; \Gw2.00611FE0
009FDDC8 |> 8B5D 08 MOV EBX,DWORD PTR SS:[ARG.1]
009FDDCB |. 85DB TEST EBX,EBX
009FDDCD |. 75 14 JNE SHORT 009FDDE3
009FDDCF |. 68 700E0000 PUSH 0E70 ; /Arg1 = 0E70
009FDDD4 |. BA D4D32F01 MOV EDX,OFFSET 012FD3D4 ; |ASCII "..\..\..\Services\Msg\MsgConn.cpp"
009FDDD9 |. B9 083D2D01 MOV ECX,OFFSET 012D3D08 ; |ASCII "rawData"
009FDDDE |. E8 FD41C1FF CALL 00611FE0 ; \Gw2.00611FE0
009FDDE3 |> 83FE 02 CMP ESI,2
009FDDE6 |. 73 14 JNB SHORT 009FDDFC
009FDDE8 |. 68 710E0000 PUSH 0E71 ; /Arg1 = 0E71
009FDDED |. BA D4D32F01 MOV EDX,OFFSET 012FD3D4 ; |ASCII "..\..\..\Services\Msg\MsgConn.cpp"
009FDDF2 |. B9 D0D92F01 MOV ECX,OFFSET 012FD9D0 ; |ASCII "rawDataBytes >= sizeof(word)"
009FDDF7 |. E8 E441C1FF CALL 00611FE0 ; \Gw2.00611FE0
009FDDFC |> 83BF A8000000 CMP DWORD PTR DS:[EDI+0A8],2
009FDE03 |. 0F85 4A010000 JNE 009FDF53
009FDE09 |. 837F 7C 00 CMP DWORD PTR DS:[EDI+7C],0
009FDE0D |. 0F84 40010000 JE 009FDF53
009FDE13 |. 0FB713 MOVZX EDX,WORD PTR DS:[EBX]
009FDE16 |. 8B4F 0C MOV ECX,DWORD PTR DS:[EDI+0C]
009FDE19 |. 8955 F0 MOV DWORD PTR SS:[LOCAL.4],EDX
009FDE1C |. E8 3F0F0000 CALL 009FED60
009FDE21 |. 8BF0 MOV ESI,EAX
009FDE23 |. 85F6 TEST ESI,ESI
009FDE25 |. 75 14 JNE SHORT 009FDE3B
009FDE27 |. 68 7F0E0000 PUSH 0E7F ; /Arg1 = 0E7F
009FDE2C |. BA D4D32F01 MOV EDX,OFFSET 012FD3D4 ; |ASCII "..\..\..\Services\Msg\MsgConn.cpp"
009FDE31 |. B9 C8D92F01 MOV ECX,OFFSET 012FD9C8 ; |ASCII "sendMsg"
009FDE36 |. E8 A541C1FF CALL 00611FE0 ; \Gw2.00611FE0
009FDE3B |> 8B06 MOV EAX,DWORD PTR DS:[ESI]
009FDE3D |. 8378 10 00 CMP DWORD PTR DS:[EAX+10],0
009FDE41 |. 75 14 JNE SHORT 009FDE57
009FDE43 |. 68 800E0000 PUSH 0E80 ; /Arg1 = 0E80
009FDE48 |. BA D4D32F01 MOV EDX,OFFSET 012FD3D4 ; |ASCII "..\..\..\Services\Msg\MsgConn.cpp"
009FDE4D |. B9 A8D92F01 MOV ECX,OFFSET 012FD9A8 ; |ASCII "sendMsg->defArray[0].defSize"
009FDE52 |. E8 8941C1FF CALL 00611FE0 ; \Gw2.00611FE0
009FDE57 |> 8B0E MOV ECX,DWORD PTR DS:[ESI]
009FDE59 |. 8B55 F8 MOV EDX,DWORD PTR SS:[LOCAL.2]
009FDE5C |. 3B51 10 CMP EDX,DWORD PTR DS:[ECX+10]
009FDE5F |. 74 14 JE SHORT 009FDE75
009FDE61 |. 68 840E0000 PUSH 0E84 ; /Arg1 = 0E84
009FDE66 |. BA D4D32F01 MOV EDX,OFFSET 012FD3D4 ; |ASCII "..\..\..\Services\Msg\MsgConn.cpp"
009FDE6B |. B9 78D92F01 MOV ECX,OFFSET 012FD978 ; |ASCII "rawDataBytes == sendMsg->defArray[0].defSize"
009FDE70 |. E8 6B41C1FF CALL 00611FE0 ; \Gw2.00611FE0
009FDE75 |> 8D4D 08 LEA ECX,[ARG.1]
009FDE78 |. 51 PUSH ECX ; /Arg6 => OFFSET ARG.1
009FDE79 |. 8B4D F8 MOV ECX,DWORD PTR SS:[LOCAL.2] ; |
009FDE7C |. 8D55 FC LEA EDX,[LOCAL.1] ; |
009FDE7F |. 52 PUSH EDX ; |Arg5 => OFFSET LOCAL.1
009FDE80 |. 03CB ADD ECX,EBX ; |
009FDE82 |. 51 PUSH ECX ; |Arg4
009FDE83 |. 8D55 F4 LEA EDX,[LOCAL.3] ; |
009FDE86 |. 52 PUSH EDX ; |Arg3 => OFFSET LOCAL.3
009FDE87 |. 33C0 XOR EAX,EAX ; |
009FDE89 |. 50 PUSH EAX ; |Arg2 => 0
009FDE8A |. 8945 FC MOV DWORD PTR SS:[LOCAL.1],EAX ; |
009FDE8D |. 8B16 MOV EDX,DWORD PTR DS:[ESI] ; |
009FDE8F |. 57 PUSH EDI ; |Arg1
009FDE90 |. 8D4D D8 LEA ECX,[LOCAL.10] ; |
009FDE93 |. 895D F4 MOV DWORD PTR SS:[LOCAL.3],EBX ; |
009FDE96 |. C745 08 FFFFF MOV DWORD PTR SS:[ARG.1],-1 ; |
009FDE9D |. E8 0EE7FFFF CALL 009FC5B0 ; \Gw2.009FC5B0
009FDEA2 |. 837D EC 00 CMP DWORD PTR SS:[LOCAL.5],0
009FDEA6 |. 75 14 JNE SHORT 009FDEBC
009FDEA8 |. 68 920E0000 PUSH 0E92 ; /Arg1 = 0E92
009FDEAD |. BA D4D32F01 MOV EDX,OFFSET 012FD3D4 ; |ASCII "..\..\..\Services\Msg\MsgConn.cpp"
009FDEB2 |. B9 68D92F01 MOV ECX,OFFSET 012FD968 ; |ASCII "err.IsSuccess()"
009FDEB7 |. E8 2441C1FF CALL 00611FE0 ; \Gw2.00611FE0
009FDEBC |> 8B5D F0 MOV EBX,DWORD PTR SS:[LOCAL.4]
009FDEBF |. 8B55 FC MOV EDX,DWORD PTR SS:[LOCAL.1]
009FDEC2 |. 8BCB MOV ECX,EBX
009FDEC4 |. E8 E71F0000 CALL 009FFEB0
009FDEC9 |. 8B77 18 MOV ESI,DWORD PTR DS:[EDI+18]
009FDECC |. 85F6 TEST ESI,ESI
009FDECE |. 74 55 JE SHORT 009FDF25
009FDED0 |. 813E DDDDDDDD CMP DWORD PTR DS:[ESI],DDDDDDDD
009FDED6 |. 75 14 JNE SHORT 009FDEEC
009FDED8 |. 68 73010000 PUSH 173 ; /Arg1 = 173
009FDEDD |. BA 982D1001 MOV EDX,OFFSET 01102D98 ; |ASCII "p:\code\arena\core\Collections/List.h"
009FDEE2 |. B9 F02E1001 MOV ECX,OFFSET 01102EF0 ; |ASCII "m_linkOffset != static_cast<int>(LINK_OFFSET_UNINIT)"
009FDEE7 |. E8 F440C1FF CALL 00611FE0 ; \Gw2.00611FE0
009FDEEC |> 8B06 MOV EAX,DWORD PTR DS:[ESI]
009FDEEE |. 8B4C38 04 MOV ECX,DWORD PTR DS:[EDI+EAX+4]
009FDEF2 |. 8B1438 MOV EDX,DWORD PTR DS:[EDI+EAX]
009FDEF5 |. 8B52 04 MOV EDX,DWORD PTR DS:[EDX+4]
009FDEF8 |. 03C7 ADD EAX,EDI
009FDEFA |. 83E2 FE AND EDX,FFFFFFFE
009FDEFD |. 83E1 FE AND ECX,FFFFFFFE
009FDF00 |. 2BCA SUB ECX,EDX
009FDF02 |. 8B10 MOV EDX,DWORD PTR DS:[EAX]
009FDF04 |. 891401 MOV DWORD PTR DS:[EAX+ECX],EDX
009FDF07 |. 8B48 04 MOV ECX,DWORD PTR DS:[EAX+4]
009FDF0A |. 8B10 MOV EDX,DWORD PTR DS:[EAX]
009FDF0C |. 894A 04 MOV DWORD PTR DS:[EDX+4],ECX
009FDF0F |. 8B4E 04 MOV ECX,DWORD PTR DS:[ESI+4]
009FDF12 |. 8908 MOV DWORD PTR DS:[EAX],ECX
009FDF14 |. 8B10 MOV EDX,DWORD PTR DS:[EAX]
009FDF16 |. 8B4A 04 MOV ECX,DWORD PTR DS:[EDX+4]
009FDF19 |. 8948 04 MOV DWORD PTR DS:[EAX+4],ECX
009FDF1C |. 8B56 04 MOV EDX,DWORD PTR DS:[ESI+4]
009FDF1F |. 897A 04 MOV DWORD PTR DS:[EDX+4],EDI
009FDF22 |. 8946 04 MOV DWORD PTR DS:[ESI+4],EAX
009FDF25 |> 833D BC4D6501 CMP DWORD PTR DS:[1654DBC],0
009FDF2C |. 74 10 JE SHORT 009FDF3E
009FDF2E |. 8D87 D4020000 LEA EAX,[EDI+2D4]
009FDF34 |. 3947 78 CMP DWORD PTR DS:[EDI+78],EAX
009FDF37 |. 74 05 JE SHORT 009FDF3E
009FDF39 |. E8 52D2FFFF CALL 009FB190 ; [Gw2.009FB190
009FDF3E |> 8B4D 08 MOV ECX,DWORD PTR SS:[ARG.1]
009FDF41 |. 8B55 FC MOV EDX,DWORD PTR SS:[LOCAL.1]
009FDF44 |. 8B47 08 MOV EAX,DWORD PTR DS:[EDI+8]
009FDF47 |. 51 PUSH ECX ; /Arg4 => [ARG.1]
009FDF48 |. 52 PUSH EDX ; |Arg3 => [LOCAL.1]
009FDF49 |. 53 PUSH EBX ; |Arg2
009FDF4A |. 50 PUSH EAX ; |Arg1
009FDF4B |. E8 00BFC0FF CALL 00609E50 ; \Gw2.00609E50
009FDF50 |. 83C4 10 ADD ESP,10
009FDF53 |> 5F POP EDI
009FDF54 |. 5E POP ESI
009FDF55 |. 5B POP EBX
009FDF56 |. 8BE5 MOV ESP,EBP
009FDF58 |. 5D POP EBP
009FDF59 \. C2 0400 RETN 4
Re: [Request] Guild Wars 2 DAT
Posted: Sat May 19, 2012 9:29 pm
by Loumie
Multiple things here:
- strs files: I can confirm that in GW2 they still use some kind of RC4 encryption on the encoded strings. What you call lchar is actually an offset used when outputting the string, (if value < 32 then tableLookup else offset+value-32) so by construction it is the lowest char. Finally the last value 0x10 in UTF-16 strings is the number of significant bits, it means than at each pass it reads X bits to decode a value. (I don't know about the value received from the server as I didn't look further)
- Unpacker: Please take a look at the unpacker of Rhoot, it is quite good and will most likely suit your needs:
https://bitbucket.org/Daegalus/gw2re/src
- crypt: On page 1 post 5 Ekey displayed the output of signsrch
Cheers,
Loumie
Re: [Request] Guild Wars 2 DAT
Posted: Sat May 19, 2012 10:53 pm
by ral
So here is what I know about PF/ABNKBKCK files:
Code: Select all
Start:
16 bytes File Header (PF ABNKBKCK):
50 46 01 00 00 00 0C 00 41 42 4E 4B 42 4B 43 4B
40 bytes ABNKBKCK Header:
4 bytes (number of bytes in file - 20)
4 bytes (02 00 10 00)
4 bytes (number of bytes in file - 80)
16 bytes (always 0x00)
4 bytes (always 0A 00 00 00)
4 bytes (always 08 00 00 00)
4 bytes (always 0x00)
10 Entries with 44 bytes each entry (entries can be full 0x00) (440 bytes in total):
4 bytes (always counting up +1 in a file but I don't know how to find out what the start number is. Empty entry still counts the counter up)
4 bytes (always 01 00 00 00 or 00 00 00 00 but I don't know what it means)
16 bytes (always 0x00)
4 bytes (unknown)
4 bytes (unknown)
4 bytes (always 0x00)
4 bytes (bytes for the file. Always a ASNDASND (can include a MODLMODL after the ASNDASND I think))
4 bytes (unknown)
Rest of the file are the single ASNDASND and MODLMODL files.
Re: [Request] Guild Wars 2 DAT
Posted: Sun May 20, 2012 3:05 am
by xtridence
GW2Unpacker CLI is no longer actively developed. Please use the repository version (see rhoot's post following this post).
Some general tips to compiling repository version:
* Install and compile wxwidgets 2.9.3+. Their recommendation is to compile using the VC9 solution file in the build directory via VS2008. You can use VS2010 to compile this as well.
* For an easier life, set the option for Runtime library to "Multi-threaded Debug" (Configuration properties, C/C++, Code Generation) for all Wxwidgets projects and the bitbucket project.
* Follow the instructions on the
wxWidgets wiki for editing several project properties if needed (don't forget to set environment variables for your computer and link the libraries to the project)
* If using VS2010 to compile the repository version (and VS2008 to compile wxwidgets), set platform toolset to use v90 instead of v100. If using VS2010 for both, ignore this step (both should be set to v100).
* Tweak where needed (May need to use stdint.h instead of cstdint, which can be obtained online. Replace nullptr with NULL if having problems with nullptr.)
As for PF files, they consist of a 12 byte header, followed by several blocks of data.
Each block/chunk has a format like this:
Code: Select all
Header:
* 4 byte name
* 4 byte offset to next chunk
* 2 byte something
* 2 byte block header size
* 4 byte offset to a table (no table if this value is 0)
Data: Contains offsets to actual data, as well as the data itself
Table: Usually occurs after data.
* First 4 bytes is the number of entries in the table
* Each entry is an offset starting from the end of the header. It points to another offset that starts from that point.
ABNK can contain more than one PF file (usually ASND).
ASND can contain MP3 (which usually start with something like FF FB) or Ogg, as well as MODL PF file (it's a file within a file within a file within a file)
(The PFviewer can be used to extract these.)
Re: [Request] Guild Wars 2 DAT
Posted: Sun May 20, 2012 6:27 pm
by Rhoot
The bitbucket repository has moved, actually. The reason being that so far only me and Loumie have put anything in it, and of us two only I had a preference of DVCS client. Individual projects will (from now on) be pushed to it as submodules, so it's more of a listing of projects. It can still be cloned as one big repository if wanted though, but you'll have to do a recursive clone.
On top of this, I uploaded the browser I've been working on for a couple of weeks now. It acts as an extension of the unpacker. Scanning a .dat takes roughly 10 minutes but once that is done, every subsequent start the indexer should load the contents in about 5-10 seconds. Files can be exported if wanted, by right clicking on them in the tree. So far it only supports viewing power-of-two textures and binary data as hex. The reason it does not do non-power-of-two textures is cause they seem to have a different compression algorithm. Even with the ATexReader they turn out as garbage blocks.
Anyway, the new repository is located on github, at:
https://github.com/Daegalus/gw2re
The browser is located at:
https://github.com/rhoot/Gw2Browser
If you prefer binaries over source, there are binaries of the browser at
http://skold.cc/gw2browser/
Edit:
xtridence wrote:Some general tips to compiling bitbucket version (in the event you run into problems):
* Install and compile wxwidgets 2.9.3+. Best to compile using the VC9 solution file in the build directory via VS2008.
* For an easier life, set the option for Runtime library to "Multi-threaded Debug" (Configuration properties, C/C++, Code Generation) for all Wxwidgets projects and the bitbucket project.
* Follow the instructions on the
wxWidgets wiki for editing several project properties (don't forget to set environment variables for your computer and link the libraries to the project)
* If using VS2010 to compile the BitBucket version (and VS2008 to compile wxwidgets), set platform toolset to use v90 instead of v100.
* Tweak where needed (May need to use stdint.h instead of cstdint, which can be obtained online. Replace nullptr with NULL if not compiling using /clr option. Read the readme file.)
The unpacker was actually written in VS2010. It was linked against a version of wxWidgets built in 2010 while developing as well, so there shouldn't be any need to go to 2008 unless you don't have 2010. Also, nullptr is actually
valid C++ starting with C++11, and is supported by MSVC
starting with 2010. That said, the unpacker doesn't use it anywhere, so I'm assuming this line applies to wxWidgets.
Edit 2: Fixed a bug in the browser that made it unable to save the index on computers where it had not saved an index before (a directory didn't exist for it). The binary linked to above has been replaced, so if you're having troubles, get a new one.
Re: [Request] Guild Wars 2 DAT
Posted: Mon May 21, 2012 1:40 am
by xtridence
My guide is intended for those who were having issues compiling the code, so I propose an alternative workaround. The best solution of course, is to simply use VS2010.
nullptr is supported in VS2010 (but there are caveats for VS2008 and lower).
Re: [Request] Guild Wars 2 DAT
Posted: Mon May 21, 2012 11:52 pm
by Rhoot
I just figured out the DXTA format, thanks to some data obtained by Loumie. It's really simple. Each block contains no color, but only an alpha channel, so 64 bits per block. The alpha is calculated in the exact same way it is for DXT5.
In order to get the decompression to work, it needs the image format 0x0A1. You can get that in the ATexReader by feeding 0x14 to the AtexDecompress function (though in GW2 it's really format 0x1a, since the format table was updated).
That's really all there is to it.
Re: [Request] Guild Wars 2 DAT
Posted: Tue May 22, 2012 1:52 am
by xtridence
So DXTA is similar to the BC4 format? I guess that could be arranged.
Update: Here's the ATEX reader.
Re: [Request] Guild Wars 2 DAT
Posted: Wed May 23, 2012 7:30 pm
by Grax
Can someone tell me, what those "Bank files" are?
Looks like mp3, but i couldn't figure out how to get a real .mp3 file out of it.
EDIT:
Ooh, got it
Re: [Request] Guild Wars 2 DAT
Posted: Thu May 24, 2012 5:32 pm
by lordsavyj
Has anyone, who is willing to share, discovered where to look for item details in the gw2.dat? I'm specifically looking for recipe info. I've been parsing the data...slowly...but with so many files a pointer would be much appreciated.
Re: [Request] Guild Wars 2 DAT
Posted: Thu May 24, 2012 5:58 pm
by nicoli_s
I'll say this, you arent just going to stumble upon the item/recipe/skill data. Anet did a good job making it very hard to access
Re: [Request] Guild Wars 2 DAT
Posted: Thu May 24, 2012 7:56 pm
by stalja
From what I gathered from old Guild Wars 1 Dat they found out that items are basically files with flags that determine that item;s name, stats and special characteristics. They are not ASCII, but binary files with a special structure holding IDs of stats and such. Which file of the ones extracted so far I do not now. Again, person with RE skills has probably figured it through reversing the .exe
Re: [Request] Guild Wars 2 DAT
Posted: Thu May 24, 2012 10:09 pm
by kthackeray
stalja wrote:From what I gathered from old Guild Wars 1 Dat they found out that items are basically files with flags that determine that item;s name, stats and special characteristics. They are not ASCII, but binary files with a special structure holding IDs of stats and such. Which file of the ones extracted so far I do not now. Again, person with RE skills has probably figured it through reversing the .exe
Not sure what file you're talking about here, but in GW1 all item characteristics are sent by the server via an item packet. The dat doesn't contain anymore info than the name of the item and its icon. I assume GW2 is using a similar system, but time will tell.