Re: Dead or Alive 3,U,X tools [download]
Posted: Sat Sep 24, 2011 9:09 am
doau format is not very different from the doa3 xpr. "DOAXExporter.exe" is building a "debug.txt", which you can use to understand the difference.mariokart64n wrote:do you have format specs on DOAU?
EDIT
found an old doc; http://www.viewdocsonline.com/document/wpe6kq
its not very detailed, only reconfirmed what I had just figured out in the past hour.
do you have any additional information?
Does this mean after the header there can be 3 types of files and a magic number determines which type it is?After the xpr header, there is are the 3 types of “files”
First there is a magic number
0x80000000 for a model
0x00040001 for a texture
0x00800001 for a vertex buffer
Okay, what are these dwords for? He could at least meantion it's unknown.Following the model header is an array of dwords the count is in the model header “numobj”
Starting from this it doesn't make sense anymore. What object offset, what "beginning of the file"?So the 3 object offsets are
38 00 00 00
D2 0C 00 00
D6 11 00 00
These offsets are from the beginning of the file
So looking at offset 38 we see the first object block
EDITThe models are stored into a xpr file, which from what I recall. someone mentioned that XPR was a microsoft file container for the xbox. I don’t know if this is true, but I’ve seen the xbox360 SDK, and it uses xpr too, only its a bit different. but the xbox360 xpr is used for storing textures.. and you can also embed a “user” file
so keeping that in mind the XPR is just a container holding the various “components” that the game reassembles during rendering. More commonly, Textures, and whatever else the dev stuffs in there relating to that texture resource.
Although I’m still not sure why they store part of the model in a separate file, since they could have stacked everything within the XPR.. but whatever, I’ll break down my findings so far
XPR FORMAT
the format uses chunk ids, meaning you keep reading an ID, and keep reading the data until a terminator is met. usually 0xFFFF
the structure is pretty simple, keep reading the chunks into memory, and use them to unpack the “Data Buffer” the data buffer always seems to be aligned to 2048bytes [0x0800]. the byte 0xAD is used to pad any between the start of the file and the start of the buffer. Then padding is used again from the end of the buffer to the end of the file.
Its best to read the chunks all into memory, then unpack once you’ve hit the 0xFFFF terminator.
*Note: Data Stored as Little Endian
*Note: there are different vertex types used, some storing vertex weights, some not.
*Note: Face Data seem to be Stored as Triangle Strips, separated by material
**Note:
a 0xFF terminator is seen at the end of the file, after you read the
texture and model entries. A 0xFF terminator is also used within the MDL Chunk.
Also a 0x00 terminator is seen at the start of the file, after the MDL chunk.
So keep that in mind, when re-writting a new XPR.
Model Format
After you read the model table, it’ll jump you to another table, before finally retrieving the geometry data in the buffer. But its a bit confusing because they do this weird filetree thing where chunks are inside chunks which are inside other chunks.
LONG: ChunkID
ChunkID = 0x[58505230] // “XPR0” Header, the first chunk [12bytes]
{
LONG: File Size
LONG: Data Buffer Offset
}
ChunkID = 0x[00000080] // Jumps to texture and mesh tables.
{
LONG: Offset, jump your cursor from this spot, continue to loop chunks until 0xFF is met
then return cursor to where you you left off reading the 0x[00000080] ChunkID
And then continue your loop until the terminator 0x00000000 is met.
)
ChunkID = 0x[01000400] // Texture Entry [32bytes per entry]
{
LONG: Data Offset, Jump From Start Data Buffer. Offset given in XPR0 header
LONG: Unknown: usually 0x00
LONG: Texture Details, data is stored in bits, dimensions etc.. haven’t broken it down yet
LONG: Unknown: usually 0x00
SHORT: Unknown: usually 0x01
SHORT: Unknown: usually 0x83
LONG: Data Size
LONG: Unknown: usually 0x00
}
ChunkID - 0x[01008000] // Model Entry [12bytes per entry]
// This Table is kinda useless, but just points you to the vertices used in the buffer
{
Long: Data Offset, from start of data buffer
LONG: Unknown: usually 0x00
}
ChunkID = 0x[290F6106] // Unknown
{
LONG: Unknown: usually 0x00
}
ChunkID = 0x[4D444C00] // “MDL ”
{
LONG: Object Count
LONG: Texture Count
LONG: Unknown: usually 0x00
LONG: Object Count, count duplicated ????
{ // Loop = Object Count
LONG: Object Offset, Jump from start of file
}
}
ChunkID = 0x[4F424A00] // “OBJ ”
{
LONG: Vertex Type ???
LONG: Unknown: usually 0x00 | 0x04
LONG: Unknown, may be a count??
LONG: Unknown, may be a count??
FLOAT[32]: Unknown, may be position coordinate
FLOAT[32]: Unknown, may be position coordinate
FLOAT[32]: Unknown, may be position coordinate
LONG: Vertex Count
LONG: Vertex Offset, points to data in data buffer, jump from start of file
LONG: Face Count
LONG: Face Offset, jump from start of file
BYTES[112]: Reserved Space??? data within range are all 0x00
{
LOOP: Loop until subID is 0xFFFFFF
{
LONG: SubChunkID, ** these IDs seem to denote different material properties
0x00000080 | 0x01000080 | 0x02000080 | 0x05000080
{
LONG: SubChunkSize
LONG: Unknown, Always 0x00
LONG: Unknown, Always 0x08
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Unknown
LONG: Number of Faces used by this material??? need to add 2?
LONG: Unknown, always 0x02
}
}
}
ChunkID = 0x[FFFFFFFF] // Stop Loop!
ChunkID = 0x[00000000] // Stop Loop!
****Anything above is stuff I’ve figured out my own by visual inspection of the files via a hexeditor. I still don’t understand the material chunks which are the key to the mesh import.
And theres alot of values there I need to test ingame to discover there function. but I’ll update once I solve this.
-mariokart64n
You might want to put an "if" before ChunkID = something, although I'm probably one of the few who didn't get it straight away.LONG: ChunkID
ChunkID = 0x[58505230] // “XPR0” Header, the first chunk [12bytes]
...
Can be easily checked by simply creating the vertices in max. You can easily tell random junk from something real even only given vertices in a 3d view. Or you could wait until I get something done in Blender.FLOAT[32]: Unknown, may be position coordinate
FLOAT[32]: Unknown, may be position coordinate
FLOAT[32]: Unknown, may be position coordinate
Code: Select all
# -*- coding: utf-8 -*-
import struct
class globals:
pass
file = open('ayaA0.xpr', 'rb')
# the chunk ids seem big endian to me
# The file consists of smaller chunks, which start with a chunk id.
# Each chunk type stores different data and is different Just go
# through all chunks until you reach the end.
# functions for each chunk type
def header():
print '\nHeader chunk\n----------------'
#LONG: File Size
globals.filesize = struct.unpack('<l', file.read(4))[0]
print 'filesize:', globals.filesize
#LONG: Data Buffer Offset
globals.databufferoffset = struct.unpack('<l', file.read(4))[0]
print 'databufferoffset:', globals.databufferoffset
def textureAndMeshTable():
print '\ntextureAndMeshTable\n----------------'
# LONG: Offset, jump your cursor from this spot, continue to loop chunks until 0xFF is met
# then return cursor to where you left off reading the 0x00000080 ChunkID
# And then continue your loop until the terminator 0x00000000 is met.
offset = struct.unpack('<l', file.read(4))[0]
print 'offset:', offset
def textureEntry():
print '\ntextureEntry\n----------------'
# LONG: Data Offset, Jump From Start Data Buffer. Offset given in XPR0 header
dataoffset = struct.unpack('<l', file.read(4))[0]
print 'dataoffset:', dataoffset
# LONG: Unknown: usually 0x00
file.read(4)
# LONG: Texture Details, data is stored in bits, dimensions etc.. haven’t broken it down yet
file.read(4)
# LONG: Unknown: usually 0x00
file.read(4)
# SHORT: Unknown: usually 0x01
file.read(2)
# SHORT: Unknown: usually 0x83
file.read(2)
# LONG: Data Size
datasize = struct.unpack('<l', file.read(4))[0]
print 'datasize:', datasize
# LONG: Unknown: usually 0x00
file.read(4)
def modelEntry():
# This Table is kinda useless, but just points you to the vertices used in the buffer
print '\nmodelEntry\n----------------'
# LONG: Data Offset, from start of data buffer
dataoffset = struct.unpack('<l', file.read(4))[0]
print 'dataoffset:', dataoffset
# LONG: Unknown: usually 0x00
file.read(4)
def unknown():
print '\nUnknown\n----------------'
# LONG: Unknown: usually 0x00
file.read(4)
def MDL():
print '\nMDL\n----------------'
# LONG: Object Count
objcount = struct.unpack('<l', file.read(4))[0]
print 'objcount:', objcount
# LONG: Texture Count
texcount = struct.unpack('<l', file.read(4))[0]
print 'texcount:', texcount
# LONG: Unknown: usually 0x00
file.read(4)
# LONG: Object Count, count duplicated ????
file.read(4)
objoffsets = []
print 'For each object:'
for i in range(objcount):
# LONG: Object Offset, Jump from start of file
objoffset = struct.unpack('<l', file.read(4))[0]
print ' objoffset:', objoffset
objoffsets.append(objoffset)
def OBJ():
print '\nOBJ\n----------------'
# LONG: Vertex Type ???
vertextype = struct.unpack('<l', file.read(4))[0]
print 'vertextype:', vertextype
# LONG: Unknown: usually 0x00 | 0x04
file.read(4)
# LONG: Unknown, may be a count??
file.read(4)
# LONG: Unknown, may be a count??
file.read(4)
# (FLOAT[32]) * 3: Unknown, may be position coordinate
posx, posy, posx = struct.unpack('<3f', file.read(12))
print 'position:', posx, posy, posx
# LONG: Vertex Count
vertexcount = struct.unpack('<l', file.read(4))[0]
print 'vertexcount:', vertexcount
# LONG: Vertex Offset, points to data in data buffer, jump from start of file
vertexoffset = struct.unpack('<l', file.read(4))[0]
print 'vertexoffset:', vertexoffset
# LONG: Face Count
facecount = struct.unpack('<l', file.read(4))[0]
print 'facecount:', facecount
# LONG: Face Offset, jump from start of file
faceoffset = struct.unpack('<l', file.read(4))[0]
print 'faceoffset:', faceoffset
# BYTES[112]: Reserved Space??? data within range are all 0x00
file.read(112)
subids = [0x00000080, 0x01000080, 0x02000080, 0x05000080]
while True: # Loop until subID is 0xFFFFFF
# LONG: SubChunkID, ** these IDs seem to denote different material properties
subchunkid = struct.unpack('<l', file.read(4))[0]
print 'Subchunk id:', subchunkid
if subchunkid == 0xFFFFFF: break
# LONG: SubChunkSize
subchunksize = struct.unpack('<l', file.read(4))[0]
print ' subchunksize:', subchunksize
# LONG: Unknown, Always 0x00
file.read(4)
# LONG: Unknown, Always 0x08
file.read(4)
# (LONG: Unknown) * 30
file.read(4*30)
# LONG: Number of Faces used by this material??? need to add 2?
unknown = struct.unpack('<l', file.read(4))[0]
# LONG: Unknown, always 0x02
file.read(4)
# chunk ids and the functions to run
chunkids = {0x58505230: header,
0x00000080: textureAndMeshTable,
0x01000400: textureEntry,
0x01008000: modelEntry,
0x290F6106: unknown,
0x4D444C00: MDL,
0x4F424A00: OBJ}
# start looping through the chunks
while True:
chunkid = struct.unpack('>l', file.read(4))[0]
if chunkid in (0xFFFFFFFF, 0x00000000):
break
else:
chunkids[chunkid]()
Do you mean jump to offset, counting from current position, and continue reading chunks as usual before 0xFF is met and then come back to this position? But why do you mention to "continue your loop until the terminator 0x00000000 is met." here? Shouldn't we have been doing it all along?# LONG: Offset, jump your cursor from this spot, continue to loop chunks until 0xFF is met
# then return cursor to where you left off reading the 0x00000080 ChunkID
# And then continue your loop until the terminator 0x00000000 is met.
Code: Select all
00000000 01 00 04 00 00 00 00 00 00 00 00 00 2A 0B 71 07 00 00 00 00 01 00 83 00 00 40 00 00 00 00 00 00 ............*.q.......ƒ..@......
00000020 01 00 04 00 00 44 00 00 00 00 00 00 2A 0B 71 07 00 00 00 00 01 00 83 00 00 84 00 00 00 00 00 00 .....D......*.q.......ƒ..„......
00000040 01 00 04 00 00 88 00 00 00 00 00 00 2A 0B 71 07 00 00 00 00 01 00 83 00 00 C8 00 00 00 00 00 00 .....ˆ......*.q.......ƒ..È......
00000060 01 00 04 00 00 CC 00 00 00 00 00 00 2A 0B 71 07 00 00 00 00 01 00 83 00 00 0C 01 00 00 00 00 00 .....Ì......*.q.......ƒ.........
00000080 01 00 04 00 00 10 01 00 00 00 00 00 2A 0B 71 07 00 00 00 00 01 00 83 00 00 50 01 00 00 00 00 00 ............*.q.......ƒ..P......
000000A0 01 00 04 00 00 54 01 00 00 00 00 00 2A 0B 71 07 00 00 00 00 01 00 83 00 00 94 01 00 00 00 00 00 .....T......*.q.......ƒ..”......
000000C0 01 00 04 00 00 98 01 00 00 00 00 00 2A 0B 71 07 00 00 00 00 01 00 83 00 00 D8 01 00 00 00 00 00 .....˜......*.q.......ƒ..Ø......
000000E0 01 00 04 00 00 DC 01 00 00 00 00 00 2A 0B 71 07 00 00 00 00 01 00 83 00 00 1C 02 00 00 00 00 00 .....Ü......*.q.......ƒ.........
00000100 01 00 04 00 00 20 02 00 00 00 00 00 2A 0B 71 07 00 00 00 00 01 00 83 00 00 60 02 00 00 00 00 00 ..... ......*.q.......ƒ..`......
00000120 01 00 04 00 00 64 02 00 00 00 00 00 2A 0B 71 07 00 00 00 00 01 00 83 00 00 A4 02 00 00 00 00 00 .....d......*.q.......ƒ..¤......
00000140 01 00 04 00 00 A8 02 00 00 00 00 00 2A 0B 71 07 00 00 00 00 01 00 83 00 00 E8 02 00 00 00 00 00 .....¨......*.q.......ƒ..è......
00000160 01 00 04 00 00 EC 02 00 00 00 00 00 2A 0B 71 07 00 00 00 00 01 00 83 00 00 2C 03 00 00 00 00 00 .....ì......*.q.......ƒ..,......
Code: Select all
00000000 01 00 80 00 00 00 00 00 00 00 00 00 ..€.........
0000000C 01 00 80 00 00 6B 00 00 00 00 00 00 ..€..k......
00000018 01 00 80 00 00 C4 01 00 00 00 00 00 ..€..Ä......
00000024 01 00 80 00 40 E2 01 00 00 00 00 00 ..€.@â......
00000030 01 00 80 00 80 00 02 00 00 00 00 00 ..€.€.......
0000003C 01 00 80 00 40 EA 02 00 00 00 00 00 ..€.@ê......
00000048 01 00 80 00 78 12 03 00 00 00 00 00 ..€.x.......
00000054 01 00 80 00 58 13 03 00 00 00 00 00 ..€.X.......
00000060 01 00 80 00 D0 22 03 00 00 00 00 00 ..€.Ð"......
0000006C 01 00 80 00 2C 4B 03 00 00 00 00 00 ..€.,K......
00000078 01 00 80 00 EC 4B 03 00 00 00 00 00 ..€.ìK......
00000084 01 00 80 00 88 5B 03 00 00 00 00 00 ..€.ˆ[......
00000090 01 00 80 00 C8 F2 03 00 00 00 00 00 ..€.Èò......
I know, I just mean you could repost what you "think" you know. Right now some things are missing. Seems like you forgot to post some info which you forgot only you know, and there are places which are just confusing (the wording).I posted what I know just to give others an idea of my own thoughts.