XeNTaX

I only remember hearing that they are very similar. Guess they use the same format. DOA3 XPR is different though, I've compared some chunks.

I'll have a look at the DOA3 format tonight, I think only the material section is different?

EDIT

If you guys will want me to add links to the first post, let me know in your posts.

I had a look at the DOAX XPR, its likely easier

Only differences I can gather are
1.) Start of XPR has a file HASH for file verification
2.) OBJ block's Now Use VertexTypes 2 & 3
in my vertex line I just tossed this in;
if vertextype==1 do fseek fstream 4 #seek_cur
if vertextype==2 do fseek fstream 8 #seek_cur
if vertextype==3 do fseek fstream 12 #seek_cur
I'm still not reading in the vertex data, so its skipped for now
3.) I found notes indicating that the Texture Table uses different compression type Flags. putting the wrong flag into the list apparently causes a game crash??

anyway hackers from before were able to get a DOAX character into DOAU. not very surprising, since both games share the same engine and were built around the same time.
in the image below you can see that it almost works.. the problem is pretty obvious though... DOAX has more bones then DOA2U .. either the CAT file is asking for things that don't existing or the skeleton doesn't support the model entirely... being the skeletons may be hardcoded



EDIT
I'm going to start organizing the DOA format spec into a spread sheet and base it off the current one floating around here.

tonight I looked at the morph target block, and it was SUCH A PAIN! because the specs i had were FREAKING TERRIBLE!! pretty much had to reverse the entire bloc over from scratch!!
although big thanks to chrrox for helping figure out the counts in the morph table.

I think I'm good enough to explain the morph bloc now;

Basically the format stores vertex and normals, but tries to optimize by only defining vertices or normals that
change between each morph target. so because of that we are given a index of which vertex entries from
our object need to be altered.

The hard part for me was the index table, it actually stored offsets rather then indices. the offset was relative to the XPR, and the offset of that objects vertex buffer.
so for me to index, I had to convert the offsets back by way of a formula ' i = ((o/s)+1) ' I took the supplied offset, and divided by the size of the XPR's vertex length. then added 1 to make it a base of 1 again.

K, here's the format;

CAT Format - 3th Bloc (Morph Targets "APV")
==================================
APV Header
[LONG] ChunkID "APV "
[LONG] Number of Objects
==================================
[LONG] Offset to Object Entry
loop for "Number of Objects"
==================================
==================================
Object Entry
[LONG] Number of Morph Targets
[BYTE] Object Index
[BYTE] ??? Always 0x00
[BYTE] ??? Always 0x00
[BYTE] Flag, for X axis 0x00=no change to x vertex 0x80=inverse x axis coordinate
[LONG] Number of Morphable Vertices
[LONG] Number of Morphable Vertex Normals
[LONG] Offset to Buffer Offset Table
[LONG] Offset to Vertex Index
[LONG] Offset to Vertex Normal Index
loop for "Number of Objects"
==================================
Buffer Offset Table
[LONG] Offset to Vertex Buffer
[LONG] Offset to Vertex Normal Buffer
loop for "Number of Morph Targets"
==================================
Offset to Vertex Index
[LONG] Vertex Offset from XPR, Relative to the start of that Obj's Vertex Buffer Offset
==================================
Offset to Vertex Normal Index
[LONG] Vertex Normal Offset from XPR, Relative to the start of that Obj's Vertex Buffer Offset
==================================
==================================
Vertex Definition
[FLOAT32] X Vertex Position
[FLOAT32] Y Vertex Position
[FLOAT32] Z Vertex Position
Remember is you get a flag of 0x80, the X coordinate needs to be inverted x=(x*-1)
==================================
==================================
Vertex Normal Definition
[FLOAT32] X Vertex Normal
[FLOAT32] Y Vertex Normal
[FLOAT32] Z Vertex Normal
==================================
==================================

Good job. I dont get why those objects are in the file?

OK, this spreedsheet doc was good. I understood most of it. Few things are still not clear. Could you explain?
http://i.imgur.com/dL5uX.png
Another one, each XPR has only one model entry, right? Because the offset in the beginning wouldn't make sense then.

Sorry if you have answered some already (like the model entry offset), it's just I've read few docs and they give different explanations. I'll stick with this one from now.

ok since you numbered your questions in the image, I'll provide answers in that order

1.) What is the Point of this? [0x80000000]
the documents are wrong in some cases..
That Int is actually a XPR chunk or instruction telling the engine to continue reading at a new address

if you have a look at Ninja Gaiden, you'll see the same tags used.. and the structure is pretty simple
[LONG] Instruction 0x80000000
[LONG] Offset, Offset relative from the end of this LONG/Offset

in this case after [0x80000000] A offset is given to some Tables, which can be used to decompile components of the XPR.
For Example A Table for the Textures, and a Table for each vertex SubBuffer


2.) Is this Needed? (Model + Objects + Materials + Index Buffers)
this fits in with above, the docs are wrong. but this offset brings you to the resource tables.

theres no point in dumping the vertex buffers, however you can use the texture table to extract textures.
so that soo called "Size" is actually the offset to the texture table, which you need for texture dumping.


3.) Object Offsets???? What do I do with them?
the object offsets bring you to every "OBJ " header in the file

the offsets are given in the "MDL " Header
4D 44 4C 00 1D 00 00 00 0F 00 00 00 00 00 00 00 1D 00 00 00 MDL.................
A0 00 00 00 06 03 00 00 5E 19 00 00 1A 1E 00 00 9A 1F 00 00
12 21 00 00 0E 23 00 00 46 27 00 00 D2 28 00 00 32 2D 00 00
84 2F 00 00 88 31 00 00 D6 33 00 00 42 35 00 00 56 37 00 00
0A 39 00 00 5A 3B 00 00 DA 3C 00 00 52 3E 00 00 4C 40 00 00
64 44 00 00 F0 45 00 00 5C 4A 00 00 A8 4C 00 00 A8 4E 00 00
EA 50 00 00 56 52 00 00 6A 54 00 00 0A 56 00 00 00 00 00 00

The first 5 [LONG]'s are various counts, then it loops until returns 0
those are the OBJ offsets, you need this to import the model parts.


4.) Always 8 IV Buffer Info Fields? Does this Depend on an Unknown?
I never figured this out either? I haven't seen anything there but zeros.. I read the Counts then skip 112bytes past that padding?
[LONG] FileID "OBJ "
[LONG] Vertex Type
[LONG] Unknown, sometimes 0, sometimes 4? maybe OBJ version
[LONG] Number of Indices
[FLOAT32] ???
[FLOAT32] ???
[FLOAT32] ???
[FLOAT32] ???
[LONG] Vertex Count
[LONG] Vertex Buffer Offset
[LONG] Number of Indices, duplicated count?
[LONG] Index Buffer Offset
[BYTES{112}] SKIP!

5.) Is everything in IV Buffer Info 0? Why is it called IVBufferInfo?
I got no idea who wrote this, or why they called it the IVBufferInfo...

but from what I tested, the data always seems to be 0. anyway its not critical for mesh import

6.) What does the Material 1..n mean??
I'm not sure what that is either? maybe they just mean to say loop till the end? lol I don't know

starts with something like this 0x01000080, you stop your loop when that =0x00000000

7.) What is Texture 1..n Suppose to mean??
a material can have any amount of textures, so I'm going to have to assume you keep reading texture indeices until you get an index of -1 or 0

I can't give you more detail on the material section since I'm not correctly parsing it either. I grab my counts for mesh import and skip the rest.

EDIT
oh yeah the cat file stores alot of the mesh, I'd say like 80% of the model is repeated in the CAT file... of course all used in different ways. physics, morphs, bends..

Another question, do XPRs store dds textures? Can we just dump those sections and add a header or something?

1.

mariokart64n wrote: the documents are wrong in some cases..
That Int is actually a XPR chunk or instruction telling the engine to continue reading at a new address

So the int is the address where I should jump to? I don't get the idea behind it, what's in the space between where I am and where I must jump to?

2.

this fits in with above, the docs are wrong. but this offset brings you to the resource tables.

Resource tables? Are you talking about the second part of the file? (vertex buffers and textures, because that location is stored in the beginning of the file)

3.

the object offsets bring you to every "OBJ " header in the file

I didn't ask that. I meant what's the 0x00000000 in the end for?

4.

I never figured this out either? I haven't seen anything there but zeros.. I read the Counts then skip 112bytes past that padding?
[LONG] FileID "OBJ "
[LONG] Vertex Type
[LONG] Unknown, sometimes 0, sometimes 4? maybe OBJ version
[LONG] Number of Indices
[FLOAT32] ???
[FLOAT32] ???
[FLOAT32] ???
[FLOAT32] ???
[LONG] Vertex Count
[LONG] Vertex Buffer Offset
[LONG] Number of Indices, duplicated count?
[LONG] Index Buffer Offset
[BYTES{112}] SKIP!

Looks liek you are readin only 1 'IVBuffer' and skipping the other 3, whatever the hell they are for.

5.

but from what I tested, the data always seems to be 0. anyway its not critical for mesh import

What I meant was does he mean 4 dwords, or 4 IVBuffer structs, which themselves have 4 dwords?

6.

I'm not sure what that is either? maybe they just mean to say loop till the end? lol I don't know

starts with something like this 0x01000080, you stop your loop when that =0x00000000

Oh, so you just read material entries here and the "0x00000000" written in the end is a terminator?

This particular doc isn't very good at showing the "indentation" of the entries.


In words, is the Material entry inside the Object entry, or inside the Model entry (together with the Object entry). The first would make more sense.

7.

a material can have any amount of textures, so I'm going to have to assume you keep reading texture indeices until you get an index of -1 or 0

Ah, OK. So it's just an array of texture indexes which end when you get -1 or 0.

Thanks

mariokart64n wrote:I just put the mesh objects together, and copied the coordinates from 3dsmax. from objects like the left and right hand, the distance between the two should have been relative to the positions in the file. however there not.

I'm afraid I'm out of ideas on importing meshes with proper positions :\

i didn't find where is stored the skeleton info in the executable. but you should gather the skeleton info only once for each character - it should be valuable for all skins of one character. the info on to which body part correspond a certain obj can be extracted from the cat file. as i said before skeletons for loran, mam, young tina and rabits is stored in obj files you can try to import these models. use the info on cat files i posted, if something is unclear feel free to ask.

mariokart64n wrote: as for the DOA3 rotation problem, it sounds like you already explored all these options. can you share one of the converted models so I can see the shadow problem?

to rotate a certain object, i found that first 3 floats in a vertex chunk do represent x-y-z coordinates, and i've build a program to manipulate these values to rotate the object at 90 degrees. for example to rotate the object 90 degrees on the Y axis i do swap the X with the Z values, now the object is mirrored and i do flip the X value.

i don't touch the UV info and the normals info, i think those values somehow contain that lighting information, but i don't know how can i handle it

yes, I've already attempted to reassemble the objects based on the CAT.

not not all objects are placed through block1, only half of the model is placed (static only)
the other half is positioned in block4, but the positioning is off. importing additional data from block4
shows that there are some vertex lists that correspond with the positioning. but I am not yet
sure how to correct the objects global position.

this may be far fetched, but I don't believe these is a skeleton at all. its a node based system, so the joints that are missing
in the images below are the bones.






EDIT
@ganjul


1.) do XPRs store dds textures? Can we just dump those sections and add a header or something?
yeah the textures are stored in the XPR file, after you read the table you'll have enough info to create new DDS headers.

its very easy



2.) So the int is the address where I should jump to? I don't get the idea behind it, what's in the space between where I am and where I must jump to?
no the int is just an ID, like the start of the file where it says "XPR " well the ID 00 00 00 80 is just an ID.. its not an offset. the following INT is the offset to the table.

the table is actually before the vertex buffer. you'll of course have to jump back to the start of the file since the buffer is past the obj headers

3.) Resource tables?
at address 0x10 from the start of the XPR, is the table offset.
I call it a resource table because it lists off the offsets to each texture and vertex buffer

read the offset at 0x10, and jump to the table. and parse the texture tables. they are also ID driven, simular to like the ID 00000080 telling the engine to jump.

4.) what's the 0x00000000 in the end for?
I assume that the terminator to end your read.. either way its identifying the value 0.. I doubt thats of any use

5.) does he mean 4 dwords, or 4 IVBuffer structs, which themselves have 4 dwords?
he lists 8 structs of long[4]
so do this eight times over
[LONG] IVBufferIndex1
[LONG] IVBufferIndex1
[LONG] IVBufferIndex1
[LONG] IVBufferIndex1

but the first one is the only thing with data in it.

Thanks mario.
Sorry, I edited my post with another question after you had posted. Can you answer that too?

no the int is just an ID, like the start of the file where it says "XPR " well the ID 00 00 00 80 is just an ID.. its not an offset. the following INT is the offset to the table.

It says the following int is the size of the model. Again maybe I dont understand what you mean by "table". Too much terms are used for the same thing, do you mean chunk/entry or the vertex buffer?

EDIT: How long is a LONG?

EDIT2:

at address 0x10 from the start of the XPR, is the table offset.
I call it a resource table because it lists off the offsets to each texture and vertex buffer

Woa, man, now you confuse me even more. I see no mention of that in the docs. The index to vertex buffers is stored in the Object entries. What?

?what how the file flows?

MDL
>OBJ
>>MTL
>>MTL
>>MTL
>OBJ
>>MTL
>OBJ
>OBJ
>OBJ
>>MTL
>>MTL


.... something like that, each object houses it own materials. then the control file {CAT} forces these into a common material..
thats why I said before there are less like materials, and more like elements

If each object can have multiple materials, how do you know which are his and how many? The material count is specified for the Model, not the Object. The Object entry doesn't have any info for that.

Also, i edited by previous post with questions. Sorry, I'll stop doing that.

2mariokart64n
i can see from your research that you've put a lot of effort in what you're doing. i think you have a good experience in things like that and you're very skilled in exploring the file formats. i want to give you an advice from my own experience - did you heard something like "everything that gets on the pc, gets broke"? this is the power of the pc! and you should use it. the idea is to use the debugger "ollydbg" and disassembler "ida"(illegally this one), feed them the executable and this will make you're live much easier. in ollydbg you can stop the program when a certain file is loaded and watch how the game itself is handling a certain value, to reconstruct the format specs from the "first hands"! these two programs can give you answers that you'll never find by just blindly exploring the file values.
the "team ninja" crew are professional programmers, which means they are developing and using fully functional and complex code, which means the game formats support more stuff than is actually used, for example in doau xpr formats are used only fvf(flexible vertex format) of types 1 and 2, and in doa3 types 1 and 3, and some are never used, also some dds(direct draw surface) types are never used in the game files but the game can handle them because the developers used the XDK(Xbox Development Kit) to interact with textures, which added compatibility for all dds formats(not really sure about that ). this way you can make mod the skins using "unspecified" format abilities. the idea is that by using the debugger and disassembler you'll get more professional, i.e. you just can reverse engineer and grab from the game the actual code for handling the format. i hope i convinced you

I wish I knew dis-assembly but I don't have any experience lol


EDIT

1.) you confuse me even more. I see no mention of that in the docs.

ok lol

this gets confusing, your not dealing with just one binary format here. your dealing with two
the XPR is like a Jar of jelly beans! both this Jar and the jelly beans are constructed differently
but to get all the red jelly beans you have to work your way down the jar. or well just dump the whole jar out!

The XPR is working in this same context, we're working with two different binary structures, one is inside the other.
you have the XPR structure, then the Structure of every component inside the XPR which are all different

I think this is where your getting confused the ID:0x00000080 is another strucutre

[LONG] FileID 0x00000080
[LONG] Size

the size is the entire file, excluding the tables and the buffer.
imagine that the XPR is only holding tables and buffers.. so what if theres models in there, the XPR is handling binary data

00000080 is a XPR instruction, which isn't critical to importing, you can ignore this structure and continue reading the file.


EDIT2
I've put time into creating this public spread sheet of the DOA2U format so far.
[Google Doc]

if you have knowledge of the formats, please have a look at my spread sheet and correct any mistakes in the format, or the spreadsheet layout

-mariokart64n


EDIT3
I'm making good progress on the 4th bloc of the cat file.

the data contains what seems to be welding information of the joining meshes. I'm going to assume since I never actually found vertex weights that there are NONE!
there are however meshes that have them, but only selective meshes.. commonly only the thighs..

back in the year 2000 when you wanted to rig a character you had to build nodes or envelopes throughout your model. These nodes would attach to a skeleton separately, so when your skeleton animated these "Physique" envelopes would deform your mesh. And later when 3dsmax adaptered vertex weights, it seemed for some reason not possible to convert a Physique rigged character to a Skinned character.

My guess is that DOA does the same, its using a method to bend or deform a mesh without the use for vertex weights. <_< which is f***ing DUMB!
because they're not using vertex weight control, they have to weld, and freeze certain vertices. that way when the mesh deforms its not breaking at seams or separating from the other mesh objects..

I say again!!... YOU ARE F$#@ING RETARDED TEAMNINJA, What the!?!?!?

anyway the data itself is useless ?? but I can use the data to reposition the last parts of the objects into their correct place!

means we can finally assemble the DOA2U models in a automated process!!

now only the physics section to solve, however I'm not overly concerned about it. since its not "critical" to the model.
at my current stance I could re-author new models to work in DOA2U.

I know this is the 3d/2d models section, but since mariokart wants to document all the formats of the game, I thought I'd let him know:

The ADX format used for sound effects and voice is not an in-house one. The specs can be found here:
from http://wiki.multimedia.cx/index.php?title=CRI_ADX_file

I wrote a python scanner to see if they are indeed the same format. It seems like it, though get pretty high values in some places.
http://pastebin.com/Gg9kTULM

Searching some more I found a program for converting between ADX and WAV files.
http://www.gamefront.com/files/17335991 ... er_1.8.rar
http://www.speedyshare.com/files/244837 ... er_1.8.rar

EDIT: The AFS format is also documented. http://wiki.xentax.com/index.php/GRAF:AFS_AFS

I wrote a python script which dumps files inside an AFS to a new folder.
http://pastebin.com/K28RdNvP

The format is very simple.