Page 15 of 135
Re: Dead or Alive series formats and tools
Posted: Sat Jan 14, 2012 1:29 pm
by mariokart64n
gjinka wrote:There seems to be only 1 type of OBJ chunk documented. You guys say there are multiple and they store different things, you also say they can have multiple vertex/index buffers. How do you know how much? I think it would be simpler to just document the different types of the OBJ chunk.
I was asking these to b0ny but he suggested to wait until the format will be documented in the wiki, I kind of doubt it will, you guys just moved to other formats, which is great, but it would be better if the formats were documented on the way, people could create scripts for other modelling packages and stuff. I don't think it's too much to ask.
to my knowledge there is only one type of obj block.
between DOAU and DOA3 the material block is extended. that's the only difference.
the vertex type flag doesn't mean its a all new obj block, nothing changes except the stride of a vertex definition.
again I've only dealt with one type, anywho
here's a sniplet from my maxscript, you can see that when I encounter a new vertex type I just skip more lol
Code: Select all
vx=readfloat fstream
vy=readfloat fstream
vz=readfloat fstream
nx=readfloat fstream
ny=readfloat fstream
nz=readfloat fstream
tu=readfloat fstream
tv=readfloat fstream
if vtype==1 do fseek fstream 4 #seek_cur
if vtype==2 do fseek fstream 8 #seek_cur
if vtype==3 do fseek fstream 12 #seek_cur
you could even use the vertex type as a actual stride size..
something like;
stride = ((4*vtype)+32)
^,^
Re: Dead or Alive series formats and tools
Posted: Sat Jan 14, 2012 1:40 pm
by b0ny
i had a look in some scripts and wrote this import script.
io_import_xpr.zip
it will load only the vertices from the first or the second vertex buffer for each OBJ:
Code: Select all
file = open(realpath, 'rb') # Universal read
print('Importing %s' % realpath)
try:
dwXprMagic, dwXPRSize, dwHeaderSize, dwChunkID = struct.unpack("<4sLLL", file.read(16))
except:
print("Error parsing file header!")
file.close()
return
# Handle XPR
if dwXprMagic == b'XPR0':#is "xpr"
if dwChunkID == 0x80000000:
print("mdl ID")#is first chunk a mdl one
dwMDLSize, dwMDLMagic, dwNumObj, dwNumTxt, dwUnk0, dwNumIVBuf = struct.unpack("<1L4sLLLL", file.read(24))
if dwMDLMagic == b'MDL\0':#is magic "mdl"
print("mdl magic. objects %d"% dwNumObj)
objects = []
for object in range(dwNumObj): objects.append(struct.unpack("<1L", file.read(4))[0])
# Handle OBJ
for object in range(len(objects)):
file.seek(objects[object])
dwOBJMagic = struct.unpack("<4s", file.read(4))[0]
if dwOBJMagic == b'OBJ\0':
print("%x obj magic"% (objects[object]))
verts = []
faces = []
indices = []
normals = []
diffuse = []
uv = []
#load first vertex buffer
file.seek(28, 1)
dwNumVertices, dwVBufOffset, dwNumIndices, dwIBufOffset = struct.unpack("<LLLL", file.read(16))
buffer = 0
#if first vertex buffer not found load the second one
if dwNumVertices == 0:
dwNumVertices, dwVBufOffset, dwNumIndices, dwIBufOffset = struct.unpack("<LLLL", file.read(16))
buffer = 1
#go to buffer offset and read the fvf
file.seek(dwIBufOffset)
for i in range(dwNumIndices): indices.append(struct.unpack("<1H", file.read(2))[0])#H - usigned short
for i in range(dwNumIndices - 2): faces.append((indices[i], indices[i + 1], indices[i + 2]))
file.seek(dwVBufOffset)
scale = 3
for v in range(dwNumVertices):
vx, vy, vz = struct.unpack("<fff", file.read(12))
verts.append( (scale*vx, -scale*vz, scale*vy) )
if buffer == 0:
normals.append(struct.unpack("<3f", file.read(12)))
else:
diffuse.append(struct.unpack("<f", file.read(4))[0])
uv.append(struct.unpack("<2f", file.read(8)))
#create object from verts and faces
name = "vtf%d"% (object + 100)
me = bpy.data.meshes.new(name)
me.from_pydata(verts, [], faces)
me.update()
#add new obj to scene
scn = bpy.context.scene
ob = bpy.data.objects.new(name, me)
scn.objects.link(ob)
scn.objects.active = ob
#go to edit mode and fix the mesh
bpy.ops.object.mode_set(mode='EDIT')
bpy.ops.mesh.select_all(action='SELECT')
bpy.ops.mesh.normals_make_consistent(inside=False)
bpy.ops.object.editmode_toggle()
yet have no idea how to handle anything else like material, textures, weight, normals or uv
i think now i'll look in the cat file for the "obj types"(head, body,...), and in the skeleton for coordinates and will place the body-parts to their places
Re: Dead or Alive series formats and tools
Posted: Sat Jan 14, 2012 1:47 pm
by mariokart64n
not everything is listed in the first block.
the first block contains a bunch of node linking info.. which we believed positioned the various objects to the skeleton.
however not everything is listed or assigned.
some things are revisited in other blocks. its complicated :\
Re: Dead or Alive series formats and tools
Posted: Sat Jan 14, 2012 4:41 pm
by b0ny
mariokart64n wrote:not everything is listed in the first block.
the first block contains a bunch of node linking info.. which we believed positioned the various objects to the skeleton.
however not everything is listed or assigned.
some things are revisited in other blocks. its complicated :\
the point zero in the objects coordinates is used as a pivot point and as a bone connection, and is also shared by the joint attached to this body-part.
in the cat file we find the the info about which OBJs are shadow shapes, low poly, high poly and joints; that allows as to assemble almost all objs to a model(by moving them to the points listed in the skeleton file).
the morphs can be loaded as time keyframes to make an animation. and i'm wondering if the "apa" magic block contains the lips speech info. and the physics can be imported as a hierarchy and interpreted by some python script for the blender game engine
i'm very impressed by this gta script:
http://www.gtagarage.com/mods/show.php?id=9798
download it and look in it's manual - i think the doa script should be something like that. all non model info is available as an editable hierarchy. a script like this is like a fully-functional editor that will allow as to make complex skins from the scratch, without using an existing model as a base.
[Edit]
in the
skeleton files(used by the cat file), the bones coordinates are relative to the parent bones, i've made a hierarchy list to calculate the real coordinates. but this hierarchy list doesn't contain branches for all bones
Code: Select all
2-ass
|_6-lheap
| |_7-lknee
| |_3-lfoot
|
|_c-rheap
| |_d-rknee
| |_9-rfoot
|
|_0-chest
|_1-head
|
|_8-lshoulder
| |_4-lelbow
| |_5-lpalm
|
|_e-rshoulder
|_a-relbow
|_b-rpalm
i added some lines to the script and it now looks for "skeletons" folder, and for cat file in the same directory as the xpr file we want to import, and use the information from these files to setup the high poly model with the joints.
xpr script.7z
this model was setup automatically by the script:
Re: Dead or Alive series formats and tools
Posted: Sun Jan 15, 2012 6:08 pm
by mariokart64n
can you get DOAX skeletons? I can send you the XBE
Re: Dead or Alive series formats and tools
Posted: Sun Jan 15, 2012 7:41 pm
by b0ny
mariokart64n wrote:can you get DOAX skeletons? I can send you the XBE
send me the executable, i'll have a look...
you did a good job in researching "apv" morphing format. can you please research also the "apa" morphing format in doau cat files, and the "LNK" morphing format in doa3 cat files. i'd like to try to convert the doa3 morphing to doau but don't have enough experience and knowledge to do this myself.
Re: Dead or Alive series formats and tools
Posted: Sun Jan 15, 2012 8:18 pm
by mariokart64n
apa is morph data? thats strange that there would be anything outside of the face, hands and chest
Re: Dead or Alive series formats and tools
Posted: Sun Jan 15, 2012 9:12 pm
by gjinka
Oh, a Blender script, thats enough for me.
btw b0ny, do you still have questions about Blender Python? I'm still learning the new version of Blender, but maybe I could help.
Re: Dead or Alive series formats and tools
Posted: Mon Jan 16, 2012 12:59 am
by mariokart64n
the APA controls which morph target does what i think. and its hard to break it down, but there are basically 3 listings
1.) Hands
2.) Eyes(Blinking)
3.) Mouth
breasts morphs don't seem to be listed here. I deleted this whole bloc and they still bounced <_< soo...
anyways I think? this is the structure... but I don't know about the indinces, or the IDs given???..
Either way it is going to be hard to map out this section and what all the IDs actually mean?
above you should see an image breaking the bloc into sections, I'll now attempt to describe each section
61 70 61 00
File Id "APA "
03 00 00 00
File Count, always 3??
2C 05 00 00 4C 05 00 00 74 05 00 00
3 Offsets that jump from the start of cat bloc1 to each APA subHeader
1.) Hands
00 00
01 01
0A 04
0D 05
06 08
03 0B
Index #1 [MorphID,TargetID]
01 02 02 03
05 02 06 07
08 02 09 0A
0B 02 0C 0D
FF 00 00 00
SubIndex #1 From second Table?? larger then count, donno what this is
00 00 00 00 08 05 00 00
04 00 00 00 08 05 00 00
SubTable From SubHeader #1 [count,offset]
offset from start of cat bloc1, jumps to another index
06 00 00 00 FC 04 00 00
02 00 00 00 1C 05 00 00
SubHeader #1 [Count,Offset]
following the two offsets they bring us to Index #1 and Table #1 from earlier in this bloc
0E 00 00 00
SubHeader #1 ID?
2.) Eyes(Blinking)
00 00
01 01
02 04
65 02
66 03
00 00
Index #2 [MorphID,TargetID]
05 00 00 00 40 05 00 00
00 00 00 00 00 00 00 00
SubHeader #2 [Count,Offset]
the count and offset are 0 for the 2nd entry, thus the second table is not seen above
05 00 00 00
SubHeader #2 ID ??
3.) Mouth
00 00
01 00
02 01
03 01
04 03
06 02
05 07
07 06
65 04
66 05
Index # 3 [MorphID,TargetID]
0A 00 00 00 60 05 00 00
00 00 00 00 00 00 00 00
SubHeader #3 [Count,Offset]
the count and offset are 0 for the 2nd entry, thus the second table is not seen above
05 00 00 00
SubHeader #3 ID ? ?
EDIT1
I've been looking at the first morph listing (Hands)
seems the model contains more morph targets then are actually used ingame
I'll list as I find them out, these might not even be consistent?
Anyway changing the ID changes both hands.. so I'm not sure why the hands have 2 different lists ??
Hand IDs (Partial)
0x00 - Neutral
0x01 - Fist
0x02
0x03 - Cliffed Fingers
0x04
0x05
0x06 - Fingers Out
0x06
0x07
0x08
0x09
0x0A - Middle Finger Arched
0x0B
0x0C
0x0D - Finger Gun
0x0E
Eye IDs (Partial)
0x00 - Neutral
0x01 - Eyes Closed
0x02 - Wink
0x65 - Eyes Closed (Sad)
0x66 - Eyes Open (Sad)
Mouth IDs (Partial)
**note index from list did not matchup with my dump.
example I have 12 face morphs total, 6 are eyes morphs and the other 6 are mouth morphs
my morph index is obviously 0 - 12, however the game doesn't index from 0 - 12, it indexes from 0 - 6
..the eye and mouth morphs for the face are in two different sets. I'm unsure how to seperate the face morphs into these 2 categories
>>>index resets to 0
0x00 / 0x01 - Neutral / Mouth Closed
0x02 / 0x03 - Mouth Open
0x04 - LowerLip Down
0x05 - Smile
0x06 - Teeth Clenched
0x07 - Mouth Oval
Re: Dead or Alive series formats and tools
Posted: Tue Jan 17, 2012 2:28 pm
by b0ny
doax skeletons have exactly the same format as doau skeletons(both games use the same engine),
but in doau the doax bones are not used(are null).
i added these null bones to the doa3 skeleton and now it's also in doau format(can be inserted in doau cat file).
here is the archive:
skeletons.7z
both doau and doax games contain the same model - "prarie dog". it's great that i can now compare the differences between doau and doax format to make a converter from doax to doau
gjinka wrote:Oh, a Blender script, thats enough for me.
btw b0ny, do you still have questions about Blender Python? I'm still learning the new version of Blender, but maybe I could help.
yes, i have a lot of questions on python importing functions. in this script i only imported the coordinates from the vertex buffers, but i'd like also to now how to import the materials, uvmapping, normals and textures
Re: Dead or Alive series formats and tools
Posted: Tue Jan 17, 2012 4:21 pm
by gjinka
1. You make textures like this:
Code: Select all
image = bpy.data.images.new(name, width, height, alpha = 1)
image.pixels = pixels
# save in blend
image.pack(as_png = True)
texture = bpy.data.textures.new(name, type = 'IMAGE')
texture.image = image
texture.use_alpha = True
Where "pixels" is a python list of rgbargbargbargba... color values. The color values have to be a float for some reason, so if you have an int in the range (0,255), divide it by 255.
2. Make materials like this:
Code: Select all
material = bpy.data.materials.new('materialname')
# ambient
material.mirror_color = ambient[:-1]
material.ambient = ambient[3]
# diffuse
material.diffuse_color = diffuse[:-1]
material.alpha = diffuse[3]
# specular
material.specular_color = specular[:-1]
# emissive
material.emit = emissive[3]
# shininess
material.specular_hardness = hardness
# assign textures to materials
mtex = material.texture_slots.add()
mtex.texture = textures[index] # textures is a list of texture objects you created above
mtex.texture_coords = 'UV'
3. As for assigning the materials to faces, do this:
Code: Select all
for material in materials:
mesh.materials.append(material)
Code: Select all
for face in mesh.faces[fromindex: toindex]:
face.material_index = yourindex
4. For UVs:
Code: Select all
uvtex = mesh.uv_textures.new()
uvtex.name = 'DefaultUVMap'
for face in len(mesh.faces):
uvtex.data[face].uv1 = uvs[face * 3]
uvtex.data[face].uv2 = uvs[face * 3 + 1]
uvtex.data[face].uv3 = uvs[face * 3 + 2]
# 3 if you have triangles
where "uvs" is a python list of uv values stored one after another.
5. For normals:
Code: Select all
index = 0
for vertex in mesh.vertices:
vertex.normal = normals[vindex]
index += 1
6. For vertex colors:
Code: Select all
mesh.vertex_colors.new(name = 'VertexColor')
index = 0
for face in mesh.vertex_colors[0].data:
face.color1 = colors[index]
face.color2 = colors[index + 1]
face.color3 = colors[index + 2]
index += 3
7. For armatures and bones:
Code: Select all
bpy.ops.object.add(type='ARMATURE', enter_editmode=True)
object = bpy.context.object
object.name = 'ArmatureObj'
armature = object.data
armature.name = 'Armature'
Code: Select all
for i in range(your_bone_count):
bone = armature.edit_bones.new('yourname')
bone.tail = mathutils.Vector([0,0,1])
# is it stored as matrices?
#bone.transform(matrix)
# go to Object mode
bpy.ops.object.mode_set(mode='OBJECT')
That's as much as know. If you'll learn how to assign bone indexes and bone weights, let me know.
Re: Dead or Alive series formats and tools
Posted: Wed Jan 18, 2012 5:26 am
by mariokart64n
yes! finally I understand the DOA3 material block. MAN! O_O
I had structures detailing the doa3 material section before hand. but they called the section a subset?? and didnt give any info on how to read it or what it did..
Then I used the DOA3 Exporter, and it dumps a debug.txt file, which describes all the sections.
basically instead if the materials just giving you a offset to the face buffer and a count. in doa3 they detail EVERY strip.
its like there giving a new index for each degenerate face I think... but thats a total waste of space. they should have just used triangle list.
whatever, finally DOA3, DOAX, and DOAU are cracked once again! now to start working on importing data from the cat
Re: Dead or Alive series formats and tools
Posted: Wed Jan 18, 2012 2:18 pm
by gjinka
Oh nice. I havent had enough time to study the whole blender script, but two things I noticed bony:
1) you can actually store a list of length 0 or 1 in python. Why wouldn't you be able to?
Code: Select all
list.append([]) # add an empty list to a list
# OR
a = []
list.append(a)
list.append([1]) # add a list with 1 member to a list
# OR
a = [1]
list.append(a)
2) instead of writing "BBBBB" for the struct module, you can just write "5B". Even "2B3B" would work.
Re: Dead or Alive series formats and tools
Posted: Wed Jan 18, 2012 7:53 pm
by b0ny
gjinka wrote:Oh nice. I havent had enough time to study the whole blender script, but two things I noticed bony:
1) you can actually store a list of length 0 or 1 in python. Why wouldn't you be able to?
Code: Select all
list.append([]) # add an empty list to a list
# OR
a = []
list.append(a)
list.append([1]) # add a list with 1 member to a list
# OR
a = [1]
list.append(a)
2) instead of writing "BBBBB" for the struct module, you can just write "5B". Even "2B3B" would work.
i spent a lot of time with a "integer has no length" error, till i found this page
http://en.wikibooks.org/wiki/How_to_Thi ... Solution_1(CH 9 - Solution 1) that says that you can't access lists inside a list if it contains lists of length 1(and maybe 0):
Code: Select all
mylist.append((1,2,3))
mylist.append((98,32))
mylist.append((3))
#because of the third "append", you can't do this:
len(mylist[1]) #"integer has no length" error
#or this
myvar = mylist[1][1] #"can't subscript an integer" error (or something like that)
#it's like when you can't do like this:
myvar = struct.unpack('<L', file.read(4))
#you should do that instead
myvar = struct.unpack('<L', file.read(4))[0]
[edit]
i looked in the mot files. a move is like a bunch of offsets that point to some "keyframe" data. i have no idea about the actual animation data but i found that one move have exactly
44 offsets there is what every offset means('3.' - means that there are three offsets):
mot struct(size44):
3.side/ud/fb body
3.fb/rot/side-angle ass
3.fb/rot/side-angle chest
3.fb/rot/side-angle head
3.side/ud/fb back palm pos
3.global back palm rotation
2.? (shoulder+elbow rotation)
3.side/ud/fb forw palm pos
3.global forw palm rotation
2.? (shoulder+elbow rotation)
3.side/ud/fb back foot pos
3.global back foot rotation
2.? (knee+heap rotation)
3.side/ud/fb forw foot pos
3.global forw foot rotation
2.? (knee+heap rotation)
doax animation is different it have 56 'bones' (offsets) and some additional data in the offsets block. possible the animation stream is also different
Re: Dead or Alive series formats and tools
Posted: Wed Jan 18, 2012 9:12 pm
by gjinka
It works for me and I believe it should for you too, unless there is some math paradox I've never heard of.
So what are you doing? Can you post an exact code which crashes with the error message?
Also not that len() is meant for lists/tuples and dictionaries, which can have multiple elements. len() just tells the number of elements. An integer has "one element", its value, so len() on it is pointless.
If you want to use an integer for a while loop, do this instead:
Code: Select all
for i in range(intgervalue):
# do something
Code: Select all
#it's like when you can't do like this:
myvar = struct.unpack('<L', file.read(4))
#you should do that instead
myvar = struct.unpack('<L', file.read(4))[0]
That's because struct.unpack() returns a list, even if it has one value, so you need to do [0] to assign the first value instead of the whole list.
Also, did my Blender code help in any way?