[PC] Devil May Cry 4

[Surveyor]

Hey revelation, nice to see you here!!
I have 010Editor but I keep getting errors and can't run the script!!! but I verified its content and yeah most of the data is correct
So far I've seen only 3 vertex declarations

This is found in stages
0 D3DDECLTYPE_FLOAT3 POSITION
12 D3DDECLTYPE_UBYTE4N NORMAL
16 D3DDECLTYPE_UBYTE4N TANGENT
20 D3DDECLTYPE_FLOAT16_2 TEXCOORD
24 D3DDECLTYPE_FLOAT16_2 TEXCOORD
28 D3DDECLTYPE_FLOAT16_2 TEXCOORD

This is used for bone models
0 D3DDECLTYPE_SHORT4N POSITION
8 D3DDECLTYPE_UBYTE4 BLENDINDICES
12 D3DDECLTYPE_UBYTE4N BLENDWEIGHT
16 D3DDECLTYPE_UBYTE4N NORMAL
20 D3DDECLTYPE_UBYTE4N TANGENT
24 D3DDECLTYPE_FLOAT16_2 TEXCOORD
28 D3DDECLTYPE_FLOAT16_2 TEXCOORD

And the 3rd declaration, makes use of stream1, this is in case verts are influenced by more than 4 bones, very rare though!
0 D3DDECLTYPE_SHORT4N POSITION
8 D3DDECLTYPE_UBYTE4 BLENDINDICES
12 D3DDECLTYPE_UBYTE4 BLENDINDICES index1, null all the times
16 D3DDECLTYPE_UBYTE4N BLENDWEIGHT
20 D3DDECLTYPE_UBYTE4N BLENDWEIGHT index1, null all the time
24 D3DDECLTYPE_UBYTE4N NORMAL
28 D3DDECLTYPE_FLOAT16_2 TEXCOORD

I think we had enough of this MOD format, there is some unknown data that I don't care about, for me, the next steps are skinning and animation
Of course there are some other formats that are very interresting to study like the sdl (schedule) files that drive the game

[revelation]

Ok, cool.

i have found a few more vertex declarations used by ripping with 3D Ripper DX, but i need to catalog them all and compare to the files again. Alot of them have the same format for stream 0 with just a few additional bits of data in stream 1. i just need to catalog how many there actually are and see if i can find some sort of specifier for which is used. i also have breakpoints for the places LMT files are loaded as well so i can start looking into how those are structured if need be. i just like to know how the MOD files interpret some of this remaining data so i am not guessing or hard coding which is used, at least in the long term. As with you and poly i can retrieve the information just fine with what is known now. What problems are you having with skinning? i haven't coded up my skeletal animation code yet, but its something i can look into if there are currently issues. Also, have you found any models that use the additional texture 10 index found at the end of the material? If you could it would be helpful if you could list which textures you have found associated with which layer indices. i have found in the code, layers 5 and 7 seem to be somewhat mutually exclusive, but i am still investigating. i just saw that you had a good deal of the textures for the stages worked out, and knowing the order would be helpfuly. On that note, did you ever get multitexturing working in your stage viewer? Once mine code is up and running i will be writing shaders hopefully able to display things correctly. Also, some of the remaining data may effect how the joint data is processed at runtime, so will be necessary to propery update the skeleton in some cases i believe (unknownData0 and unknownDat1 in the jointData), i am hoping it will not be as involved as the method used in Street Fighter 4 though as i am still working on determining the last bits of the update process.

ResiHax: it is a structure file defining the layout of the MOD file format. it is in a script format use but the 010 Editor hex editor to automatically parse file when they are loaded an present them as structures. The code ss pretty close to c++ though so should be fairly easy to convert.

Hopefully i will have some more information soon.

Thanks again.

[revelation]

Ok, just another quick update.

i believe i have found where it creates the various vertex declarations used in the game, as well as reading the associated indices into them. i also think i have a clearer picture of the values used to determine the start vertex and index buffer values and associated rendering setup. Still a number of unknowns in some areas but for the most part things seems pretty well nailed down to the point that other parts can be delved into. Still need to record all of the 16 possible vertex declarations and compare them to files that index some of them, as well as revisit the code location where i unknowingly found the indices previously, and further investigate the with the new information i have (as the index values seem to be listed twice in the data, so i need to confirm why this would be). Hopefully it will all pan out. Still not sure what things have been found out since the initial code release with regard to what i have posted so far, but i hope to have another update shortly with the new information presented accordingly. Then i hope you all will be able to help confirm or disprove the findings rather quickly i hope, heh.

Either way, back to it i guess.

[shadowmoy]

something like that ?

Code: Select all

Type CGROUPS Align 1 '//lot of infos are based on IDirect3DDevice9::DrawPrimitiveUP params for faster rendering process
	usId As word '// id from the index table //unsigned short  'could be a degenerate face counter ?
	usMatId As word '// material Id //unsigned short
	
	usFlag[0 To 7] As word '// 0xFF01, 0, the 3rd flag determines what vertex declaration to use //unsigned short
	'//usFlag[4] = vertexcount =>  == (usFlag[6]-usFlag[5]+1)
	'//usFlag[5] 'a end offset in vertex array
	'//usFlag[6] 'a start offset in vertex array
	'//usFlag[7]'always zero ????
	
	%uiBase1 '// base multiple of 32 '// lodmesh id ?: full mesh = 0, > are degenerated lodmesh ?
	%uiZero '//unsigned int
	%uiStart '// start index 		//unsigned int   
	%uiCount '// faces (NBindices - 2)	//unsigned int  =primitivecount
	%uiBase '//
	uiInconnu[0 To 5] As word '//			//unsigned int
	'//uiInconnu[1]= first face vertex id A on ABC == usFlag[6] ->strange to store same info twice ???
	'//uiInconnu[4]= an adress (or offset to an adress) perhaps next lod group id ?
End Type

[revelation]

Code: Select all

struct
{
	// 0x00
	uint16    usId0 : 5;              // used as power of 2, tested against value indexed by usId1
	uint16    usId1 : 11;             // used as an index into a list of 32-bit (4-byte) values
	uint16    usMatId;                // material Id
	uint8     usFlag0x04[2];          // [0] - processing skipped if == 0
	                                  // [1] - tested against an unknown bitmask
	uint16    usFlag0x06;             //
	uint8     usFlag0x08[4];          // [2] - unknown value, &= 0xFE if (prev.endVertex0x0E + 1) != startVertex0x2A
	                                  // [2] - &= 0xBF, if 0 == tex4
	uint16    vertexCount0x0C;        // vertex count
	uint16    endVertex0x0E;          // end vertex - last vertex index used, compared to next object.startVertex0x2A
	// 0x10
	uint32    minVertex0x10;          // min vertex index
	uint32    streamOffset0_0x14;     // start offset for vertex stream 0 (in bytes)
	uint32    streamOffset1_0x18;     // start offset for vertex stream 1 (in bytes)
	uint32    startIndex0x1C;         // start index
	// 0x20
	uint32    indexCount0x20;         // index count - faces (indices - 2)
	uint32    uiBase0x24;             //
	uint8     unknown0x28[2];         // [0], [1] - indices into vertex declarations
	uint16    startVertex0x2A;        // start vertex, compared to previous object.endVertex0x0E + 1
	uint8     unknown0x2C[4];         // [0] - unknownData count, number of unknownData blocks assigned
	                                  // [1] - if 0x98 == file version, set to ((usFlag0x08[3] >> 1) & 0x0F)
	// 0x30
	uint32    unknown0x30 <format = hex>;    // seems to be unused, overwritten with unknownData start address
} objectData[fileHeader.objectCount];

Still finalizing some values, and i will post the vertex declarations used as well when i get the chance. Hopefully this will help verify and find some other things in the format. i still need to go back through some of the remaining flag values and fins where it specifies which of the 2 vertex declaration indices to use. The first usually appears to be the bare bones stream 0 format, the second contains more data or slightly different data layout and is usually the one that contains information used with vertex stream 1. Also trying to nail down the material flags that specify the initial vertex declaration information. This is also more than like the specifiers for the shaders used, but for now i can only directly tie them to the vertex declarations without digging deeper into some of the directx structures.

[revelation]

ok, getting a little busy with some things, so i wanted to go ahead and post some of th einfo i have so far in hopes that those able to make sens of it will be able to use it accordingly and move things even further forward.

i am still working on determining the purpose of most of the values in the flags fields but i have commented at least the most important one concerning this update, the values used to determine the vertex declaration(s) used. As i mentioned previously this is probably also a custom material class used by the engine so it probably has shader references as well, but i have not put breakpoints at the other direct3d functions with regard to this yet.

Code: Select all

struct
{
	// 0x00
	union
	{
		uint32 flags0x00 <format = hex>; // material flags
		struct
		{
			uint32 unknown0x00_00 : 2;
			uint32 unknown0x00_02 : 12;
			uint32 unknown0x00_0E : 8;
			uint32 unknown0x00_16 : 3;
			uint32 unknown0x00_19 : 2;
			uint32 unknown0x00_1B : 3; // vertex declaration formats specifier (0 - 5)
			uint32 unknown0x00_1E : 2; // multiplied by 16, xor'ed with unknown value and and'ed against 0x10
		} flagBits;
	} unknownFlags0x00;
	union
	{
		uint32 flags0x00 <format = hex>; // material flags
		struct
		{
			uint32 unknown0x00_00 : 4;
			uint32 unknown0x00_04 : 2;
			uint32 unknown0x00_06 : 4;
			uint32 unknown0x00_0A : 4;
			uint32 unknown0x00_0E : 4;
			uint32 unknown0x00_12 : 4;
			uint32 unknown0x00_16 : 4;
			uint32 unknown0x00_1A : 6;
		} flagBits;
	} unknownFlags0x04;
	uint32 uiUnknown0x08 <format = hex>; // material id? - replace with an index value after comparison
	uint32 uiUnknown0x0C <format = hex>;
	// 0x10
	uint32 uiUnknown0x10 <format = hex>; // overwritten with first vertex declaration address
	uint32 uiUnknown0x14 <format = hex>; // overwritten with second vertex declaration address
	// 0x18
	uint32 uiLayers[9];		// tex0 ... tex8 -  base 1, 0 means not bound
	// 0x3C
	float fAlpha; // material alpha? - used as float4(1,1,1,fAlpha) in game code
	// 0x40
	float3 fUnknown0x40;
	float fUnknown0x4C;
	// 0x50
	float4 fUnknown0x50;	// [3] - ignored?, assumed to be 0
	// 0x60
	float2 fUnknown0x60;
	float fUnknown0x68;		// ignored?
	float fUnknown0x6C;
	// 0x70
	float4 fUnknown0x70;
	// 0x80
	float4 fUnknown0x80;	// [2] - if < 0, = -1, else =1
							// [3] - ignored?
	// 0x90
	uint32 unknown0x90 <format = hex>;		// some sort of flags/masked values
	uint32 unknown0x94;						// index
	uint32 unknown0x98;
	uint32 uiLayer0x9C;						// tex10
} materialData[fileHeader.materialCount];

here is the way the vertex declaration value is used when modifying the material information. Each material keeps the addresses of 2 of these declaration structures. i am still in the process of determining how it decides which one to use for each object. Also since declarations 0 and 5 are the same the objects will usually use 5 in the index values if you look at the files themselves, but so far these values have not been touched again in the code once determined from the object's material index.

Code: Select all

format = unknownFlags0x00.unknown0x00_1B
if 0 == format
- vertex declaration 5
- vertex declaration 0
if 1 == format
- vertex declaration 7
- vertex declaration 2
if 2 == format
- vertex declaration 6
- vertex declaration 1
if 4 == format
- vertex declaration 10
- vertex declaration 11
if 5 == format
- vertex declaration 5
- vertex declaration 12
if 3 == format
- vertex declaration 6
- if 1 == unknownFlags0x04.unknown0x00_16
- - vertex declaration 9
- else
- - vertex declaration 8

and here are the data values used in all of the vertex declaration created through the course of the game so far. The information above determines which are selected, so some of them are not all that important to file formats, those not being used often pertaining to other visual effects not present in the file, like some of the water rendering, etc. Soory, like i said, i have not coded things up into a viewer yet, so i have not yet taken the time to relate everything back into values relevant to direct3d, but they are simply the values inorder used to populate a D3DVERTEXELEMENT9 structure array for use in CreateVertexDeclaration calls. If i happen to do this before anyone else gets to it i will post things in code form.

Code: Select all

// vertex declarations

// 0EE78680 - [00] - same as [05]
 0,  0, 10,  0,  0,  0
 0,  8,  5,  0,  2,  0
 0, 12,  8,  0,  1,  0
 0, 16,  8,  0,  3,  0
 0, 20,  8,  0,  6,  0
 0, 24, 15,  0,  5,  0
 0, 28, 15,  0,  5,  1
FF,  0, 17,  0,  0,  0

// 0EE787C0 - [01]
 0,  0,  2,  0,  0,  0
 0, 12,  8,  0,  3,  0
 0, 16,  8,  0,  6,  0
 0, 20, 15,  0,  5,  0
 0, 24, 15,  0,  5,  1
 0, 28, 15,  0,  5,  2
 1,  0, 15,  0,  5,  3
FF,  0, 17,  0,  0,  0

// 0EE78860 - [02]
 0,  0, 10,  0,  0,  0
 0,  8,  5,  0,  2,  0
 0, 12,  5,  0,  2,  1
 0, 16,  8,  0,  1,  0
 0, 20,  8,  0,  1,  1
 0, 24,  8,  0,  3,  0
 0, 28, 15,  0,  5,  0
 1,  0, 15,  0,  5,  1
 1,  4,  8,  0,  6,  0
FF,  0, 17,  0,  0,  0

// 0EE782A0 - [03]
 0,  0,  2,  0,  0,  0
 0, 12,  4,  0, 10,  0
 0, 16,  1,  0,  5,  0
FF,  0, 17,  0,  0,  0

// 0EE78330 - [04]
 0,  0,  2,  0,  0,  0
 0, 12,  4,  0, 10,  0
 0, 16,  1,  0,  5,  0
 0, 24,  1,  0,  5,  1
FF,  0, 17,  0,  0,  0

// 0EE78680 - [05] - same as [00]
 0,  0, 10,  0,  0,  0
 0,  8,  5,  0,  2,  0
 0, 12,  8,  0,  1,  0
 0, 16,  8,  0,  3,  0
 0, 20,  8,  0,  6,  0
 0, 24, 15,  0,  5,  0
 0, 28, 15,  0,  5,  1
FF,  0, 17,  0,  0,  0

// 0EE78570 - [06]
 0,  0,  2,  0,  0,  0
 0, 12,  8,  0,  3,  0
 0, 16,  8,  0,  6,  0
 0, 20, 15,  0,  5,  0
 0, 24, 15,  0,  5,  1
 0, 28, 15,  0,  5,  2
FF,  0, 17,  0,  0,  0

// 0EE78720 - [07]
 0,  0, 10,  0,  0,  0
 0,  8,  5,  0,  2,  0
 0, 12,  5,  0,  2,  1
 0, 16,  8,  0,  1,  0
 0, 20,  8,  0,  1,  1
 0, 24,  8,  0,  3,  0
 0, 28, 15,  0,  5,  0
FF,  0, 17,  0,  0,  0

// 0EE78910 - [08]
 0,  0,  2,  0,  0,  0
 0, 12,  8,  0,  3,  0
 0, 16,  8,  0,  6,  0
 0, 20, 15,  0,  5,  0
 0, 24, 15,  0,  5,  1
 0, 28,  4,  0, 10,  0
FF,  0, 17,  0,  0,  0

// 0EE789B0 - [09]
 0,  2,  0,  0,  0,  0
 0, 12,  8,  0,  3,  0
 0, 16,  8,  0,  6,  0
 0, 20, 15,  0,  5,  0
 0, 24, 15,  0,  5,  1
 0, 28,  4,  0, 10,  0
 1,  0, 15,  0,  5,  2
FF,  0, 17,  0,  0,  0

// 0EE78BB0 - [10]
 0,  0, 10,  0,  0,  0
 0,  8,  5,  0,  2,  0
 0, 12,  8,  0,  1,  0
 0, 16,  8,  0,  3,  0
 0, 20,  5,  0,  5,  0
 0, 24, 15,  0,  5,  1
 0, 28, 15,  0,  5,  2
FF,  0, 17,  0,  0,  0

// 0EE78C50 - [11]
 0,  0, 10,  0,  0,  0
 0,  8,  5,  0,  2,  0
 0, 12,  8,  0,  1,  0
 0, 16,  8,  0,  3,  0
 0, 20,  5,  0,  5,  0
 0, 24, 15,  0,  5,  1
 0, 28, 15,  0,  5,  2
 1,  0,  8,  0,  6,  0
FF,  0, 17,  0,  0,  0

// 0EE78D00 - [12]
 0,  0, 10,  0,  0,  0
 0,  8,  5,  0,  2,  0
 0, 12,  8,  0,  1,  0
 0, 16,  8,  0,  3,  0
 0, 20,  8,  0,  6,  0
 0, 24, 15,  0,  5,  0
 0, 28,  4,  0, 10,  0
FF,  0, 17,  0,  0,  0

// 0EE783D0 - [13]
 0,  0,  2,  0,  0,  0
 0, 12,  5,  0,  2,  0
 0, 16,  5,  0,  2,  1
 0, 20,  8,  0,  1,  0
 0, 24,  8,  0,  1,  1
 0, 28,  0,  0,  3,  0
 0, 32,  0,  0,  6,  0
 0, 36,  9,  0,  5,  0
 0, 40,  9,  0,  5,  1
 0, 44,  9,  0,  5,  2
 0, 48,  9,  0,  5,  3
 0, 52,  4,  0, 10,  0
FF,  0, 17,  0,  0,  0

// 0EE78A50 - [14]
 0,  0, 10,  0,  0,  1
 0,  8,  5,  0,  2,  0
 0, 12,  8,  0,  1,  0
 0, 24, 15,  0,  5,  0
 0, 28, 15,  0,  5,  1
 2,  0, 10,  0,  0,  2
 2,  8,  8,  0,  3,  0
 2, 12,  8,  0,  6,  0
FF,  0, 17,  0,  0,  0

// 0EE78B00 - [15]
 0,  0, 10,  0,  0,  1
 0,  8,  5,  0,  2,  0
 0, 12,  8,  0,  1,  0
 0, 24, 15,  0,  5,  0
 0, 28,  4,  0, 10,  0
 2,  0, 10,  0,  0,  0
 2,  8,  8,  0,  3,  0
 2, 12,  8,  0,  6,  0
FF,  0, 17,  0,  0,  0

// 0EE78490 - [16]
 0,  0,  0,  0,  0,  0
FF,  0, 17,  0,  0,  0

// 0EE785E0 - [?0]
 0,  0,  3,  0,  0,  0
 0, 16,  3,  0,  2,  0
 0, 32,  3,  0,  2,  1
 0, 48,  3,  0,  1,  0
 0, 64,  3,  0,  1,  1
 0, 80,  3,  0,  3,  0
 0, 96,  3,  0,  6,  0
 0, 112,  3,  0,  5,  0
 0, 128,  3,  0,  5,  1
 0, 144,  3,  0,  5,  2
 0, 160,  3,  0,  5,  3
 0, 176,  3,  0, 10,  0
 0, 192,  3,  0,  7,  0
FF,  0, 17,  0,  0,  0

// 0EE78520 - [?1] - empty?
 0,  0,  0,  0,  0,  0
FF,  0, 17,  0,  0,  0



-------------------------------------
// 0EE78DA0 - [?2]
 0,  0,  2,  0,  0,  0
 0, 12,  4,  0, 10,  0
 0, 16, 10,  0,  5,  0
 0, 24,  5,  0,  5,  1
 0, 28,  5,  0,  5,  2
FF,  0, 17,  0,  0,  0

// 0EE78E40 - [?3]
 0,  0,  2,  0,  0,  0
 0, 12,  4,  0, 10,  0
 0, 16, 10,  0,  5,  0
 0, 24,  9,  0,  5,  1
FF,  0, 17,  0,  0,  0

// 0F800680 - [?4]
 0,  0,  2,  0,  0,  0
 0, 12,  1,  0,  5,  0
 0, 20,  2,  0,  5,  1
 0, 32,  2,  0,  5,  2
 0, 44,  2,  0,  3,  0
FF,  0, 17,  0,  0,  0

// 0F800720 - [?5]
 0,  0,  2,  0,  0,  0
FF,  0, 17,  0,  0,  0

Let me know if there is anything you need clarified. i'll report when i have any new findings.

Hope this helps.

[Surveyor]

Hi,
It was about time I release something so here's my new MOD to Max converter, this is a whole new approach: the program will generate a max script file to run directly inside 3dsmax. I have max8 and I think this'll work on all the later versions.
All you have to do is run the program, type in the model you want, go to max, type F11, select File->RunScript and choose your freshly created MS file. It'll take some time before the model fully loads into max. On my celeron 2.00Ghz 768Mb ram nvidia fx5200 that bugs all the time, this takes about 4 minutes for each model
You'll also need to increase the maxscript memory: go to Customize -> Preferences -> MaxScript tab and change the memory amount, the default is 7.5Mb, I changed it to 150Mb and that seemed to work fine on all the models I tested!!
The main feature of this new converter is the full support for bone/skinning on Dmc4 & Lostplanet. But for the rest (texture coords, normals) you'll have to wait. The program also fixes the pancake effect, now all the models are nicely scaled!
You can experience the normal bug on any object, just select an objet, go to edit mesh and notice how normals become normal (correct)
Caution: to use only with bone models, don't try this on stages, items, ....

Now for the technical stuff,
As you know in MOD files, all objects share the same vertex buffer, so I had 2 choices: export objects per vertex buffer basis in which case I get only one object per model and an export per object basis so I get all the objects separated. Each method has it's advantages and disadvantages.
Exporting to OBJ was the easiest way to get the job done: I wrote the whole vertex buffer then went through all the objects and wrote the indices. The 1st approach is exactly what I did with OBJ files, but in max script there isn't any face group separator so all the faces belong to one group and I get only one object per model.
Well the reason Capcom decided to use one vertex buffer for the model is obvious: performance. They also arranged the vert's to make use of the stripping, DirectX Sdk Docs say you should use D3DPT_TRIANGLESTRIP whenever possible and you should minimise the SetVertexBuffer() and SetIndexBuffer() calls to gain performance, 1 SetVertexBuffer() call per model draw is really the minimum. The vertex buffer can hold different vertex types but this doesn't matter as long as you know what vertex declaration to use at a given draw. It's the first time I see such a system so the 1st conclusion is: Capcom did a hell of a job to optimize their data.
The problem with this method is that it is virtually impossible to strip a mesh without duplicating some verts but in the case of MOD files, this is too exagerated: there are way too many duplicate verts: more than 30 identical successive verts can be seen in hex editor and there are not few of these duplications!!! This is seen in the hi res models and as an example I took Nero hi res head model that says there are 49613 verts and 12184 indices. This is relatively very high res for any model and this is only the head!!!
When I click RunScript under max for this head model, there is the "unknwon exception" error in max that simply means that there are way too many verts for a single object. This makes me wonder what is the maximum vertex count an object can have in maxscript. The 2nd conclusion is: Capcom models aren't that optimized!!!
The 2nd approach exports per object basis. Under max script, every mesh must have it's set of verts, meshes can't share some vertex buffer. The 1st problem with this is that on hi res head models, there are too many objects, capcom could've made the head in a sngle object; I wonder why they didn't!!
Another problem with this approach is that on export, verts are rearranged the index buffer way, these index buffers are far from being optimized and there are some duplicate indices here too!!! which leads to duplcate verts on the export!!!! But at least this method doesn't crash maxscript and all the models load OK! This is themoethod used in the converter posted here.

Here are some numbers:
with Mod2Obj I get this:
verts: 5305, faces: 5804, when the verts of this model are weld, I get this:
verts: 2917, faces: 5804, the fact that the face count didn't change shows that there are indeed many duplicate verts. Now with the new converter:
verts: 11935, faces: 11846 and with weld we have:
verts: 2928, faces: 5804
Even if we consider that there are some duplicate verts for a reason (eyelashes for example have 2 sides so usually the artists make the same mesh that points upwards point downwards, this of course leads to duplication), this doesn't exceed 10% of the initial vertex count!!! But here we have a serious duplication problem (from 49613 verts originally to 11935 verts in the worst case). My 3rd conclusion is: Capcom engineers must be drunk to come up with such a nonsense!!! no wonder Lostplanet doesn't perform well even on Core 2 Duo processors!!!

Well all this was just to say that depending on what will be done with the converter (reimport back to game, make renders, torture the poor guy, ...), this may or may not pose problems. Me, I'd like to make a proper Direct3D viewer out of this so this shouldn't annoy me, on the other hand, there is the normals bug that makes my nerves burn!!!!
OK enjoy this new converter and let me know if there are any bugs!

[DMCdesigns]

So is there a way to use this to mod the game now? And can you make it work with Maya?

[AceAngel]

@Surveyor: Wow, amazing work so far mate. I'll be sure to report ant bugs that might come up.

@DMCdesigns: Don't you think we should wait for a clean version of the script of this version first BEFORE asking for other 3DS Packages Scripts?

[shadowmoy]

ok seems we need some infos , normal bug is not a bug, the normals you import are dynamic normals based on the skeleton pose, they are recalculated dynamicly by the engine ,differently than max do (they need to be rotated by the inverse matrix of the vertex associated joint and normalized after.

the vertex duplication can also be explain easily, when welding vertices, the weld is based on vertex pos and not on uv mapping, so if you weld only this way , the rendering using vertex arrays and indices will completly fuck up the model mapping, the only way to fix it is to add degenerated faces or to duplicate vertices, degenerated faces could not be used here because they optimised the strip indices to get a mutlires mesh strip array, i encountered the same problem while coding my xandreale 3d modeler and getting wrong normals or fucked uv's, then i added a model cleaner that rebuild the mesh by duplicating some vertex an it fixed the problem on uv...

so there is no bug, we just need to add a lot more computations on the whole mesh before exporting and this is another story ...

[pixellegolas]

this is some a+ stuff right here. Good work and so on

[revelation]

to follow on what has been said so far, the main reasons for the vertex duplication are more than likely tri-strip length, uv coordinates and batch size. All of which affect things differently as it can be very hard to get good tri-strip length when having to also keep uv information intact, even more so with multitexturing where it is not always possible for the textures mapped to use the same uv coordinates (as is very evident given the variation of the vertex declarations). Lightmaps and Ambient occlusion maps are often big hindrances in this area as well since they do not usually allow for vertices being mapped to the same texel in an image as is possible with the other texture maps used. As far as batch count goes, even on today's hardware there are not any cards i know of that have a recommended vertex batch size of anywhere near 49000+ vertices as such even if it was all contained in one object it would not be performance friendly to render the whole thing as such. But at this point i can only speculate as to why capcom chose to go in this direction, heh.

It would be really nice if one or both of either opengl and/or direct3d added support for multiple index streams similar to the way the gamecube/wii works. Would come in handy for things like this as it would pretty much eliminate the need for duplicate vertices in most cases, and possible cut down a great deal on repeated data in others. oh well.

How many objects were you getting on the model(s) you were testing?

Once i get things setup in a better way to extract things more efficiently i'll have to look into this more to compare against your findings.

EDIT:
Wow, you weren't kidding. i'll have to attempt rendering this model, but if i am looking at the same one you mentioned, those really are some small batches. Will have to play around with these files some.

[poly]

Hi,
It was about time I release something so here's my new MOD to Max converter, this is a whole new approach: the program will generate a max script file to run directly inside 3dsmax. I have max8 and I think this'll work on all the later versions.
All you have to do is run the program, type in the model you want, go to max, type F11, select File->RunScript and choose your freshly created MS file. It'll take some time before the model fully loads into max. On my celeron 2.00Ghz 768Mb ram nvidia fx5200 that bugs all the time, this takes about 4 minutes for each model
You'll also need to increase the maxscript memory: go to Customize -> Preferences -> MaxScript tab and change the memory amount, the default is 7.5Mb, I changed it to 150Mb and that seemed to work fine on all the models I tested!!
The main feature of this new converter is the full support for bone/skinning on Dmc4 & Lostplanet. But for the rest (texture coords, normals) you'll have to wait. The program also fixes the pancake effect, now all the models are nicely scaled!
You can experience the normal bug on any object, just select an objet, go to edit mesh and notice how normals become normal (correct)
Caution: to use only with bone models, don't try this on stages, items, ....

Now for the technical stuff,
As you know in MOD files, all objects share the same vertex buffer, so I had 2 choices: export objects per vertex buffer basis in which case I get only one object per model and an export per object basis so I get all the objects separated. Each method has it's advantages and disadvantages.
Exporting to OBJ was the easiest way to get the job done: I wrote the whole vertex buffer then went through all the objects and wrote the indices. The 1st approach is exactly what I did with OBJ files, but in max script there isn't any face group separator so all the faces belong to one group and I get only one object per model.
Well the reason Capcom decided to use one vertex buffer for the model is obvious: performance. They also arranged the vert's to make use of the stripping, DirectX Sdk Docs say you should use D3DPT_TRIANGLESTRIP whenever possible and you should minimise the SetVertexBuffer() and SetIndexBuffer() calls to gain performance, 1 SetVertexBuffer() call per model draw is really the minimum. The vertex buffer can hold different vertex types but this doesn't matter as long as you know what vertex declaration to use at a given draw. It's the first time I see such a system so the 1st conclusion is: Capcom did a hell of a job to optimize their data.
The problem with this method is that it is virtually impossible to strip a mesh without duplicating some verts but in the case of MOD files, this is too exagerated: there are way too many duplicate verts: more than 30 identical successive verts can be seen in hex editor and there are not few of these duplications!!! This is seen in the hi res models and as an example I took Nero hi res head model that says there are 49613 verts and 12184 indices. This is relatively very high res for any model and this is only the head!!!
When I click RunScript under max for this head model, there is the "unknwon exception" error in max that simply means that there are way too many verts for a single object. This makes me wonder what is the maximum vertex count an object can have in maxscript. The 2nd conclusion is: Capcom models aren't that optimized!!!
The 2nd approach exports per object basis. Under max script, every mesh must have it's set of verts, meshes can't share some vertex buffer. The 1st problem with this is that on hi res head models, there are too many objects, capcom could've made the head in a sngle object; I wonder why they didn't!!
Another problem with this approach is that on export, verts are rearranged the index buffer way, these index buffers are far from being optimized and there are some duplicate indices here too!!! which leads to duplcate verts on the export!!!! But at least this method doesn't crash maxscript and all the models load OK! This is themoethod used in the converter posted here.

Here are some numbers:
with Mod2Obj I get this:
verts: 5305, faces: 5804, when the verts of this model are weld, I get this:
verts: 2917, faces: 5804, the fact that the face count didn't change shows that there are indeed many duplicate verts. Now with the new converter:
verts: 11935, faces: 11846 and with weld we have:
verts: 2928, faces: 5804
Even if we consider that there are some duplicate verts for a reason (eyelashes for example have 2 sides so usually the artists make the same mesh that points upwards point downwards, this of course leads to duplication), this doesn't exceed 10% of the initial vertex count!!! But here we have a serious duplication problem (from 49613 verts originally to 11935 verts in the worst case). My 3rd conclusion is: Capcom engineers must be drunk to come up with such a nonsense!!! no wonder Lostplanet doesn't perform well even on Core 2 Duo processors!!!

Well all this was just to say that depending on what will be done with the converter (reimport back to game, make renders, torture the poor guy, ...), this may or may not pose problems. Me, I'd like to make a proper Direct3D viewer out of this so this shouldn't annoy me, on the other hand, there is the normals bug that makes my nerves burn!!!!
OK enjoy this new converter and let me know if there are any bugs!

Hey Surveyor!
So you finally cracked the skinning, awesome!

Would you mind sharing the C++ code so that I can try to develop my COLLADA exporter? Using COLLADA should allow for all DDC application to edit the models (max/xsi/maya).

[revelation]

Finally got around to writing some simple code to preview MOD files. Not texturing or shaders yet, so hopefully you won't mind the over saturated default direct3d lighting, heh. Things seems to load pretty well with no modifications between DMC4 and RE5, haven't tried LP yet. Hopefully i'll have material, shaders, lighting, and skinning support up and running at some point in the near future. i plan on rewriting this framework in a much more manageable way seeing as i have worked on a number of other formats i have not even coded up for viewing yet.

[DMCdesigns]

@revelation: Have you flipped Nero's model or your tool is loading it flipped?