Forza Motorsport Resource Extraction (.carbin)

[revelation]

i'll update this post in a little bit with the various vertex declaration information. i have also disassembled the shaders, so i may be able to give some insights on the conversion calculation. Haven't updated my code for the changes in the carbin format yet though. And the vertices appear to simply be SHORT4N format, so simply divide by 32767.0f. The conversion seems similar, but there has been a slight change, but i need to sit down and update my carbin info, so if you have any more info on that it might speed up the process.

EDIT:
Actually seems like it is simply another scale placed in the part position:

fm4 v16
vertexPosition.xyz = (position.xyz * (position.w * pack_partPosition.w)) + pack_partPosition.xyz

instead of:

fm3 v11
vertexPosition.xyz = (position.xyz * position.w) + pack_partPosition.xyz

Haven't tried it yet though.

[Ernegien]

I won't be able to check this until later tonight but I thought that min/max xyz bounding information followed immediately after the part offsets and no w component was present. You are however correct (or wrong with me :p) about the vertex format it seems. I'll also try to update this post later tonight with structure info. I do appreciate the input though, hopefully things will get ironed out soon


EDIT: Below contains a loose layout of the two most important structures in .carbin files. Also keep in mind that the game reads this as a stream so some components might not be present depending on the counts...

Code: Select all

Forza4CarPart
{
	int type;

	// applied to all vertices belonging to this part
	float xOffset;
	float yOffset;
	float zOffset;
	
	float xBoundMin;
	float yBoundMin;
	float zBoundMin;
	float xBoundMax;
	float yBoundMax;
	float zBoundMax;

	float fltUnk1;
	float fltUnk2;	// possibly rotations in ypr or quaternion form?
	float fltUnk3;
	float fltUnk4;
	float fltUnk5;

	int padding1;
	int padding2;
	int damageVertexCount;
	float[][] damageVertices[damageVertexCount][4];	// xyzw

	int padding3;
	int damageIndexCount;
	short[] damageIndexCount[damageIndexCount];

	int padding4;
	int padding5;
	int padding6;	// Forza 4 Addition
	
	byte nameLength;
	char[] name[nameLength];

	// additional info for lod1-lod5 .carbin (unconfirmed layout for forza 4)
	int padding7;
	int lodVertexCount;
	int lodVertexSize;
	byte[][] lodVertices[lodVertexCount][lodVertexSize];
	
	int padding8;
	int subPartCount;
	SubPart subParts[subpartCount];

	// lod0 vertex information
	int padding9;
	int vertexCount;
	int vertexSize;
	byte[][] lodVertices[vertexCount][vertexSize];
	
	// forza 4 additions
	int padding10;
	byte padding11;	// this threw me off a bit...
	int padding12;
	int padding13;
	int padding14;
	int padding15;

	// extra per-vertex information it seems, haven't verified if it's always per-vertex though
	int unkCount;
	int unkSize;
	byte[][] unkData[unkCount][unkSize];
}

SubPart
{
	int type;
	byte padding;

	// forza 4 additions
	float[] fltUnknowns[8];
	int[] intUnknowns[8];	// can't remember what datatype this was, I just know the size...

	int padding2;
	int nameLength;
	char[] name[nameLength];

	int lod;	// what lod this sub-part belongs to (if applicable)
	int intUnk;
	int intUnk2;
	int intUnk3;
	int intUnk4;
	int intUnk5;
	int intUnk6;
	int intUnk7;
	float fltUnk;
	float fltUnk2;
	float fltUnk3;
	float fltUnk4;
	float fltUnk5;
	float fltUnk6;
	float fltUnk7;
	float fltUnk8;
	int intUnk8;

	// these indices reference the parent part's vertices (no vertex data per sub-piece)
	int indexCount;
	int indexSize;
	byte[][] indices[indexCount][indexSize];
	
	int padding2;
}

[revelation]

Yeah, that is what i thought too. i am simply going by what i see in the shader files at the moment. There are some other calculations using the bounding box offset and scale, that i have not fully worked through yet to see what it affects or is used for. Since they also have values for morphing between damage states, have to track down the values they are using and see where they fit in.

The fxobj file have both the shader bytecode and the vertex declarations, so it should be accurate in that regard. But i guess they cannot really show any preprocessing steps that might be occurring in the code. Unfortunately the carbin format does not have many identifying markers that i can easily search for in the code to find the loading procedures. Has been a while since i originally worked with the fm3 format, so i will see what i can do.

[bigBear]

Why did they changed the scale?
It doesn't make sense at all.

[Ernegien]

I would assume they changed it to better accommodate the extra precision needed in their models...

[bigBear]

Ok, that might be that.

[Ernegien]

Stepping away for a bit seemed to have worked this time . The answer, as it turns out, is quite straightforward. They've given us bounding box information for each part, so all we need to do is calculate the current bounding information by iterating over each vertex and then figuring out the per-component ratio between the two which will be used to scale it back up to the original intended size. I somehow had it in my head that they would have stored this scale value somewhere in the part data (they still might, but I can't seem to find it) so they wouldn't need to manually calculate it each time, but I was obviously wrong. As you can see, I still need to add proper normals among other things, but the hard part should now be over...

[MMWsmokey]

amazing work, thank you so much ernegien & everyone who helped

[D1Racer]

Wow that's great, Awesome work!

[toolieo]

Awesome progress.


Pretty neat that FM4 isn't released yet and yet we will have your forza studio updated to support FM4 cars.

[bigBear]

Really great progress, guys!
Really!
The official full release date of full game will be out on October 11th.
And now you already have succeeded to crack the FM4 carbin!
You made history haha.


Energien, I assume the zip of FM4 is the same as the zip from FM3?

[toolieo]

Energien, I assume the zip of FM4 is the same as the zip from FM3?

As much as I am not Energien.


I can answer that though, yes the ZIP format is the same as Forza 3. When I looked inside of FM4 cars I just used the zip script for QuickBMS.

[bigBear]

Energien, I assume the zip of FM4 is the same as the zip from FM3?

As much as I am not Energien.


I can answer that though, yes the ZIP format is the same as Forza 3. When I looked inside of FM4 cars I just used the zip script for QuickBMS.

Ah, but of course.
I should have known that.
I have read the demo have limited features, so I didn't planned to download the demo.
But you have anwered and thank you

[Ernegien]

i'll update this post in a little bit with the various vertex declaration information. i have also disassembled the shaders, so i may be able to give some insights on the conversion calculation. Haven't updated my code for the changes in the carbin format yet though. And the vertices appear to simply be SHORT4N format, so simply divide by 32767.0f. The conversion seems similar, but there has been a slight change, but i need to sit down and update my carbin info, so if you have any more info on that it might speed up the process.

EDIT:
Actually seems like it is simply another scale placed in the part position:

fm4 v16
vertexPosition.xyz = (position.xyz * (position.w * pack_partPosition.w)) + pack_partPosition.xyz

instead of:

fm3 v11
vertexPosition.xyz = (position.xyz * position.w) + pack_partPosition.xyz

Haven't tried it yet though.

Any notes on the normal format yet? it's not 101010, 1010102, or some of the others I've tried...

[revelation]

Oops, sorry about that.

The new format does not store the normals explicitly. They store a quaternion representing the orthonormal tangent basis transform. So i believe you just convert that to a matrix and you will have the row vectors you need for the tangent, bitangent, and normals.

Code: Select all

fm3 v11

Stream  Offset  DeclType                Method  DeclUsage                   UsageIndex      Name
0       0       D3DDECLTYPE_FLOAT16_4   0       D3DDECLUSAGE_POSITION (0)   0               scaledPosition
0       8       D3DDECLTYPE_FLOAT16_2   0       D3DDECLUSAGE_TEXCOORD (5)   0               uv
0       12      D3DDECLTYPE_FLOAT16_2   0       D3DDECLUSAGE_TEXCOORD (5)   1               uv2
0       16      D3DDECLTYPE_HEND3N      0       D3DDECLUSAGE_NORMAL   (3)   0               normalOS
0       20      D3DDECLTYPE_SHORT4N     0       D3DDECLUSAGE_TEXCOORD (5)   2               SH0
0       28      D3DDECLTYPE_HEND3N      0       D3DDECLUSAGE_TEXCOORD (5)   8               damageDelta
0       32      D3DDECLTYPE_HEND3N      0       D3DDECLUSAGE_TEXCOORD (5)   9               damageNormalDelta
0       36      D3DDECLTYPE_HEND3N      0       D3DDECLUSAGE_NORMAL   (3)   1               tangentOS

-----------------------------------------------------------------------------------------

fm4 v16

0       0       D3DDECLTYPE_SHORT4N     0       D3DDECLUSAGE_POSITION (0)   0               scaledPosition
0       8       D3DDECLTYPE_USHORT2N    0       D3DDECLUSAGE_TEXCOORD (5)   0               uv
0       12      D3DDECLTYPE_USHORT2N    0       D3DDECLUSAGE_TEXCOORD (5)   1               uv2
0       16      D3DDECLTYPE_SHORT4N     0       D3DDECLUSAGE_TEXCOORD (5)   2               TanFrameQuat
0       24      D3DDECLTYPE_SHORT4N     0       D3DDECLUSAGE_TEXCOORD (5)   3               SH0