Page 1 of 3

ccs file format... part 3

Posted: Mon Feb 11, 2008 7:05 pm
by Forte
*Sigh* This just doesn't end.

I figured out how to get the textures, how the archive is structured and I even know where the models are, but I can't read them.
Here's what I think is a model (cut out from the archive).

The archive refers to this as "MDL" and it's the biggest file in there (apart from the texture).

The first 24h bytes seem to be the header. If it's the "barrier" I think it is, it's nothing more than a big box or plane but reading the bytes as float or double doesn't produce any sensible results, so I think this is not actually a list of vertices? What else could it be?("list of vertices" is the only thing I know -.-)

Re: ccs file format... part 3

Posted: Thu Jan 01, 2009 1:23 am
by Forte
I've decided to work on it some more so if someone has the time, Here's the summary of everything I know about this file format (last page explains my problem).

Re: ccs file format... part 3

Posted: Fri Jan 08, 2010 5:07 am
by Satoh
BREAK in the file. I have managed to take your assumtion on the CCCC0800 chunk you suggested might be the MDL info, and decipher what I believe to be XYZ and UV data. IE: Verteces and texture coordinates

Look at this file, starting at offset 0x00000268, if I'm correct, the first three bytes are in fact X, Y, Z, followed by U, and V texture coords, followed by a 00 byte and then the next XYZUV.

Granted this file is from //Quarantine, since it's all I had with me when I was looking through. Also, I've determined that MDL_'s are not simply models, but individual mesh objects contained within a single model.

This file is a standard katana like the Kotetsu Sword. I have ripped this type of sword from PCSX2, and know it's basic shape and construction... Unfortunately the ripped models are heavily distorted due to the way the camera's depth is rendered... so the ripped models are largely useless-BUT, they give us an accurate vertex/polycount to work with.

As far as I've seen from the comparison between the cmp and tmp, they seem to be the exact same format, all of the data in the documentation you made fits with the cmp. This is what leads me to believe this is a relevant discovery.


Also, each 'object' listed under the initial file list, contains a trailing byte that determines exactly which file it belongs to. This byte is found exactly two bytes before the start of the next 'object' name. This byte is a number equal to the files as they are listed in order. This numbering system begins with 01, NOT 00.

All objects are listed once for every file they relate to, such as the OBJ_ objects in character models. These I've determined are bones, as they are listed once for the model file (../max/filename.max) and each of the animation files (../anm/animnam.max)


All I really need to do now is plot the numbers in a modeling program, though I'm not sure how yet, and then compare what I learned to the GU series files for final verification. I hesitate to say it, but I think I may have cracked my very first format...

Re: ccs file format... part 3

Posted: Wed Jan 20, 2010 12:46 am
by FurryFan
Satoh wrote:BREAK in the file. I have managed to take your assumtion on the CCCC0800 chunk you suggested might be the MDL info, and decipher what I believe to be XYZ and UV data. IE: Verteces and texture coordinates

Look at this file, starting at offset 0x00000268, if I'm correct, the first three bytes are in fact X, Y, Z, followed by U, and V texture coords, followed by a 00 byte and then the next XYZUV.

Granted this file is from //Quarantine, since it's all I had with me when I was looking through. Also, I've determined that MDL_'s are not simply models, but individual mesh objects contained within a single model.

This file is a standard katana like the Kotetsu Sword. I have ripped this type of sword from PCSX2, and know it's basic shape and construction... Unfortunately the ripped models are heavily distorted due to the way the camera's depth is rendered... so the ripped models are largely useless-BUT, they give us an accurate vertex/polycount to work with.

As far as I've seen from the comparison between the cmp and tmp, they seem to be the exact same format, all of the data in the documentation you made fits with the cmp. This is what leads me to believe this is a relevant discovery.


Also, each 'object' listed under the initial file list, contains a trailing byte that determines exactly which file it belongs to. This byte is found exactly two bytes before the start of the next 'object' name. This byte is a number equal to the files as they are listed in order. This numbering system begins with 01, NOT 00.

All objects are listed once for every file they relate to, such as the OBJ_ objects in character models. These I've determined are bones, as they are listed once for the model file (../max/filename.max) and each of the animation files (../anm/animnam.max)


All I really need to do now is plot the numbers in a modeling program, though I'm not sure how yet, and then compare what I learned to the GU series files for final verification. I hesitate to say it, but I think I may have cracked my very first format...
I read you post and you are right about a few things, but to clarify:
The vertexes do begin at off set 268, but those are only the vertexes there. Four bytes back there is B6 or 182, which is the number of vertexes.
The vertexes like most PS1 games, and some PS2 games are 32 Bit Signed Integer. Which means they can take both Positive and Negative Values. So its
2ByteX, 2ByteY, 2ByteZ for each vertex. Most games (but not this one) that use the 32 Bit Signed, have a null 00 00 after each vertex to make it 8 bytes for better processing.
Here is a quick 2d plot of the vertexes of the sword in Excel
Image
and the Excel spread sheet that provides a rough conversion of 32bitsigned to decimal.
http://ps23dformat.wikispaces.com/file/view/2BYTESS.xls

Re: ccs file format... part 3

Posted: Wed Jan 27, 2010 3:05 am
by Chueco
I'm in the process of writing up a document on what all I've found on the CCS file format, as well as a program that can open and view the models/textures from them.

I was just wondering, have you figured out how the triangles are stored yet?

Re: ccs file format... part 3

Posted: Wed Jan 27, 2010 9:12 pm
by FurryFan
Chueco wrote:I'm in the process of writing up a document on what all I've found on the CCS file format, as well as a program that can open and view the models/textures from them.

I was just wondering, have you figured out how the triangles are stored yet?
Okay, I'm not 100% sure about this, but if this is like other games I've seen then what I am saying is correct:
After all of the 6byte vertex array, there is an 4byte array for each vertex. The fourth byte in each array is the "Connection" byte which only takes on the values of either 00 or 01.
If the Connection byte is 01 then you connect that vertex with the vertexes right before it and right after it.
If the Connection byte is 00 then do not connect this vertex with the next vertex.
So the Model is sets of Tristripes with separated with either double end zeros ( ....00 ...... 00).
I think you still connect the Connections with only one zero, you only do not connect if there are 2.
A similar tristrip model format is used in 'Godzilla Unleashed" (but with floats for the vertexes).
I want to show you the mesh, but I need to find a way to batch convert the Signed integers into floats so that Ogre3d/Blender can read them.

Re: ccs file format... part 3

Posted: Thu Jan 28, 2010 6:43 am
by Chueco
Yup, that's it. took me forever to figure it out though. I wrote a giant python script which reads the CMP file and converts the meshes to a plain text format which I then import into 3dsMax with a custom maxscript. If I had blender I could probably whip up an importer which could take a CMP file directly.

Re: ccs file format... part 3

Posted: Sat Feb 13, 2010 3:14 pm
by Forte
Awesome, and thanks for PMing me Satoh.

tmp is indeed almost the same as cmp and the triangles/quads (tmp uses both) are stored similarly to what FurryFan described, but instead the "connection" byte can be 00, 01 or 02 (and possibly other values):
02, 02, 00 = triangle using these vertices
02, 02, 00, 00 = quad(/two triangles) using these vertices
01, 01, 00, 00 = quad(/two triangles) using these vertices, but inverted order

Characters and other big models seem to have a slightly different format, but I haven't really looked into it yet.
Other stuff is looking good though, like the arena's monitor:

Image

Re: ccs file format... part 3

Posted: Sun Feb 14, 2010 5:45 pm
by Satoh
ZOMG Why did I stop getting my email alerts on this thread!!? There's been REAL progress and NOW you stop sending the alerts!? Xentax you cruel cruel woman! :constipated:



Anyway, this is absolutely awesome...

So, it seems we can grab some scenery objects, yes? What about weapons, they should be fairly simple... at least they seem un-complex compared to characters. Also, by any chance could I get my hands on those aforementioned scripts?

I know even less about writing import export scripts than I do about file exploration.

Either way, I'll have to look back through the files and see what I can learn with this new info...

Forte wrote:Awesome, and thanks for PMing me Satoh.
No problem. I didn't want to keep any info to myself since I'm so bad at this type of thing XD

Re: ccs file format... part 3

Posted: Mon Feb 15, 2010 1:53 am
by Forte
Yeah, most weapons work fine, too.
It also looks like there's more data for each vertex, but I can't even tell if it's somehow UV related or not. A simple file would be again barrier.tmp (added as attachment):

0008CCCC is located at 0x0D88, amount of vertices(=34) is at 0x0DB4 and after xyz (6 * 34 bytes), there's the connection thing (by the way, any ideas what the other three bytes of that stand for?), size being 34*4 bytes.
After that, there's 34*4 bytes of some data mostly consisting of the values 64, 128, etc and at the end, there're 34*4 bytes again, but they don't make sense to me either <_<

By the way, here's a simple program to extract the meshes as obj files:
http://www.forte.spacequadrat.de/upload/CCS2OBJ.rar

If it shows "File xxx processed" but you don't see any obj files, it most likely means that the program encountered some kind of error while trying to parse the data.

Edit: So yeah, the last block of data is indeed texture coordinates, but I can't figure that out exactly.
So there's two bytes for u and two bytes for v, but how can I convert them to float values that make sense? It works fine for weapons because they only use one byte per u/v component, so dividing that value by 256 gives me good floats, with the overall result being:

Image

But barrier, for example, uses both bytes for u and v (tex coords start at 0x0F94) but they don't make sense no matter how I convert them (without breaking the weapon's uv that is)

Re: ccs file format... part 3

Posted: Tue Feb 16, 2010 1:49 am
by Satoh
well... it hardly seems necessary to ask, but are there any preceding or trailing bytes between the UV coords... or are there unknown UV header segments?

If so I'd bet that's what determines the UV map type... it's probably different for static objects... most games use different formats for scenery vs. active objects... Though I would have expected the files to be more different in this case...


Also I should remind that the Barrier object had an animated texture... I've seen games that store animated tex's in the models instead of using code to change them... So that's another possibility...

Tried looking at the viewscreen UV's? I doubt it'll have animated textures... Either way I suggest investigating some other model...but I can't think of a perfect example right off.


And now I'ma check out this OBJ converter...

Re: ccs file format... part 3

Posted: Tue Feb 16, 2010 4:01 am
by chrrox
Sounds like half floats you can convert half floats with
http://www.fox-toolkit.org/ftp/fasthalf ... ersion.pdf

Re: ccs file format... part 3

Posted: Tue Feb 16, 2010 12:19 pm
by Forte
Eh, damn. They do use the normal u(and v) = (byte_0 / 256 + byte_1) format, but I always viewed them through a program I made which had a problem with texture coordinates being over 1.0, barrier and (most) others works fine with blender for example.

I've updated the obj extractor to include texture coordinates in the obj and extract the textures as well. (still doesn't work properly because some MDLs are unhandled)

2ova01.tmp has a pretty big MDL chunk at 0x135B0, but the vertex count is nowhere to be found(it doesn't seem to be 337 (value at 0x135F0))

Re: ccs file format... part 3

Posted: Tue Feb 16, 2010 10:38 pm
by Satoh
Forte wrote:Eh, damn. They do use the normal u(and v) = (byte_0 / 256 + byte_1) format, but I always viewed them through a program I made which had a problem with texture coordinates being over 1.0, barrier and (most) others works fine with blender for example.

I've updated the obj extractor to include texture coordinates in the obj and extract the textures as well. (still doesn't work properly because some MDLs are unhandled)

2ova01.tmp has a pretty big MDL chunk at 0x135B0, but the vertex count is nowhere to be found(it doesn't seem to be 337 (value at 0x135F0))
I recently dealt with Sonic Adventure DX's models for the PC, they had an interesting structure that when exported to OBJ, left each section of verts weighted to a bone separated as individual objects... (Meaning the elbows would be separate objects from wrists and shoulders, etc etc...)

Also, looking through the IMOQ character files, it seems each body part has a separate mdl that are all linked together in some way...

This could be the same with Ovan there. It could be that the character model header has more info than other models, including a bone count, character identifier, any other extra data characters may have...

I can't imagine what 337 could be referring to though...

EDIT:
However! Ovan has several soft body rigged clothing pieces, it could be that those are handled in there as well... for instance it could be that 337 is the weight or falloff of the softbody rig, or that it refers to root verts that don't get displaced by soft rigging... any number of things... I would suggest using someone like Bordeaux who doesn't have that kind of clothing...

EDIT: Also the UV dump function seems to be a bit off...
Image
That doesn't look like w2dgn00's texture at all... (and it should since UVs are dropped on top of textures...)
Also, a lot of textures aren't dumping at all.

Also EDIT:

It seems the UV's on some are dumping correctly, and some textures are as well... However, it seems the UV's are dumping incorrectly but in similar shapes to the objects they represent... It could be that there is an issue separating the UVs into different meshes and they are getting massively distorted because of it... I've been dumping directly from CCS, I'll try TMP dumping shortly.

Also also EDIT: TMP dump made no difference...

Re: ccs file format... part 3

Posted: Wed Feb 17, 2010 6:59 pm
by Forte
The problem isn't really what 337 stands for (I've ignored almost all values in the header of each mdl and they still come out fine), but how to read the chunk. And every character/avatar has that strange structure 2ova01 has.
I'm expecting the vertex count at 0x135D8 (start of chunk + 0x24), but instead there's some huge int (look like indices to something) and the whole format seems to be different than other mdls' anyway.

Also, most of the bugs are fixed and the program will now extract every non-character/avatar/etc obj and every texture(although separately for now). (there was a problem with trailing zeros if the vertex count * 6 is not a multiple of 4, which messed up about everything)