Important information: this site is currently scheduled to go offline indefinitely by end of the year.

Tekken Hybrid

The Original Forum. Game archives, full of resources. How to open them? Get help here.
gjinka
Banned
Posts: 95
Joined: Sun Apr 10, 2011 5:30 pm
Been thanked: 18 times

Re: my first attempt at file format exploring (Tekken 3 form

Post by gjinka »

OK, convinced :D

(why do you think it will become more complicated in the future though? More shader programming? I don't see anything revolutionary new in the new DirectXs and OpenGLs)

Anyway, we should post a new topic in the 3D section. I could change the topic title, but there is no archive format to figure out. There are tools out there to extract PKGs.

And I'm a proud Python user. I usually use my game engine for testing, but I can use the old Blender too. I don't know anything about Noesis python library. I couldn't find any docs.
MrAdults
Moderator
Posts: 1007
Joined: Mon Mar 23, 2009 2:57 am
Has thanked: 44 times
Been thanked: 505 times

Re: my first attempt at file format exploring (Tekken 3 form

Post by MrAdults »

gjinka wrote:(why do you think it will become more complicated in the future though? More shader programming? I don't see anything revolutionary new in the new DirectXs and OpenGLs)
It's already possible to use custom data types/sets (to some extent, it isn't exactly practical yet) with the programmable shader pipelines we have today, but it will become far more practical and desirable as our data sets continue to grow and dedicated graphics hardware becomes more versatile. Eventually, it's just going to be a lot more reasonable to scanline/raytrace directly through your data set instead of having to convert it to fixed triangles before you feed it to the GPU.
gjinka wrote:And I'm a proud Python user. I usually use my game engine for testing, but I can use the old Blender too. I don't know anything about Noesis python library. I couldn't find any docs.
Noesis documentation is pretty minimal, it's still on my todo list to make a real set of documents and examples. For now, there's a __NPReadMe.txt file that comes with Noesis containing all of the functions and constants with brief descriptions and parameter definitions, as well as a plethora of example scripts, and chrrox has posted some very useful tutorials in the tutorials section here on Xentax.
gjinka
Banned
Posts: 95
Joined: Sun Apr 10, 2011 5:30 pm
Been thanked: 18 times

Re: my first attempt at file format exploring (Tekken 3 form

Post by gjinka »

If you use docstrings, you can just use doxygen to generate a html documentation.

EDIT:
@howfie: are you sure that's an obvious index buffer? Looks like ASCII character codes in nice order.

I got the obvious stuff:

Code: Select all

Little Endian

85 A8 A4 2D - header
char[8]     - file name?
uint[4]     - always 64?
Can I ask a friendly mod to cut this topic from the part we talk about tekken Hybrid and move it to the 3d section?
howfie
double-veteran
double-veteran
Posts: 929
Joined: Fri Jul 08, 2011 12:06 pm
Location: Torrance, CA
Has thanked: 10 times
Been thanked: 274 times

Re: my first attempt at file format exploring (Tekken 3 form

Post by howfie »

It sure is. That ASCII looking table is the first thing I look for. It says, "Index Buffer, and I'm probably going to be easy to rip." If it's not there in the model file, I usually give up right away. There's one game company that purposely obfuscates their index buffer (was it TTARCH files I don't remember) and as far as I know, no one has ripped models from those games yet.

BTW, python it is then ha ha ha. It's been several years since I've messed with it (using Panda errrr I mean Crap3D), but I'll get used to it again.
Last edited by howfie on Mon Jan 09, 2012 10:48 am, edited 2 times in total.
MrAdults
Moderator
Posts: 1007
Joined: Mon Mar 23, 2009 2:57 am
Has thanked: 44 times
Been thanked: 505 times

Re: my first attempt at file format exploring (Tekken 3 form

Post by MrAdults »

Doxygen isn't useful, as the necessary information and basic examples (outside of their contextual uses in actual scripts) simply don't exist. The Noesis Python API is defined in C, and the above-mentioned .txt file contains all of the info you would otherwise glean from a custom parsing of the source code. As a side note, I hate how cluttered the internet has become with people who just blindly run doxygen over their codebases and call it "documentation", despite the fact that it's completely devoid of meaningful or useful content - I'd rather it didn't exist in the first place, to save me looking through it for something useful. Seems to happen every time I google for documentation on an opensource library these days.
finale00
M-M-M-Monster veteran
M-M-M-Monster veteran
Posts: 2382
Joined: Sat Apr 09, 2011 1:22 am
Has thanked: 170 times
Been thanked: 307 times

Re: my first attempt at file format exploring (Tekken 3 form

Post by finale00 »

gjinka wrote: And I'm a proud Python user. I usually use my game engine for testing, but I can use the old Blender too. I don't know anything about Noesis python library. I couldn't find any docs.
I find myself using only like 5-6 functions in noesis to get most of the stuff done.
Mainly cause I don't know what to do with the rest of the data, but I don't really need to use too many things for simple things like exporting geometry.

But ya, 5-6 functions is enough to get geometry out. Maybe another 2-3 functions would get bones and weights, and maybe another 2-3 to get animations.

You can use this template that I wrote. It provides a simple model loading function based on the example provided and should get most stuff done. Once you realize you need more functions to render more stuff you should be familiar with the library enough.

http://xtsukihime.webs.com/Noesis%20Plu ... emplate.py

This is assuming you're using the rapi function calls. .

You can look at how repetitive the plugins end up being, especially if you start splitting up everything into different functions:

http://xtsukihime.webs.com/noesisplugins.htm

Note: the model template up there is not tested. I don't actually use that one, because it forces me to remember API functions like getting the input name, getting the directory path of the input, etc. It was just me taking the template I use, removing all references to the custom object, and copying the constructor over.

I use this one: http://xtsukihime.webs.com/Noesis%20Plu ... e_sanae.py
Which requires the package in my sig. It's pretty much the same thing except I can add stuff that tailor to my own preferences also in an attempt to write readable code (eg: self.filename vs rapi.getLocalFileName(rapi.getInputName())
howfie
double-veteran
double-veteran
Posts: 929
Joined: Fri Jul 08, 2011 12:06 pm
Location: Torrance, CA
Has thanked: 10 times
Been thanked: 274 times

Re: my first attempt at file format exploring (Tekken 3 form

Post by howfie »

Updated post on first page with some python code. I'll be gone for a couple days and if anyone wants to add stuff gooooooo right ahead. I think the data before the vertex data is the skeleton data. Each 0x40 byte entry contains 3 floats and then some other data... Probably a half-float matrix maybe I dunno. The surface stuff is probably near the top of the file. Sucks it doesn't look like they are given names like hair, head, body, knifepouch, etc.
gjinka
Banned
Posts: 95
Joined: Sun Apr 10, 2011 5:30 pm
Been thanked: 18 times

Re: my first attempt at file format exploring (Tekken 3 form

Post by gjinka »

Good job and thanks.
I'll ask the mods again to cut this topic from howfie's first post and move it to 3d section, or move the whole topic and I'll edit my first post explaining what happened.

@howfie thanks again.
I'll make a dumb student guess. We are reading the whole vertex buffer at once. The models are probably split into multiple meshes, each mesh entry probably mentions the size, offset and type of the vertex and index buffer subpart. There are probably two types: triangle and triangle strip index buffer. As to where these 'entries' are, I don't know. Anyway, just a guess from what I learned in the DOAU topic.
howfie
double-veteran
double-veteran
Posts: 929
Joined: Fri Jul 08, 2011 12:06 pm
Location: Torrance, CA
Has thanked: 10 times
Been thanked: 274 times

Re: my first attempt at file format exploring (Tekken 3 form

Post by howfie »

we can move it ourselves once we're finished.

yes, vertex buffer is one big chunk of data in the file and can be read all at once. and yep, there are more than likely multiple meshes. even though the index buffer is also one big chunk of data in the file too, it is probably divided into parts, typically called surfaces or groups or whatever you want to call them.

so say there are 10,000 indices in the index buffer. the first 1,000 could be for the face. the next 4,000 could be for the body, the next 1,000 for the hair, etc. etc. at the beginning of each group you have to stop the currently running triangle strip. i have a feeling it's at the beginning of the file but I am far from my dev machine on a crappy laptop at the moment he he he. did the python code work for you?
gjinka
Banned
Posts: 95
Joined: Sun Apr 10, 2011 5:30 pm
Been thanked: 18 times

Re: Tekken Hybrid

Post by gjinka »

Yes very likely in the beginning of the file.
Good job on the Python OBJ script.

Sad that the struct module doesnt support half floats. Maybe we could contact the Python devs to add it to the module.

Anyway, yeah, I'm right behind you
Image
viperzerofsx
veteran
Posts: 95
Joined: Thu May 27, 2010 5:07 am
Has thanked: 8 times
Been thanked: 6 times

Re: Tekken Hybrid

Post by viperzerofsx »

doesn't tekken 3 use tmd?
gjinka
Banned
Posts: 95
Joined: Sun Apr 10, 2011 5:30 pm
Been thanked: 18 times

Re: Tekken Hybrid

Post by gjinka »

People haven't even told me how you get files out of PSX ISOs. Even if I would do that the files are in archives. If we could figure out the archive format and the files are really tmd (I have no idea about this format) then maybe we could get them in Milkshape.
finale00
M-M-M-Monster veteran
M-M-M-Monster veteran
Posts: 2382
Joined: Sat Apr 09, 2011 1:22 am
Has thanked: 170 times
Been thanked: 307 times

Re: Tekken Hybrid

Post by finale00 »

gjinka wrote:Yes very likely in the beginning of the file.
Good job on the Python OBJ script.

Sad that the struct module doesnt support half floats. Maybe we could contact the Python devs to add it to the module.

Anyway, yeah, I'm right behind you
Noesis supports half-floats viewtopic.php?p=64094#p64094

:)
howfie
double-veteran
double-veteran
Posts: 929
Joined: Fri Jul 08, 2011 12:06 pm
Location: Torrance, CA
Has thanked: 10 times
Been thanked: 274 times

Re: Tekken Hybrid

Post by howfie »

viperzerofsx wrote:doesn't tekken 3 use tmd?
I don't think he knows how to properly rip PSX games. I think there was a PS2 version of the game too. LOL I don't know how to rip from PSX games either. Don't have a PSOne.

BTW, can somebody upload nina1u.chr sample. I am far from home for another day or two and in order to work on it I need the file. And did anyone check the preview part hybrid? That one is going to be hard to rip from.
MrAdults
Moderator
Posts: 1007
Joined: Mon Mar 23, 2009 2:57 am
Has thanked: 44 times
Been thanked: 505 times

Re: Tekken Hybrid

Post by MrAdults »

Most PSX games just use a plain old ISO 9960 filesystem, so you can pop them into any CD-ROM drive and copy the files right off.

Just like PS2 games, however, some PSX games will store custom filesystem structures on the disc at a given offset, and when that happens it's usually a completely proprietary filesystem that you have to figure out yourself. In that case, I'll typically just rip a direct binary image of the entire disc and sift through that for a filesystem header. This practice is actually more common in PS2 games than it is in PSX games, though, from what I've witnessed.

It's also true that rather than actually looking into the CD-ROM filesystem, lots of PSX games will actually hardcode sector offsets and sizes for each file. Usually in the program binary itself. Which means that even if you modify the filesystem and reburn the disc, the game will still be looking for the data in the same sectors, and will probably fail because of your modification to the disc contents without modifying its internal sector table. Just something to be aware of, doubtful you'll run into issues there unless you want to actually mod games. It's theoretically possible for the CD-ROM filesystem to give you invalid offsets to the files you see on a disc, but I've never seen this actually happen.
Post Reply