Page 23 of 34

Re: Extracting simple models

Posted: Thu Jul 05, 2018 3:14 pm
by onelove1210
shakotay2 wrote:It would make sense if you created a new thread for such because this one as you may have noticed is called "Extracting simple models".
That is what I wondered. Since this file was a bunch of data, not an archive nor geometry-only file, I decided to give it a chance in this thread first. I need your help to know if there were geometry data in that file first. Then if we can figure out the structure of geometry data directly from that bunch, I can get it easier in other files as well as create a new thread with more detailed information :]

Re: Extracting simple models

Posted: Thu Jul 05, 2018 3:28 pm
by Bigchillghost
onelove1210 wrote:I want to know a range or "magic" key (header) of geometry data to search them easier in other files.
How did you backup the game? A memory dump? And are the two files you uploaded before all that you got? What's the game name btw?

Re: Extracting simple models

Posted: Thu Jul 05, 2018 3:34 pm
by onelove1210
Bigchillghost wrote: How did you backup the game? A memory dump? And are the two files you uploaded before all that you got? What's the game name btw?
No I did a backup for the whole machine. Maybe it contained a memory dump. Game's name is 决战!平安京, I have no idea what is the English name of this game because I'm not Chinese. Actually I got a lot of files but I thought those two were the most useful to figure out geometry data based on what I've seen in Hex Viewer.

Re: Extracting simple models

Posted: Thu Jul 05, 2018 3:48 pm
by Bigchillghost
onelove1210 wrote:No I did a backup for the whole machine. Maybe it contained a memory dump.
Ah, I see.
onelove1210 wrote:Game's name is 决战!平安京, I have no idea what is the English name of this game because I'm not Chinese.
It's exactly the one in this post:
viewtopic.php?f=29&t=17924
and here:
viewtopic.php?f=16&t=17930
onelove1210 wrote:Actually I got a lot of files but I thought those two were the most useful to figure out geometry data based on what I've seen in Hex Viewer.
See if you can get a clue in other files. Try searching a scrambled "alphabet" table like this one:
Image

Re: Extracting simple models

Posted: Thu Jul 05, 2018 4:09 pm
by onelove1210
Bigchillghost wrote: It's exactly the one in this post:
viewtopic.php?f=29&t=17924
and here:
viewtopic.php?f=16&t=17930
You can see that there is no solution, and Ninja Ripper is not working :D
Bigchillghost wrote:See if you can get a clue in other files. Try searching a scrambled "alphabet" table like this one:
....
Ok, I will find it then. That means there is no similar scrambled "alphabet" table in 2 files I uploaded? Anyway thanks for your help :]

Edit: Is this a table you need?

Image

Re: Extracting simple models

Posted: Thu Jul 05, 2018 4:23 pm
by Bigchillghost
onelove1210 wrote: Edit: Is this a table you need?
No. This seems to be an offset table. You're supposed to search for an array of short intergers which are in an increasing order overall, as I descripted in this post:
viewtopic.php?p=139003#p139003

Re: Extracting simple models

Posted: Thu Jul 05, 2018 4:33 pm
by onelove1210
Bigchillghost wrote:
onelove1210 wrote: Edit: Is this a table you need?
No. This seems to be an offset table. You're supposed to search for an array of short intergers which are in an increasing order overall, as I descripted in this post:
viewtopic.php?p=139003#p139003
Image

This one is an only scrambled alphabet table. I've read your tutorial before. However it's difficult to find that in a bunch of data (~1.5GB) as I'm a newbie. If it's not possible, I think I will give up then :(

Re: Extracting simple models

Posted: Thu Jul 05, 2018 4:48 pm
by Bigchillghost
onelove1210 wrote:However it's difficult to find that in a bunch of data (~1.5GB) as I'm a newbie. If it's not possible, I think I will give up then :(
Well then you can search "00 00 00 01 00 02" and see if there's anything matches it.

Re: Extracting simple models

Posted: Thu Jul 05, 2018 5:10 pm
by onelove1210
Bigchillghost wrote:Well then you can search "00 00 00 01 00 02" and see if there's anything matches it.
Of course that's what I have done :D Anyway, sorry to disturb you, I'll give it a chance later

Re: Extracting simple models

Posted: Thu Jul 19, 2018 10:48 pm
by Tosyk
I have a head mesh. there's a 4 submeshes: eyes, eyelashes, face and teethguard.
I build vertices, polys and uv for all except the face — it splitted into small pieces and each piece separate with some sort of padding usually look like this:

Code: Select all

06 00 00 00 D3 9B 6A 3F
06 00 00 00 B4 A6 6C 3F
06 00 00 00 94 34 6E 3F
probably I can delete all this garbage and build head (called face here) mesh normally but right now I'm trying to automate this process. I'm using Model Researcher.

sample and Model Researcher template:
https://drive.google.com/open?id=15Pqlx ... zMJUDxdfnw

please help me. this is my first attempt

edit: there a lot of morphs, bone names at the end of the file.

etit2: I've noticed that some of paddings between "vertex size" are longer than 4 bytes and could be: 05000000 31EE0D3D 06000000 F2616A3C 07000000 E001753C. since there's are morphs I think this info could be connected to that morphs somehow.

edit3: after more research I found the following. I thought at first that normal "vertex size" long is 88 bytes but it's a bit more complicated. For example I took a block which size is 104 bytes:
3 floats (cyan): for vertex XYZ
9 floats (black): unknow at the moment, possible normals, vertex color, weights etc
4 floats (pale green): it's a usual placeholder for this format, it will be the same for each vertex in this file
1 float (yellow): I believe it stands for UV channel number but unsure
2 floats (pink): UV
1 float (blue): this one is tricky, it defines how many float pares will be after it. I think this float and the ones next to it are some kind of modifiers of this vertex
2-8 floats (pale purple): it could be from 2 to 8 floats, didn't see more. it's a vertex modifiers as I call them. they could be related to the morphs or something else

right now I need to skip this floats after UV section, any tips?
b_ellis_head_sample.jpg
btw how should I call this line of symbols related to single vertex? Vertex block? vertex size long?

Re: Extracting simple models

Posted: Fri Jul 20, 2018 7:18 am
by Bigchillghost
Tosyk wrote: 1 float (blue): this one is tricky, it defines how many float pares will be after it. I think this float and the ones next to it are some kind of modifiers of this vertex
2-8 floats (pale purple): it could be from 2 to 8 floats, didn't see more. it's a vertex modifiers as I call them. they could be related to the morphs or something else
It's nothing special but merely a simple structure for skin info:

Code: Select all

long	boneCount;
for(i = 0; i < boneCount; i++)
{
	long	boneID;
	float	boneWeight
}

Re: Extracting simple models

Posted: Fri Jul 20, 2018 10:27 am
by Tosyk
ahh, thats what 00 00 80 3F stands for it's 100% (red) weight for head bone, am I right?

also, I think I need to create some sort of script or plugin to build mesh out of this data. So any advice on this would be greatfully usefull! Noesis, maxscript? Can I use structure I found right away with noesis python to build the mesh and then put in order this code to load other models correctly?

so, for example, in Model Researcher I can set vertex block start and vertex count to build vertex cloud in a few seconds, can I do the same in noesis to just understand the basic?

btw, this is too complicated at this point of my knowledge :(
Bigchillghost wrote:It's nothing special but merely a simple structure for skin info:

Code: Select all

long	boneCount;
for(i = 0; i < boneCount; i++)
{
	long	boneID;
	float	boneWeight
}

Re: Extracting simple models

Posted: Fri Jul 20, 2018 11:10 am
by Bigchillghost
Tosyk wrote:ahh, thats what 00 00 80 3F stands for it's 100% (red) weight for head bone, am I right?
If there's only one bone used by that vert, of course.
Tosyk wrote:Noesis, maxscript?
Whatever you prefer.
Tosyk wrote:Can I use structure I found right away with noesis python to build the mesh and then put in order this code to load other models correctly?
If you mean loading a certain model only, you might have already got enough info. But if you wanna automate the process for this format, you need to know how to locate to the next mesh, or you just search for a certain byte sequence which I don't really recommend unless there's no other way.
Tosyk wrote:so, for example, in Model Researcher I can set vertex block start and vertex count to build vertex cloud in a few seconds, can I do the same in noesis to just understand the basic?
Actually there's not much difference between the two processes. The only difference is that to make an automatic tool your program should be able to obtain necessary info by itself rather than depending on a human mind.
Tosyk wrote:btw, this is too complicated at this point of my knowledge :(
It's just a piece of pseudo code in C to help you understand the structure of the vertex format. It's equivalent to the following pseudo code.

Code: Select all

Type	Specification

long	boneCount
for i = 1 to boneCount
{
	long	boneID
	float	boneWeight
}

Re: Extracting simple models

Posted: Fri Jul 20, 2018 12:28 pm
by shakotay2
Tosyk wrote:So any advice on this would be greatfully usefull! Noesis, maxscript?
Why not use python with ModelResearcher (MR)? http://lazov.ru/mr/tutorial.php?name=tut_mr_scripts

(I did not use it but if you're using MR it would make sense.)

Anyways, here's a Noesis script to just load ellis_head (the eyeball, to be exact) and nothing more:

Code: Select all

from inc_noesis import *
import noesis
import rapi

def registerNoesisTypes():
    handle = noesis.register("ellis_head", ".wwm")
    noesis.setHandlerTypeCheck(handle, noepyCheckType)
    noesis.setHandlerLoadModel(handle, noepyLoadModel)
    return 1

def noepyCheckType(data):
    bs = NoeBitStream(data)
    idstring = bs.readUInt()
    if idstring != 0x00000049:
        print("not a wwm file!")
        return 0
    return 1

def noepyLoadModel(data, mdlList):   
    ctx = rapi.rpgCreateContext()
    bs = NoeBitStream(data)
    #rapi.rpgClearBufferBinds()
    bs.seek(0x8b420, NOESEEK_ABS)
    size = 26864#= 5037/3*4*4
    idxbuf_tmp = bs.readBytes(size)

    numFaces = 5037//3
    print("numFaces", numFaces)
    idxBuf = []
    for i in range(numFaces):
        idx = idxbuf_tmp[i*16 + 4] + idxbuf_tmp[i*16 + 5]*256
        idxBuf.append(idx)
        idx = idxbuf_tmp[i*16+ 8] + idxbuf_tmp[i*16 + 9]*256
        idxBuf.append(idx)
        idx = idxbuf_tmp[i*16 + 12] + idxbuf_tmp[i*16 + 13]*256
        idxBuf.append(idx)


    #print(idxBuf)

    bs.seek(0x368, NOESEEK_ABS)
    vertsCount = 884
    VertBuf = bs.readBytes(vertsCount * 88)
    b = struct.pack('H' * len(idxBuf), *idxBuf)

    rapi.rpgBindPositionBufferOfs(VertBuf, noesis.RPGEODATA_FLOAT, 88, 0)
    rapi.rpgCommitTriangles(b, noesis.RPGEODATA_USHORT, numFaces*3, noesis.RPGEO_TRIANGLE, 1)

    mdl = rapi.rpgConstructModel()
    mdlList.append(mdl)

    return 1
ellis_head(eyeballs)_Noesis.jpg
Very important:
Bigchillghost wrote:But if you wanna automate the process for this format, you need to know how to locate to the next mesh,
or you just search for a certain byte sequence which I don't really recommend unless there's no other way.
well, I wouldn't say so. It's the most recommended way for beginners, imho.

Re: Extracting simple models

Posted: Fri Jul 20, 2018 12:34 pm
by Tosyk
Bigchillghost wrote:It's just a piece of pseudo code in C to help you understand the structure of the vertex format. It's equivalent to the following pseudo code.

Code: Select all

Type	Specification

long	boneCount
for i = 1 to boneCount
{
	long	boneID
	float	boneWeight
}
but why you used 'long' for boneCount and boneID (?) — it's 4 bytes each so it should be float, no?

I don't understand all the terminology atm. I know that flot stands for 4 bytes, and half-float is 2 bytes, that's why I used 'float' to define what I found.
alos, how should I call this line of symbols related to single vertex? Vertex block? vertex size long?
shakotay2 wrote:
Tosyk wrote:So any advice on this would be greatfully usefull! Noesis, maxscript?
Why not use python with ModelResearcher (MR)? http://lazov.ru/mr/tutorial.php?name=tut_mr_scripts
wow, I didn't know that, thanks for pointing me :)
shakotay2 wrote:Anyways, here's a Noesis script to just load ellis_head (the eyeball, to be exact) and nothing more
oh, thanks a lot. that's a good start to understand basics :mrgreen: