Why does it need to match with the calculated bounding box? I think they're using unsigned shorts to map with the floats within the BBox so that they won't lose too much precision. So we just need to divide the bounding box space into 65535 segments on each axis to get their corresponding scale units.shakotay2 wrote:because the BBox calculated from the unsigned vertex data in hex2obj doesn't match with the real values so far. Need to look what's going wrong there:
# BBox 0.000000 255.468750, 0.000000 255.468750, 0.000000 255.468750
Important information: this site is currently scheduled to go offline indefinitely by end of the year.
JX3 HD Remake 剑网3 (.dat files)
- Bigchillghost
- double-veteran
- Posts: 1026
- Joined: Tue Jul 05, 2016 9:37 am
- Has thanked: 31 times
- Been thanked: 1212 times
Re: JX3 HD Remake 剑网3 (.dat files)
May you find peace in this puzzle-solving game.
- An Imitable Workflow for Reverse Engineering a Game Model
- Advanced Mesh Reaper
- Reverse Model Wireframe
- shakotay2
- MEGAVETERAN
- Posts: 4287
- Joined: Fri Apr 20, 2012 9:24 am
- Location: Nexus, searching for Jim Kirk
- Has thanked: 1149 times
- Been thanked: 2242 times
Re: JX3 HD Remake 剑网3 (.dat files)
yeah, my bad, I thought the BBox auto calculation went wrong, but seems it doesn't
65535? depends on your divider when creating the mesh - I just need to divide 5376.0376 by 255.469 for the x-scaling for example.
well, seems to fit:
65535? depends on your divider when creating the mesh - I just need to divide 5376.0376 by 255.469 for the x-scaling for example.
well, seems to fit:
You do not have the required permissions to view the files attached to this post.
Tuts: a) Bigchillghost, viewtopic.php?f=29&t=17889
b) Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
"Quoting the whole thing. Would u ever stop this nonsense?"
b) Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
"Quoting the whole thing. Would u ever stop this nonsense?"
- petventh18
- beginner
- Posts: 35
- Joined: Sat Mar 25, 2017 1:40 am
- Has thanked: 14 times
- Been thanked: 4 times
Re: JX3 HD Remake 剑网3 (.dat files)
I see, thank you...shakotay2 wrote:Is possible, but generally that splitting results in random submeshes (SMs). It's just kind a luck that the static models seem to have roundabout 500 faces clusters here, imho.
So for characters it's more sensefull to insert those lines manually, as there's aren't so many SMs, head, upper body, lower body for example.
I'm just wondering, is it possible for you to add some code to the Make_obj-JX3.exe tool to make it so the output .obj file had the same name as the .mesh file?
And if it's not too much trouble then maybe add batch feature to the tool too? Cuz this game had too many files and going through each .mesh at a time is really tedious and tiring to find the player models. Sorry for nagging so much... Just ignored my babbling if it's too much trouble =)
EDITED:
I tested a few of the meshes using Make_obj-JX3.exe tool and these 3 samples are the one that cannot be imported into 3D Software after forming the obj file.
https://drive.google.com/open?id=1FoZn5 ... t1Qogf4E5y
- shakotay2
- MEGAVETERAN
- Posts: 4287
- Joined: Fri Apr 20, 2012 9:24 am
- Location: Nexus, searching for Jim Kirk
- Has thanked: 1149 times
- Been thanked: 2242 times
Re: JX3 HD Remake 剑网3 (.dat files)
yeah, I was thinking about this since a long time. The batch feature is simple for console apps, but a GUI app needs an auto exit feature like SendMessage (hwnd, WM_CLOSE, 0L, 0L) ; which I used; but not sure about the side effects.petventh18 wrote:I'm just wondering, is it possible for you to add some code to the Make_obj-JX3.exe tool to make it so the output .obj file had the same name as the .mesh file?
And if it's not too much trouble then maybe add batch feature to the tool too? Cuz this game had too many files
But didn't you plan to create a Noesis script?
yeah, I see. Would require to use the addresses from the "magic table" at 0x90 where I only take the counts from so far. (Auto calculation of addresses doesn't seem to work for all files.)EDITED:
I tested a few of the meshes using Make_obj-JX3.exe tool and these 3 samples are the one that cannot be imported into 3D Software after forming the obj file.
Tuts: a) Bigchillghost, viewtopic.php?f=29&t=17889
b) Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
"Quoting the whole thing. Would u ever stop this nonsense?"
b) Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
"Quoting the whole thing. Would u ever stop this nonsense?"
- petventh18
- beginner
- Posts: 35
- Joined: Sat Mar 25, 2017 1:40 am
- Has thanked: 14 times
- Been thanked: 4 times
Re: JX3 HD Remake 剑网3 (.dat files)
I've only started learning C# language a year ago (so still super noob when come to program languages)... And when I tried python script along with some example plugin from other games and some tutorial on Youtube... it's basically a nightmare for noobie such as myself since every script seem to different from one another when it come to section likeshakotay2 wrote:But didn't you plan to create a Noesis script?
Code: Select all
def noepyLoadModel(data, mdlList):
orclass mesh(object):
Code: Select all
class mesh(SanaeObject):
Since can't change the offset from the tool, I guess will have to manually using Hex2Obj...yeah, I see. Would require to use the addresses from the "magic table" at 0x90 where I only take the counts from so far. (Auto calculation of addresses doesn't seem to work for all files.)
- shakotay2
- MEGAVETERAN
- Posts: 4287
- Joined: Fri Apr 20, 2012 9:24 am
- Location: Nexus, searching for Jim Kirk
- Has thanked: 1149 times
- Been thanked: 2242 times
Re: JX3 HD Remake 剑网3 (.dat files)
I have a deja vuepetventh18 wrote:it's basically a nightmare for noobie such as myself since every script seem to different from one another when it come to section like
Yea, I know that feeling; here's some simple script for beginners with some Noesis/python knowledge:
viewtopic.php?f=16&t=9534&p=141865&hili ... an#p141865
(ah, see, there's some math with the face indices buffer, feel free to ask, why, that's a matter of the format, not of python)
(Noesis is a powerful tool, there's absolutely no doubt, but the first scripting steps, are not so nice. )
That's a good start, imho.Since can't change the offset from the tool, I guess will have to manually using Hex2Obj...
But for hundreds of *.mesh files you might prefer this one (cmd line tool, GUI pops up and exits): tested with 3 files only
- you need the dlls from the previous zip
- you need to adjust the "path to the exe" in the .cmd file.
- blanks in *.mesh pathname not supported!
- start the cmd file in the folder where your *.mesh files reside
You do not have the required permissions to view the files attached to this post.
Tuts: a) Bigchillghost, viewtopic.php?f=29&t=17889
b) Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
"Quoting the whole thing. Would u ever stop this nonsense?"
b) Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
"Quoting the whole thing. Would u ever stop this nonsense?"
- petventh18
- beginner
- Posts: 35
- Joined: Sat Mar 25, 2017 1:40 am
- Has thanked: 14 times
- Been thanked: 4 times
Re: JX3 HD Remake 剑网3 (.dat files)
Thanks a lot for the info and for the tool update... it'll save me lots of time to sort through those models...shakotay2 wrote:...here's some simple script for beginners with some Noesis/python knowledge:
viewtopic.php?f=16&t=9534&p=141865&hili ... an#p141865
(ah, see, there's some math with the face indices buffer, feel free to ask, why, that's a matter of the format, not of python)....
But for hundreds of *.mesh files you might prefer this one (cmd line tool, GUI pops up and exits):
You mentioned that the UVs is 2 bytes off... so restart the tool and type in the new value on the editbox... then load the same mesh again, correct? Because it's hard to find the correct texture for the correct model since they're name differently... would be so nice if the name of texture match well like how Umodel arrange them... or even like how Ninja Ripper did it...
I wonder how the developer even keep track on all their models/texture with all these kind of names... or is it that the extract tool automatically rename them?
- Bigchillghost
- double-veteran
- Posts: 1026
- Joined: Tue Jul 05, 2016 9:37 am
- Has thanked: 31 times
- Been thanked: 1212 times
Re: JX3 HD Remake 剑网3 (.dat files)
There do seem to be some text recording the asset names but not every package outputs them, and some of these info is in json format which makes it so annoying/difficult to read. Yep you're correct the tool names them automatically using the orders of their chunks.petventh18 wrote:I wonder how the developer even keep track on all their models/texture with all these kind of names... or is it that the extract tool automatically rename them?
May you find peace in this puzzle-solving game.
- An Imitable Workflow for Reverse Engineering a Game Model
- Advanced Mesh Reaper
- Reverse Model Wireframe
- petventh18
- beginner
- Posts: 35
- Joined: Sat Mar 25, 2017 1:40 am
- Has thanked: 14 times
- Been thanked: 4 times
Re: JX3 HD Remake 剑网3 (.dat files)
Ah I c... thanks for clearing that up... I opened the json file on Notepad++ and only some .srt are recorded in it...Bigchillghost wrote:There do seem to be some text recording the asset names but not every package outputs them, and some of these info is in json format which makes it so annoying/difficult to read. Yep you're correct the tool names them automatically using the orders of their chunks.
ex: "srtFile": "data\\source\\maps_source\\树入库临时保存\\s_雪竹019c4_st.srt"
But the output file will be something like 0000XXXX.SRT
And the .SRT files itself does have some texture name recorded like
(SP_wanhuashupi001H.tga SP_wanhuashupi001H_Nor.tga SP_wanhuashupi001 S_wanhuazitengluo002.tga S_wanhuazitengluo002_Nor.tga Atlas S_wanhuazitengluo002_Billboards.tga S_wanhuazitengluo002_Billboards_Nor.tga...)
So based on that we can see some diffuse and normal texture that are corresponding to the wanhuashupi001.mesh...
But because everything was renamed, it will take a lot of guessing game to match the correct texture into the model (assuming that the UV is correct after turning the mesh to obj)...
If it's not too much trouble then may I ask if it's possible to modify the extraction tool to not rename the file based on the chunk but leave it as is? (Doesn't matter if it's Chinese characters)... Some folder had over 6000 files, so trying to guess their texture based on the current name is quite literary a nightmare.
That or maybe had a .log file that keep track of the original name to the current exacted name?
And again, if it's too much of a hassle then forget about it... I'm already very thankful that you created the tool as it is.
- Bigchillghost
- double-veteran
- Posts: 1026
- Joined: Tue Jul 05, 2016 9:37 am
- Has thanked: 31 times
- Been thanked: 1212 times
Re: JX3 HD Remake 剑网3 (.dat files)
Well, about that, have you checked if they're mapping?petventh18 wrote:assuming that the UV is correct after turning the mesh to obj
Maybe. That would require more research on the organization of the chunks, which can take some time. I'll see what I can do with it. But don't expect too much yet.petventh18 wrote: If it's not too much trouble then may I ask if it's possible to modify the extraction tool to not rename the file based on the chunk but leave it as is?
Edit: I can say that it's impossible to read filenames now because:
- The assets are linked by different objects in the json files, usually more than one times for each asset, and you can't locate the asset names with the MD5 hashes coz they don't even esixt in the json;
- There do seem to be a filelist in a certain json file, and there're some texture names in a binary chunk(using the ext .def in the following version), but their amounts and extensions don't match with the actual outputs, nor their sums.
I was wondering if the names will output correctly if they contain any Chinese. I hope I don't have to deal with the character encoding stuffs.petventh18 wrote:Doesn't matter if it's Chinese characters...
Ah, I can imagine that.petventh18 wrote:Some folder had over 6000 files, so trying to guess their texture based on the current name is quite literary a nightmare.
Sounds good, but not necessary, nor possible. Because the tool didn't read any filenames at all. It just decompresses each chunk and assigns the index of the loop to the string buffer as the output filename (in hex). The simplest solution is to save every chunk with a decimal index name so that you can sort them with your Windows explorer. It still needs your guessing work though. But it's an alternative. Better than nothing.petventh18 wrote:That or maybe had a .log file that keep track of the original name to the current exacted name?
Edit: Now you can have it: Changes:
- Changed naming patterns from hex to decimal index;
- No text/json files will be merged;
- Improve detection of text/json data (first 4 bytes only).
- The files with the signature "\x08\x00\x00\x00" will be separated from the TBD files, using the extension ".8";
- The binary chunk(s) containing texture names will use the extension ".def".
You do not have the required permissions to view the files attached to this post.
Last edited by Bigchillghost on Sat Oct 06, 2018 8:51 am, edited 1 time in total.
May you find peace in this puzzle-solving game.
- An Imitable Workflow for Reverse Engineering a Game Model
- Advanced Mesh Reaper
- Reverse Model Wireframe
- shakotay2
- MEGAVETERAN
- Posts: 4287
- Joined: Fri Apr 20, 2012 9:24 am
- Location: Nexus, searching for Jim Kirk
- Has thanked: 1149 times
- Been thanked: 2242 times
Re: JX3 HD Remake 剑网3 (.dat files)
I wasn't sure if it would work for 100s of models...petventh18 wrote:Thanks a lot for the info and for the tool update... it'll save me lots of time to sort through those models...shakotay2 wrote:...here's some simple script for beginners with some Noesis/python knowledge:
http://forum.xentax.com/viewtopic.php?f ... an#p141865
(ah, see, there's some math with the face indices buffer, feel free to ask, why, that's a matter of the format, not of python)....
But for hundreds of *.mesh files you might prefer this one (cmd line tool, GUI pops up and exits):
Forget about that; that's for the old version with auto calculation of addresses; sometimes it gave incorrect values (2 bytes off for example) for the uvs' start.You mentioned that the UVs is 2 bytes off... so restart the tool and type in the new value on the editbox... then load the same mesh again, correct?
The cmdline version gets all values from the magic table at 0x90 (counts and addresses) now. But it "relies" on a 12 bytes offset in the uvs' block (3 floats to be skipped before the uv data).
btw: this format is a very good start to be used with a Noesis script (you don't need the face indices buffer math in my previous linked sample).
The codelines for getting the face indices should look like this (untested):
faceBuff = bs.readBytes(faceCount*3*4)
rapi.rpgCommitTriangles(faceBuff, noesis.RPGEODATA_UINT, faceCount*3, noesis.RPGEO_TRIANGLE, 1)
UINT because of DWord face indices
and you don't need this weird
b = struct.pack('H' * len(idxBuf), *idxBuf) for this format
here's a working Noesis script, tested with 00000161.mesh:
Code: Select all
from inc_noesis import *
import noesis
import rapi
def registerNoesisTypes():
handle = noesis.register("JX3", ".mesh")
noesis.setHandlerTypeCheck(handle, noepyCheckType)
noesis.setHandlerLoadModel(handle, noepyLoadModel)
return 1
def noepyCheckType(data):
bs = NoeBitStream(data)
bs.seek(0x54, NOESEEK_ABS)
idstring = bs.readUInt()
if idstring != 0x4D455348:
print("not a JX3 file!")
return 0
return 1
def noepyLoadModel(data, mdlList):
ctx = rapi.rpgCreateContext()
bs = NoeBitStream(data)
#rapi.rpgClearBufferBinds()
bs.seek(0x90, NOESEEK_ABS)
vertsCount = bs.readUInt()
faceCount = bs.readUInt()
bs.seek(0xA8, NOESEEK_ABS)
uvAddr = bs.readUInt()
uvAddr += 12
bs.seek(0xB4, NOESEEK_ABS)
FIaddr = bs.readUInt()
bs.seek(0x12C, NOESEEK_ABS)
VertBuf = bs.readBytes(vertsCount * 6)
bs.seek(uvAddr, NOESEEK_ABS)
uv1Buf = bs.readBytes(vertsCount * 6)
bs.seek(FIaddr, NOESEEK_ABS)
idxBuf = bs.readBytes(faceCount*3*4)
rapi.rpgBindPositionBufferOfs(VertBuf, noesis.RPGEODATA_USHORT, 6, 0)
rapi.rpgBindUV1Buffer(uv1Buf, noesis.RPGEODATA_USHORT, 6)
rapi.rpgCommitTriangles(idxBuf, noesis.RPGEODATA_UINT, faceCount*3, noesis.RPGEO_TRIANGLE, 1)
mdl = rapi.rpgConstructModel()
mdlList.append(mdl)
return 1
edit: attaching above mentioned sample, in case it's not available any more: Also I didn't care for the normals buffer, as always.
addr can be found in the magic table; refer to the struct that Bigchillghost posted.
You do not have the required permissions to view the files attached to this post.
Last edited by shakotay2 on Sun Feb 20, 2022 7:43 pm, edited 1 time in total.
Tuts: a) Bigchillghost, viewtopic.php?f=29&t=17889
b) Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
"Quoting the whole thing. Would u ever stop this nonsense?"
b) Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
"Quoting the whole thing. Would u ever stop this nonsense?"
- Bigchillghost
- double-veteran
- Posts: 1026
- Joined: Tue Jul 05, 2016 9:37 am
- Has thanked: 31 times
- Been thanked: 1212 times
Re: JX3 HD Remake 剑网3 (.dat files)
I can't believe that they actually packed the normals in this way. You need to perform the following operations to get the correct normals:Bigchillghost wrote:Now the problem is to get correct normal vectors:
Code: Select all
for (int i = 0; i < Vcount; i++)
{
dblNM[i].x = charNM[i].x / 127.0;
dblNM[i].y = charNM[i].y / 127.0;
dblNM[i].z = charNM[i].z / 127.0;
if (dblNM[i].x < 0.0)
dblNM[i].x = -dblNM[i].x - 1.0;
if (dblNM[i].y < 0.0)
dblNM[i].y = -dblNM[i].y - 1.0;
if (dblNM[i].z < 0.0)
dblNM[i].z = -dblNM[i].z - 1.0;
dblNM[i].x = -dblNM[i].x;
dblNM[i].y = -dblNM[i].y;
dblNM[i].z = -dblNM[i].z;
}
You do not have the required permissions to view the files attached to this post.
May you find peace in this puzzle-solving game.
- An Imitable Workflow for Reverse Engineering a Game Model
- Advanced Mesh Reaper
- Reverse Model Wireframe
- petventh18
- beginner
- Posts: 35
- Joined: Sat Mar 25, 2017 1:40 am
- Has thanked: 14 times
- Been thanked: 4 times
Re: JX3 HD Remake 剑网3 (.dat files)
@shakotay2
Thanks for the script... It make it a lot easier for noobie me Although I couldn't get the CMD version of the tool to work at all. I place the prev folder with some .mesh inside of it. But it just won't do anything when click or drag a mesh file to it. I made sure to rename the path inside the .cmd file as well...
But it's very hard matching the texture and model so I decided to buy an Xoyo account and tried my luck with Ninja Ripper... good thing it worked...
So if anyone don't wanna sort through the mesh manually then try Ninja Ripper... the down side is that you can only rip the model you see and you'll have fking nightmare trying to register for a game account without a chinese phone number. And the face and hair position don't match properly so you have to manually trying to put it together.
Thanks for the script... It make it a lot easier for noobie me Although I couldn't get the CMD version of the tool to work at all. I place the prev folder with some .mesh inside of it. But it just won't do anything when click or drag a mesh file to it. I made sure to rename the path inside the .cmd file as well...
But it's very hard matching the texture and model so I decided to buy an Xoyo account and tried my luck with Ninja Ripper... good thing it worked...
So if anyone don't wanna sort through the mesh manually then try Ninja Ripper... the down side is that you can only rip the model you see and you'll have fking nightmare trying to register for a game account without a chinese phone number. And the face and hair position don't match properly so you have to manually trying to put it together.
- Bigchillghost
- double-veteran
- Posts: 1026
- Joined: Tue Jul 05, 2016 9:37 am
- Has thanked: 31 times
- Been thanked: 1212 times
Re: JX3 HD Remake 剑网3 (.dat files)
Probably this is pointless but here's the truth: since I've updated my binary FBX builder to support face materials and other features,
this format had been the perfect sample for testing purpose. Wouldn't it be a waste if it'd never got released?
For example here's one of the material group of the entire mesh from 000005E0.mesh (7 in total):
Note that the UVs endoced as WORD are a bit off the correct position which I have no clue of why.
I made a little correction for this model only so it's not guaranteed to work for all UVs.
For float UVs you don't have worry about it though. Just run the tool without prams to see the usage.
this format had been the perfect sample for testing purpose. Wouldn't it be a waste if it'd never got released?
For example here's one of the material group of the entire mesh from 000005E0.mesh (7 in total):
Note that the UVs endoced as WORD are a bit off the correct position which I have no clue of why.
I made a little correction for this model only so it's not guaranteed to work for all UVs.
For float UVs you don't have worry about it though. Just run the tool without prams to see the usage.
You do not have the required permissions to view the files attached to this post.
Last edited by Bigchillghost on Sat Feb 23, 2019 8:46 am, edited 1 time in total.
May you find peace in this puzzle-solving game.
- An Imitable Workflow for Reverse Engineering a Game Model
- Advanced Mesh Reaper
- Reverse Model Wireframe
-
- advanced
- Posts: 47
- Joined: Thu Sep 20, 2018 9:28 pm
- Has thanked: 10 times
- Been thanked: 5 times
Re: JX3 HD Remake 剑网3 (.dat files)
Why can't I open?Bigchillghost wrote:Probably this is pointless but here's the truth: since I've updated my binary FBX builder to support face materials and other features,
this format had been the perfect sample for testing purpose. Wouldn't it be a waste if it'd never got released?
For example here's one of the material group of the entire mesh from 000005E0.mesh (7 in total):
Note that the UVs endoced as WORD are a bit off the correct position which I have no clue of why.
I made a little correction for this model only so it's not guaranteed to work for all UVs.
For float UVs you don't have worry about it though. Just run the tool without prams to see the usage.
You do not have the required permissions to view the files attached to this post.