Spotlight: Señor Casaroja's Noesis

[MrAdults]

Yep, what chrrox said. There's also already a function in inc_noesis.py to do the decode/strip for you, with a default argument that assumes "ASCII" for the string formatting. So you can also just go
yourString = noeStrFromBytes(bs.readBytes(Length))

[Rimbros]

Ran online its a very good game, and have a long way in the MMORPG online games, any chance to integrate this to noesis and keep out the animations? models can be readed easy coz its .x format but animations are bin format (suspected its also .x animations). Also a system to inject back the model in game its tested and working.

I make a treadh with samples here
viewtopic.php?f=16&t=7717

And video with guide to inject back models here
viewtopic.php?f=16&t=5697&hilit=RAN+ONLINE

[finale00]

you can read 20 ints in a row something like this
test = bs.read("i"*20)

I moved away from struct arguments because I usually get confused which is which, and anyone that's reading the code would probably have to figure out which is which as well.

In that case, is there any particular use for the readLong readShort methods?

[MrAdults]

I moved away from struct arguments because I usually get confused which is which, and anyone that's reading the code would probably have to figure out which is which as well.

In that case, is there any particular use for the readLong readShort methods?

Not sure what you mean by figure out which is which. A read call with a struct format string looks quite different from all of the other convenience readX (readShort, readInt, etc) methods. (which are indeed only there for convenience and speed - you don't have to use them if you don't want to)

I think readInt(20) would look a lot more ambiguous and confusing - first thing I would ask is, is it reading 20 bytes or 20 ints? It's not at all implied by the syntax, whereas "i"*20 demonstrates that it's 20 ints quite clearly.

[finale00]

I think someone that is not too familiar with binary reading/writing would probably look at it and go "so...it's reading 20 integers?" But maybe I'm assuming too little of people that will actually be writing plugins.

I've never actually asked anyone what they would think when they see such a function name.

[Roxas0Zero]

Export to another format other than FBX. Then go to Blender and convert to FBX and see if that works. Maya and Max I think have trouble with his FBX and DAE exports. Last time I tried using Noesis to convert some FF models to DAE (which was like, uhm, yesterday), Maya went bonkers when I tried to import it lol. Polygons were strewn all over the place in a gigantic ball surrounding the model. Lightwave was more well-behaved and showed the model but it wouldn't import the UV map either.

You shouldn't be trying to import the COLLADA DAE's with Autodesk's DAE importers. They are meant to be imported with something based on OpenCOLLADA. Autodesk's DAE implementation is actually pretty fucked and doesn't tend to correctly conform to any given schema.

FBX is another story. Both Maya and Max happily ingest FBX's exported by Noesis. Although Max has bugs with importing normals on skinned models (Autodesk has acknowledged this as a bug on their end and it will hopefully be fixed in the 2013 FBX SDK), and I've yet to find any issues importing Noesis-produced FBX files in Maya. Autodesk's free viewer also has no issues importing Noesis-exported FBX files, normals look fine, and now so do interpolated animations now that Noesis is adding an unroll filter to exported rotation curve nodes. Most likely, that dude's only issue is that he needs to set up his materials to use vertex colors for the diffuse. In Max that just means going through the materials and adding a diffuse map that points to the vertex color channels.

hello MrAdult , what i meant here actually is , the colors of the body in FF7 Models doesn't appear , i try Max and Blender with different file extensions or types
and its the same problem.. can you tell me whats the problem here ?
do i have to do the texturing by my self ? or is there a way ?
thank you sir

[MrAdults]

the colors of the body in FF7 Models doesn't appear

As mentioned, you need to set the diffuse color to the vertex color channel(s) on the material(s) on the model. From memory, I believe there's a "Vertex paint" or "Vertex color" map type or some such that you can assign to the material's diffuse. Don't have Max on this machine to check.

[Privateer]

Python is only able to import correct?
To export in a special format We need to code a dll correct?

[MrAdults]

Python is only able to import correct?
To export in a special format We need to code a dll correct?

Not sure why you would think that, you can also export with Python scripts. Image exporting is done in the Q2 WAL and SoF M32 scripts, and the __NPExample.txt script shows how to import and export to a full-featured model format, including skeletal animation and vertex morph animation. If you rename that file to have a .py extension instead of .txt, you'll see that you can now export to .noepy models in Noesis.

[Privateer]

My bad for not reading the files completely.
Thank you for pointing that out as sometimes I get rushed between projects.
Great program by the way!
Regards!

[finale00]

If there is no easy way to identify a file format other than by extension, should I bother making it a dedicated format module?
Also, how do I check a file's extension if I wanted to verify both header and extension?

[chrrox]

look at the start of the file

Code: Select all

#Noesis Python model import+export test module, imports/exports some data from/to a made-up format

from inc_noesis import *

import noesis

#rapi methods should only be used during handler callbacks
import rapi

#registerNoesisTypes is called by Noesis to allow the script to register formats.
#Do not implement this function in script files unless you want them to be dedicated format modules!
def registerNoesisTypes():
	handle = noesis.register("Virtual Fighter 5", ".bin")
	noesis.setHandlerTypeCheck(handle, noepyCheckType)
	noesis.setHandlerLoadModel(handle, noepyLoadModel) #see also noepyLoadModelRPG
	#noesis.setHandlerWriteModel(handle, noepyWriteModel)
	#noesis.setHandlerWriteAnim(handle, noepyWriteAnim)
	noesis.logPopup()
	#print("The log can be useful for catching debug prints from preview loads.\nBut don't leave it on when you release your script, or it will probably annoy people.")
	return 1

#check if it's this type based on the data
def noepyCheckType(data):
	bs = NoeBitStream(data)
	if bs.readInt() != 0x5062500 :
		return 0
	if len(data) < int(0x80):
 		return 0
	return 1   

this whole section here does that
extension
handle = noesis.register("Virtual Fighter 5", ".bin")
file contents
#check if it's this type based on the data
def noepyCheckType(data):
bs = NoeBitStream(data)
if bs.readInt() != 0x5062500 :
return 0
if len(data) < int(0x80):
return 0
return 1

[finale00]

Oh I see, I got it.

When assigning materials, is it possible to first create meshes and then "go back" to assign materials?

For example when I'm parsing mesh section I get a matIndex that tells me which material it uses, but then the material section comes after all of the meshes have been defined.

That way I don't have to jump around the file.

I've thought of

1: storing extra arrays for the sake of keeping track of indices (doesn't seem too bad...)
2: parsing and storing absolutely everything before beginning to build meshes and stuff (waste of memory?)

But those are awkward work-arounds. Any better approaches?

EDIT: also have question about textures. The format I have right now stores the image data directly into the model. How should I deal with this?

[MrAdults]

If you construct the meshes yourself (by creating the mesh/model/etc. objects and attaching them manually) you can do whatever you want in whatever order you want. If you are using the RPG interface then you must know the material at the time you commit triangles, just like OpenGL/Direct3D.

In actually drawing a model in any API, you must also know the material at the time you're putting the triangles on the screen. If you come across a model format that doesn't provide things in an order where this is natively possible, then the developers must not be drawing directly from the data in the order you're reading it. This means you can't expect to draw directly from that data either. I would look at how the data is stored, speculate (or if it's worth it, disassemble the original code to figure it out for sure) what the developers did to get that data into a format friendly to a state-based rendering architecture, and then I would render it into Noesis just like they rendered it into their graphics API. Or I might just shove whatever hacks in that I need to make it work with the less memory overhead.

Whether I would care about memory overhead would depend on whether their models are 2MB in memory or 200MB in memory. There's not much reason to care about memory overhead until you start getting into the 500MB+ range, where there's an actual danger of running out of process memory in 32-bit space. (once you consider concurrent preview+export and memory fragmentation) I'm pretty sure everyone has enough physical RAM nowadays to otherwise waste as much memory as you please. It's all going to get flushed anyway when the next file is loaded, after all. If you do start hitting memory ceilings you should probably consider supporting the format in a native DLL instead of in Python. Python, like most interpreted languages, imposes a pretty massive memory overhead, with a cost for every single object that goes through it. The most efficient way to represent model data in Python is as raw byte arrays, but trying to work with data in that form would pretty much defeat the entire point of using Python.

Edit: Oh, and when textures are stored in the model file, usually I load them right there and put them into the texture list attached to the model. (the same one materials go into) The Bullet Witch script does this, even though it's manually loading the textures from another package.

[finale00]

In actually drawing a model in any API, you must also know the material at the time you're putting the triangles on the screen.

Which part of the code does that?
So for example this is the load function that I use

Code: Select all

def noepyLoadModel(data, mdlList):
   ctx = rapi.rpgCreateContext()

   #create custom parser and parse data
   parser = CrucisFatalFake_LMD(data)
   parser.parse_file()

   #now make model and stuff
   mdl = NoeModel(parser.meshList, None, None)
   mdlList.append(mdl)
   mdl.setModelMaterials(NoeModelMaterials(parser.texList, parser.matList))
   return 1

I'm guessing "putting triangles on the screen" is done internally and not something I have to worry too much about?
Or is it when I'm creating the mesh?

Code: Select all

def create_mesh(self, meshName):
      
      mesh = NoeMesh(self.idxList, self.vertList, meshName, self.animList)
      mesh.uvs.extend(self.uvList)
      self.meshList.append(mesh)

Also, null-terminated strings.
I used to just read one char at a time, check if it's null, and then just add them to a string, but maybe there's a better way?

And is there a function that I can call that will parse a standard DDS file? Like maybe just take the one that's already used to load a DDS, but give me a texture object instead. Whatever I write to get a dds texture object probably isn't going to be as great lol