Spotlight: Señor Casaroja's Noesis

[MrAdults]

http://sf4viewer.sourceforge.net/
Maybe this will help you.

Nice to know that the specs are out there. But this provides even less reason for me to be the one to implement it.

Also:
Do you plan to add SMD Animation Export?

Last I checked, I couldn't find a Max 9 importer that handles sequence-only SMD's correctly. Wunderboy's had issues. But I'll probably get around to it one day.

[MrAdults]

Well, 2.3 is up:

Code: Select all

 -Added proper display of normal maps where applicable, and some new light properties. (viewable/editable in the data viewer)
 -Bayonetta model import support. Source code is included in the plugin SDK.
 -Sped model preview load up slightly. Probably won't be noticeable, this was mainly a forward-looking optimization.
 -Added vertex morph frame items for the data viewer. Highlighting a frame will render the model in that frame in the active preview.
 -Now sorting the file type mask in alphabetical order.
 -Various new plugin API functions, including some useful procedural animation functionality. (use it to test your bone weights)
 -Fixed another container bug that was causing .iso and .afs to not be correctly recognized.
 -Added CPK container extraction support. Thanks to hcs for releasing source code that describes this format! This implementation should run around 10-20 times faster than cpk_unpack.exe.

Some notes about Bayonetta:

1) Some models look messed up, but they really aren't. They just contain overlapping LOD meshes, and I haven't been able to find a LOD index or a flag that dictates something is a LOD.
2) Some models look messed up, because they are! There are some weird vertex strides occasionally and other random trouble with face indexing.
3) Haven't started on animation data yet. If anyone else has, let me know.
4) The code is right there and compilable/debuggable in Noesis. Feel free to help me out!

And speaking of helping me out, I'll probably be ceasing work on Noesis again for a little while. I've been a complete failure and neglected everything in pretty much every other facet of my life since I began this Noesis-binge. So I kindly request that you send some of those "can you add support for x" requests to some other capable programmers who can write some third-party Noesis plugins.

This latest release makes life as a plugin author even easier, and Bayonetta shows how relatively easy the task of adding 360/big-endian files can be. There's also some newly-exposed functionality for testing out your bone weights. This pretty much streamlines the whole process of reverse-engineering models and puts 0 obstructions between your debugger/data-analysis and your runtime evaluation of your results. I really suggest giving it a try. I can also tell you how to set yourself up to launch your plugin+model+Noesis directly out of MS Visual Studio, if you aren't familiar with that sort of thing. (you probably are, if you've ever made a Half-Life/Quake3/etc. mod)

[Cloud452]

2.3 already?! Nice progress MrAdults!

I hope some coders take the initiative and create various plugins. I personally hope for an directx format exporter.

[CMihai]

Damn man, that's pure epic, GJ there !

I still hope that someday someone will rip Golden Axe Beast Rider chroxx did some progress from what I remember...

[MrAdults]

UberBlack informed me that Vanquish uses almost the same format as Bayonetta, so I made a small update to support it, and fixed Bayonetta support in the process. (most models now load without trouble, but some are still kind of broken)

Code: Select all

Version 2.35:
 -Fixed Bayonetta support up, and added Vanqish support in the same plugin module. (the formats are similar)
 -Fixed broken path names for FPK extraction, made TTD's auto-load with GMO's. (when applicable)

Vanquish uses a different material system from Bayonetta, and uses global texture identity references. And unlike Bayonetta, there are many shared texture resource pools, rather than having each .dat file typically bundled with the mesh and its unique texture bundle. So a lot of models will show up textureless in default preview, though models that have a corresponding .dtt file will typically load textures correctly.

Also, as far as I can tell, the only way to correctly figure out which reference is to normal maps is to disassemble the pixel shader referenced by the model (pixel shader references were added in some new data that doesn't seem to appear in Bayonetta). But because that is crazy, I just hacked some checks in to hardcode offsets based on a few possible shader references. It covers most character/enemy models, but not all of them. If you see weird black specks on the model, it probably grabbed the wrong normal map, so you'll have to turn off lighting or per pixel lighting (in the data viewer's model root branch) to view it normally. I've only come across one of those so far, which I just couldn't be bothered to fix. (the model appears like 5-10 times, and looks fine in other instances)

Also, after I noticed that Bleach model thread, I fixed FPK support and made the GMO path auto-load TTD files if they're present. (this allows you to export textures from them as well, along with the model)

[Cloud452]

Also, after I noticed that Bleach model thread, I fixed FPK support and made the GMO path auto-load TTD files if they're present. (this allows you to export textures from them as well, along with the model)

Great! I had been having trouble using the fpk unpacker.

Question though, when I extract and view the gmo, it appears to have two models merged into one:



I recall that there was a similar issue with the Dissidia model for Ceil, and there was a command that separated them... is there something similar I can do here?


Btw, when you said

I don't think I have the data anywhere easily accessible anymore.

when it came to the Crisis Core map files, if it would be of any use, I could provide you with the RAW map file?

[MrAdults]

They aren't multiple models, they're just surfaces within the model. You can toggle them off and/or delete them after export. However, I had the GMO exporter defaulting to batching all surfaces which share the same material, and that's a problem for cases like this.

I just put 2.36 up which contains nothing but a switch to non-grouped by default. (instead it's grouped by the surface name and material, so things like the hand models are properly separated) If you want to get the old grouping functionality now, you can use -gmogroup.

Also, I'm seriously going to stop updating this thing for a while now. Until more glaring issues catch my eye, and I'm helplessly pulled back into the codebase like some kind of infected sore that I can't stop picking at.

[revelation]

MrAdults,
Are multiple calls to rpgBegin/rpgEnd formed into sub-meshes automatically within a model, or do they override one another?

[MrAdults]

Sweet, someone's actually taken a look at the plugin SDK!

All triangles get auto-batched into surfaces by material+name, which means multiple rpgBegin/rpgEnd and CommitTriangles (depending on whether you want to use immediate or buffered methods) can end up as a single surface. If you want to ensure certain geometry is batched into certain buckets without redundantly setting the material, you can use rpgSetName.

[revelation]

ah, ok. Do multiple calls within the same model have to have the same data layout? So if the first batch contains vertex colors, the following need them as well?

[MrAdults]

It does in theory handle drawing multiple primitives with differing available data, e.g. one Begin/End batch with vertex colors and another without. If you don't separate those surfaces by name, though, it's possible they could get batched together, and you'd end up with the corresponding vertex data zero'd out in part of the surface. Which in some cases might be fine, or desirable. Separating the surfaces by setting the name, however, will allow you to eliminate that possibility. Since each end-result surface has its own locally-mapped vertex data.

[revelation]

Ok, i will try that.

Now another question. Are texture references cleared when the context is destroyed?

i am getting crashes related to textures. i was following the example whereby a context is created with each model. The difference being this format has multiple models that share the same texture list. So i created the textures before the model contexts are created. i am finding that the second model seems to have valid materials but invalid textures. So would it be better to create a single context and simply make the multiple rpgConstructModel calls in the same manner instead?

[MrAdults]

Texture references stick around once the context is destroyed, but referencing the same textures between multiple model instances might get you in trouble. It's funny that you've run into this issue, because I just happened to be thinking about it in the shower this morning and wondering if it would break. I've never actually run into a situation before where I wanted to share a single texture set between multiple models. Since the texture set gets freed along with the model, pointing to the same texture data on two models could be a bad thing.

However, that would only be a problem once the data is freed, in theory. That doesn't happen until you totally close out of the preview or load another model into the preview. Destroying the rpg context has nothing to do with it, and making multiple rpgConstructModel calls under one context won't save you from crashing. The work-around would be to duplicate the texture data, but that's pretty nasty too.

If you don't mind, can you send me the plugin and the necessary data? Adding a reference count to the material data counter should fix the fundamental problem I'm thinking of (and I can handle that internally by detecting the references to a given material data set, so the plugin author doesn't have to change anything), but there's probably another problem I'm not even aware of if it's crashing outright for you.

[revelation]

My guess is that it is crashing during destruction, as it happens once you load another model or close the application. But in either case the texture references in any model but the first show nothing when accessed in the data viewer. Does setting shouldFreeData or making calls to Noesis_GetMatDataFromLists have anything to do with how the internal texture data is handled or released? i will send you the information shortly. i may just be using things incorrectly since i kind of just hacked things into an existing plugin's code from the tool i had written in another thread.

[MrAdults]

Ah, ok, that's a good sign, then. It means the only thing going wrong is probably what I think is going wrong.

Setting shouldFreeData causes Noesis to free the texture data, if you've allocated it with the Noesis_UnpooledAlloc function. (the Noesis heap) But if you've used Noesis_PooledAlloc function, you don't have to worry about setting that bool. If you're allocating it with malloc, new, or something that uses your module-local heap, then that's also bad. It means you'll end up with a memory leak (since Noesis doesn't have access to your module's local heap), or a potential crash if you set shouldFreeData, as Noesis will then try to free the data on its own heap.

So that could be a problem too, if you didn't change your code over to use the Noesis memory alloc functions. But I suspect the main problem is still on my end. It's also worth noting that the only data you need to worry about there is the pixel data itself. Everything else is pool-allocated for textures and materials, to avoid this whole mess. I'd also be happy to hear any suggestions you might have for making the use of these interfaces a little less obtuse or generally more obvious/friendly. I haven't gotten any good feedback on it yet, and am really very happy that you're messing around with it.