Spotlight: Señor Casaroja's Noesis

[Devilot]

Sin's model from FFX is still all weird, I wonder why. I think the model is kinda messed up. hope they redo it for the Re-release of FFX.

[MrAdults]

Noesis 4.069 had another pretty important crash fix. I once again strongly recommend updating. Instead of repeatedly submitting the same crash that I've already fixed.

[Devilot]

what crash? I always thought Noesis was perfect

[MrAdults]

It was a crash introduced several releases ago, there was a typo in the safe buffer handling.

For those who can't be assed to read a text file:

-4.069 - Fixed a crash in Python vertex color handling.
-4.068 - Fixed possible overwrite issues with the auto-updater.
-4.067 - Fix for a problem introduced with the DNF plugin by the last release.
-4.066 - Significant restructuring of native plugin code. Old plugin binaries will still work fine, but to compile old plugins against the new SDK, you'll have to make some changes.
-4.066 - Made removal of temp files check for failure cases.
-4.066 - Fixed a crash in the MFC splitter caused by broken IntelliMouse scroll handling, because I actually found this crash in my report list.
-4.065 - Running Noesis with "?multinstance" on the commandline will allow multiple instances.
-4.065 - Fixed a potential crash in the ISO extractor.
-4.065 - Fixed pixel order being incorrect for 4bpp PNG images.
-4.065 - Fixed log windows being limited to 30000 characters by default.
-4.065 - Fixed some JK:MotS levels not being recognized as MotS format. (thereby causing Noesis to crash)
-4.065 - Fixed TIM2's sometimes being incorrectly swizzled. This was breaking Saint Seiya.
-4.065 - Fixed a bug in the SMD parser which was causing bad results on vertices with 0 weights. Thanks to Badcrc for reporting this bug.
-4.064 - Fixed a potential crash in the GMO loader.
-4.064 - Fixed a crash when loading invalid NJCM data.
-4.063 - Made rpgCommitTriangles even more bulletproof.
-4.063 - Fixed possible crash when feeding None for the index buffer to rpgCommitTriangles.

[Demonsangel]

-4.066 - Significant restructuring of native plugin code. Old plugin binaries will still work fine, but to compile old plugins against the new SDK, you'll have to make some changes.

Does this mean you're encouraging to use the native code again?
Half a year back you told me to not bother with it vs python.

[MrAdults]

I generally advocate the use of native plugins where performance/load time matters, or where you need to do something you can only do in native code. Like make a custom interface or visualizer. Otherwise, python is probably more convenient, but it's up to you.

[Arymond]

im having issues with the NJ's from PSO:BB (From the ITEM AFS's)

i'd like to be able to view them but it can export them and the Texture files are XVM/XVR(DDS) (Xbox PVM)

[MrAdults]

Pretty much everything in the AFS files in PSO are PRS-compressed files. You can extract them, rename them to .prs, and use the PRS compression script on the official Noesis plugins repo to decompress them into .nj files which Noesis can open.

I don't know about PSO:BB. I don't have it. Since it was targeted at Windows/Xbox, they might've changed the format(s), which means Noesis doesn't support it.

[TheDude]

I just started learning Noesis and was wondering if I could get an example usage of rapi.unpackPS2VIF.

Also, with the vertex data I'm looking at, the last few bytes are weights then bone indices, except.....
they're numbered/matched w1, w2, w3, w4, b4, b3, b2, b1. It's the order of the bone indices that have me tripped up.
I was wondering if there was a way of using "rapi.rpgBindBoneIndexBufferOfs" to handle that reverse order.
Reason being I would like to keep it simple and just use the vert buffer I already have.

[MrAdults]

Something like this:

Code: Select all

				unpacks = rapi.unpackPS2VIF(vifData)
				#print("unpack count:", len(unpacks))
				for up in unpacks:
					if up.numElems == 3 and up.elemBits == 16:
						numVerts = len(up.data) // 6
						rapi.rpgBindPositionBuffer(up.data, noesis.RPGEODATA_SHORT, 6)
						rapi.rpgCommitTriangles(None, noesis.RPGEODATA_USHORT, numVerts, noesis.RPGEO_POINTS, 1)

If you wanted to run through all the unpacks and draw points for all the short vertex buffer candidates. Typically your lists will be consistent across surfaces, if not you'll see some indicator of such in the format so that the game knows to use different VU processing code.

You'll often end up with index data getting unpacked as part of VIF data too, sometimes packed in a custom format.

Presumably you're doing this for FF7:DoC. You might want to get with chrrox about that. He had some stuff up and running with correct triangle/index usage, but he was wrong about some of the offsets to the start of VIF data and about the stride of the VIF data. I think. I only looked at his code enough to test the Noesis VIF interface in Python. (it's already used for a few games now in native land, including FF12)

Edit: also, I think you are confused about the weights, but using Noesis to unpack them from the VIF stream will probably make it clearer. Also, Noesis doesn't care about order as long as the weight and index line up. You would have to do a byte swap if they didn't.

[TheDude]

Cool! Thanks!
I should have mentioned those 2 questions weren't related.
The 2nd one is concerning a X360 game.
I'm porting it to Noesis from Maxscript and I've imported plenty of characters so I know the weights - index matching is correct.

EDIT: Based on the print out, this appears to reverse the order of the last 4 bytes of each vertex (bone sub-palette indices)
in the buffer like I wanted:
#vertices
idx=[]
CNT=VCount * VSize
VertBuff = f.readBytes(CNT)
i=VSize
for x in range (0, CNT, VSize):
idx.append (VertBuff[i-1:i] + VertBuff[i-2:i-1] + VertBuff[i-3:i-2] + VertBuff[i-4:i-3])
i+=VSize
print (idx)

Of course I probably just jinxed myself by saying that.....

[MrAdults]

On 360, ubyte indices are usually a big-endian 32-bit uint, that's why they look backwards. But ubyte weights are commonly the same, which means it just works. But if they aren't using ubyte weights, you probably need to endian-swap your 4 bytes of indices yourself. I think I have a native function for you to do it quickly without using Python loops, but I can't check at the moment cause I'm at work.

[TheDude]

On 360, ubyte indices are usually a big-endian 32-bit uint, that's why they look backwards. But ubyte weights are commonly the same, which means it just works. But if they aren't using ubyte weights, you probably need to endian-swap your 4 bytes of indices yourself. I think I have a native function for you to do it quickly without using Python loops, but I can't check at the moment cause I'm at work.

I did not know that. How interesting.
Yeah, the weights are uint16s.
(please to be clicking, yes?)


Those indices reference a bone palette sub-array which in turn references the main bone palette for the main object which in turn references the actual bone IDs (Big props to Samu for figuring that out!)

Aaaaaaaany way, this is my current code:

Code: Select all

#vertices
boneids=[]
CNT=VCount * VSize
VertBuff = f.readBytes(CNT)
def buildBoneIndex():
	i=VSize
	for x in range (0, CNT, VSize):
		boneids.append (BoneInd[VertBuff[i-1:i][0]])
		boneids.append (BoneInd[VertBuff[i-2:i-1][0]])
		boneids.append (BoneInd[VertBuff[i-3:i-2][0]])
		boneids.append (BoneInd[VertBuff[i-4:i-3][0]])
		i+=VSize

The numbers that came out look right, but I haven't ported the skeleton code yet.

Whoops, I forgot to mention before, your vif command reader certainly looks a lot different from mine.
With mine you just type something like: "unpack_to vert_array". Of course, mine has some serious shortcomings.
For instance, it's hard-coded to read all 4-byte values as floats.
I suppose I could work on it, but it seems to work well enough with Doc:FF7 and tri-Ace's PS2 format.
........and it's a great way to stay in shape.

[MrAdults]

I did have a function, rapi.swapEndianArray. But it was pretty rigid, so I added a couple optional parameters for you in the build I just put up.

Code: Select all

	{"swapEndianArray", Noesis_SwapEndianArray, METH_VARARGS, "returns the entire array endian-swapped at x bytes. (Oi|ii)"}, //args=source array, swap count, (optional) offset (in bytes, into array), stride

You would use it like

Code: Select all

vertBuffer = rapi.swapEndianArray(vertBuffer, 4, offsetToBoneIndices, sizeOfVertex)

I notice in your code that you're converting to a Python list. It's fastest (and sometimes easiest, requiring much less code) to feed bytes/bytearrays directly into the rpg interface, as you'll notice the samples do. But if you want to do it the other way, it's up to you. Just a best-practices thing to keep in mind, it's one of the ways in which Noesis Python scripts have the ability to be much faster and more to the point than Blender Python scripts.

[TheDude]

Awesome! And thanks for the tip. I'm still new to Python and it seems like all the videos out there leave out things like bytes and bytearrays. Actually anything that someone might use in an importer for that matter. 〴⋋_⋌〵