Fallout 4 in Noesis

[MrAdults]

Noesis v4.171 has support for Fallout 4 archives/textures/NIFs. I guess people have been working on this stuff for weeks/months already, but no one is sharing anything, so here's what I got after half a day.


There'll certainly be stuff that's unimplemented/unsupported still, but I don't plan on working on this anymore until I actually finish playing the game myself. If you've been secretly working on this and have more info than I do, please feel free to help out. The Noesis NIF implementation is in Python, so all of this is already open to anyone. There's certainly enough there to write a usable exporter (just pick a vertex format and reuse the flags) so that the awful monsters making their furry nude mods don't have to use a hex editor anymore.

In order to get everything working like it should, you should extract all of the texture archives to a common directory like so:

Once this is done, you'll have something over 20 gigs of loose textures sitting there. Now point Noesis at the directory you extracted the textures to (and the Textures directory under it, as NIF's seem to like using relative paths on both) for its automatic texture loading, like so:


Once you've done this, the textures will be auto-loaded for albedo/normal/etc. Additionally, many character models have a flattened skeleton, and reference another file (usually called skeleton.nif) with proper hierarchy. The skeleton in those models will look like this:

To address this, use -nifloadskel <filename> (where filename would be the full path to the appropriate file, like skeleton.nif) and you'll get this instead:


Ok then, have fun.

[artworkplay]

When MrAdults surprises, he surprises big. Thank you sir for this awesome addition to Noesis.

[Highflex]

Amazing work!!! Now i can create a proper animation rig with good geometry thank you so much

[jimhsu]

Oh nice. I've been on all the major TES/FO forums watching this stuff. An exporter would definitely be of great public interest.

What format are you extracting the normals/speculars to - ATI2N?

[TheMask85]

Thank you for this! Works like a charm.
There's just one thing unclear to me. Where can i find the function to use "-nifloadskel <filename>"?
I mean how could i open a mesh with parameters such as this? Is there a button or flag i missed?
Thanks again!

[artworkplay]

Thank you for this! Works like a charm.
There's just one thing unclear to me. Where can i find the function to use "-nifloadskel <filename>"?
I mean how could i open a mesh with parameters such as this? Is there a button or flag i missed?
Thanks again!

You can add the function to the Data Viewer Default preview commands respectively export it as well with Advanced options set to that function.

[MrAdults]

Oh nice. I've been on all the major TES/FO forums watching this stuff. An exporter would definitely be of great public interest.

What format are you extracting the normals/speculars to - ATI2N?

The archive itself stores the image information including the DXGI format, so when I'm writing out the DDS files, I'm just constructing DX10 headers and relaying the DXGI format directly.

When Noesis loads DX10 DDS files, it's still assuming that BC5 wants to have z derived followed by renormalization, which is only right if it's a normal map. They're using BC5 a lot to store roughness/smoothness + metalness, so Noesis is trying to normalize those things. I'll get a fix out for that momentarily.

They're also using BC5 to store single-channel spec maps where green and red are the same, which makes no sense at all. They should be using BC4 for those, but typically aren't. It would be exactly the same data at half the size.

Edit: 4.172 is up, which adds a -ddsati2nonorm command. Like artworkplay mentioned above, you can specify it when exporting or as part of your default preview command list. It's safe to always use this option for Fallout 4, because Noesis already derives z using the non-splayed method and renormalizes in its lighting shaders when sampling normal maps. So it will still handle all of the normal maps correctly, and it will stop trying to treat spec/roughness/metalness as normals.

[Bastien]

Half a day, that's impressive.

but no one is sharing anything

Thank you for sharing the code, when you could have just done the same as the other people.

[Highflex]

Trying to load the full human skeleton before loading any parts like MaleBody.nif is just not working for me.

I tried adding ( -nifloadskel FULLPATH ) to the default commandline when starting the preview and also as export option, any idea why?

[MrAdults]

Trying to load the full human skeleton before loading any parts like MaleBody.nif is just not working for me.

-nifloadskel remaps hierarchy, as you can see:

Code: Select all

		for object in self.objects:
			if object.isBone is True:
				if object.name in skelNifBoneObjects:
					skelObject = skelNifBoneObjects[object.name]
					#see if the parent is also a bone, and if so, transfer the parent index
					if skelObject.parentIndex >= 0:
						skelParentObject = skelNif.objects[skelObject.parentIndex]
						if skelParentObject.isBone and skelParentObject.name in nifBoneObjectIndicesByName:
							#for now, we only transfer transforms for non-root objects. otherwise, we'd need to transform up the hierarchy before grabbing,
							#in case there are parents in the skeleton nif not reflected in the nif we're mapping to.
							object.matrix = skelObject.matrix
							object.scale = skelObject.scale
							object.parentIndex = nifBoneObjectIndicesByName[skelParentObject.name]

It doesn't import new skeleton nodes or rebase root transforms for hierarchies not present in the destination nif. (which you wouldn't really want for what you're probably trying to do anyway, you want to keep the relative transform intact) If you want to piece together parts on a skeleton, it's easiest to just export them all and the skeleton separately as something like FBX, then plant the pieces onto the skeleton (match by node names) in your modeler of choice. Also keep in mind -nifnotransform for any geometry you're exporting that doesn't contain the necessary nodes for skinning.

Half a day, that's impressive.

Yeah, when I tried to google for BSTriShape to see if someone else might've already done some work on it, I saw a lot of people saying they "totally redid the NIF format", which is a load of nonsense. As you can see in the Noesis script, the core of what's new there is just a few object types, including a couple of new (and much simpler) shape objects with embedded geometry data. The new bone reference and inverse pose data is very minimal too. Even the material system has very few meaningful changes that depart from Skyrim, with the majority of the core material values and the texture sets remaining mostly the same despite the addition of PBR. So as much as I love to make people think I'm smarter than I am, there wasn't much new stuff to be done.

[dracosfire83]

Love the work you put into this but I do have one question. Is there anyway to export the file with the textures still intact?

[throttlekitty]

Thanks for sharing, MrAdults, you knocked that out quick!

NifSkope has an update for fo4 now as well if you wanted to take a look.
http://niftools.sourceforge.net/forum/v ... =31&t=6305

Also, my working code/math skills are pretty low, I really don't understand how you're handling inverse skin transforms. (that's what led me to this thread) Looking at your py, you seem to move the verts instead of a global mesh transform, is that right?

[MrAdults]

No mention of Noesis, that's a shame. Have to wonder about the timing, especially given how simple and obvious the vertex flags are, but that's the only thing he seems to be having trouble with and the only thing I didn't bother fleshing out and documenting in the script. Maybe just a coincidence, I suppose.

All Noesis is doing with the bindpose mats is moving the actual bind pose to match the node space. Since it converts the position data up to 32 bits in the process, there's no reason for the bind pose given to exist anymore. This means that the node space pose now equals the bind pose. The ideal bind pose can also easily be recalculated on export by fitting joints into a more ideal precision range for FP16, then transforming geo with (newInvBindPose * fittedPose).

[devilblades]

This is my 1st time using Noesis, I see the models, and exporting textures right now. But I do have a question, under the Power armor folder, all the files are 1K How do I load the power armor like you did? I would like to get the helmet that is on the cover of the game. I just need a little help finding that, and the rest of the armor for it. Thank you.

[jonwd7]

No mention of Noesis, that's a shame. Have to wonder about the timing, especially given how simple and obvious the vertex flags are, but that's the only thing he seems to be having trouble with and the only thing I didn't bother fleshing out and documenting in the script. Maybe just a coincidence, I suppose.

Why would we mention you? We had no idea what Noesis was until after we already had 99% of files parsing, and I had models loading in NifSkope over a week ago. (Link #2) In fact, we were pretty sure you were simply working off of our 010 templates, which were publicly pushed to GitHub daily since the 11th, hence extremely easy for you to find with a code search. (You admitted to searching for 'BSTriShape' for example.)

I don't know what you're referring to me having trouble with exactly. It seems like you're talking about vertex flags, but I know that you couldn't possibly be. You're not handling them correctly either. I checked out certain problem meshes and you assumed many things incorrectly. The vertex flags are not simple OR obvious. Look at maleeyesshade.nif in Actors/Character/CharacterAssets/FaceParts. You get that one totally wrong, assuming the body has normals for example. We've analyzed all the vertex flags with 010 and there are 17 unique vertex flags across all NIF files. Currently nif.xml only handles most of them, not because we don't know how to handle the outliers (we do), but because doing so would make nif.xml very messy.

Anyway, since we found out about this, we've found your pompous attitude very offputting, so I personally don't have a desire to interface with you any further. Your figuring out *anything* in a "half a day" was highly suspect too, since we'd already published everything by that point. From our point of view, you don't seem to know anything we didn't know a week ago, save for displaying skinned models correctly. And at least for me that's simply because I know nothing about skinning and haven't even bothered to look into it. Figment on the other hand had skinned models imported into 3ds Max long before your Noesis update too.

I think maybe what you're not understanding is that what you've done in "half a day" is fine for Noesis, but not for the actual tools. We need to treat the format rigorously, and it's clear you're not doing so. Doing things correctly takes time, and we took longer to release because we were doing the responsible thing by not handing out tools which will create broken NIFs.

I guess before I go I'll leave you with a quiz regarding the vertex format:

Code: Select all

  0: 00 00 00 00 | 00 00 00 00 | 00000000 00000000 00000000 00000000 | 00000000 00000000 00000000 00000000
  2: 02 00 00 00 | 00 10 00 00 | 00000010 00000000 00000000 00000000 | 00000000 00010000 00000000 00000000
  3: 03 00 02 00 | 00 90 00 00 | 00000011 00000000 00000010 00000000 | 00000000 10010000 00000000 00000000
  3: 03 02 00 00 | 00 30 00 00 | 00000011 00000010 00000000 00000000 | 00000000 00110000 00000000 00000000
  4: 04 00 02 03 | 00 90 02 00 | 00000100 00000000 00000010 00000011 | 00000000 10010000 00000010 00000000
  4: 04 02 00 03 | 00 30 02 00 | 00000100 00000010 00000000 00000011 | 00000000 00110000 00000010 00000000
  4: 04 02 03 00 | 00 b0 00 00 | 00000100 00000010 00000011 00000000 | 00000000 10110000 00000000 00000000
  5: 05 02 03 04 | 00 b0 02 00 | 00000101 00000010 00000011 00000100 | 00000000 10110000 00000010 00000000
  5: 05 02 43 00 | 00 b0 01 00 | 00000101 00000010 01000011 00000000 | 00000000 10110000 00000001 00000000
  6: 06 02 00 30 | 00 30 04 00 | 00000110 00000010 00000000 00110000 | 00000000 00110000 00000100 00000000
  6: 06 02 43 05 | 00 b0 03 00 | 00000110 00000010 01000011 00000101 | 00000000 10110000 00000011 00000000
  7: 07 02 00 43 | 00 30 06 00 | 00000111 00000010 00000000 01000011 | 00000000 00110000 00000110 00000000
  8: 08 02 43 50 | 00 b0 05 00 | 00001000 00000010 01000011 01010000 | 00000000 10110000 00000101 00000000
  8: 08 02 43 50 | 20 b0 05 00 | 00001000 00000010 01000011 01010000 | 00100000 10110000 00000101 00000000
  9: 09 02 43 65 | 00 b0 07 00 | 00001001 00000010 01000011 01100101 | 00000000 10110000 00000111 00000000
  9: 09 02 43 65 | 20 b0 07 00 | 00001001 00000010 01000011 01100101 | 00100000 10110000 00000111 00000000
 10: 0a 02 43 65 | 90 b0 17 00 | 00001010 00000010 01000011 01100101 | 10010000 10110000 00010111 00000000

(1st number is the first byte of the flags in dec. Rest is 8 bytes of flags in hex and binary)

1) Which format is found exclusively in male eyes?
2) Which format(s) are exclusive to BTR files?
3) Which formats have normals but no tangents?
4) Which formats are found only in files under /facegeom/?
5) Which format has normals, no tangents, and vertex colors?
6) Which formats describe a half float after the vertex, and which have a ushort instead?

And a lightning round question:

1) Do you know where the bitangent is located? (Hint: It's a doozy)

... and no cheating. Some of this information is already public!


P.S. Don't flatter yourself, I haven't looked at your source. I'm not sure if our IRC logs are available somewhere, but they will attest to that. I'll let you know if our IRC logs are available somewhere and you will be free to look over the past two weeks and see for yourself that you had absolutely zero to do with our work.