Important information: this site is currently scheduled to go offline indefinitely by end of the year.

Northlight Tool (also known as "The Alan Wake Engine")

General game file tools that are useful for more than one game
volfin
ultra-veteran
ultra-veteran
Posts: 452
Joined: Sun Jul 06, 2014 6:30 am
Has thanked: 110 times
Been thanked: 326 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by volfin »

Bladers wrote: Mon Sep 09, 2019 6:40 pm That's so Neat!

What do you think about the animations? @daemon1 says its basically impossible or rather improbable. But what do you think?
Can it be done and be done in a reasonable timetable?
I have no plans to attempt animations. If I get skinned models with skeletons done I'll consider myself lucky. :P

Since I'm posting I'll give an update. I have UV Mapping working, and multi-material decoding done as well.

Image

multiple materials per model turned out to be very challenging because of a neat feature they implemented, which added a lot of complexity.
Image
Basically each LOD model can have an arbitrary number of materials applied. and even unique materials for every LOD if they wanted. In the above case, the file had 3 materials defined, material 1 and 2 were used on LOD 0, but material 3 was only used on LOD 1. I have all this working now.

I'm also seeing that texture paths aren't reliable as defined in the files:
Image
Here you see the paths given are saying 'centurion\batch03' but the texture files are in fact in 'centurion\batch01'

Image
for LOD1's unique textures, it was similarly confusing, the file stated the textures would be in 'centurion\batch03\lods' but they were instead in simply 'batch03\lods'. at least in all cases I've seen so far the filenames are consistent.

blender can't read DX10 DDS files so manual conversion to TGA or another format they can read is of course required, but with textures in blender:
Image

I'm still finding some exceptions I need to address, and then I can start trying for skinned models and skeletons. I'll probably make a release without skinning support once it's stable so people can mess with static models in the meantime.
volfin
ultra-veteran
ultra-veteran
Posts: 452
Joined: Sun Jul 06, 2014 6:30 am
Has thanked: 110 times
Been thanked: 326 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by volfin »

I found another nifty feature I had to add support for, the 'breakable' items are defined as one solid mesh, however, there's a table with information about how they get broken up into parts when you 'break' them. So I figured it would be nice to have an option ("Split breakable meshes" in the plugin options, it's on by default) to break these meshes up as intended. For example the chair:
Image

this option even works across LODS:
Image

Since it all seems to be fairly stable at this point i'm releasing 1.0 which works for blender 2.70 thru 2.79 (2.80 version *may* happen at a later date)

Remember this doesn't yet have support for skinned meshes, so trying those will probably result in failure. If you find any static/breakable meshes that won't load or don't load right, let me know the file's name and i'll try to fix it.

If you need help installing the plugin into Blender, check my tutorial here: viewtopic.php?p=103131#p103131

* New version posted below *
Last edited by volfin on Thu Sep 19, 2019 5:24 pm, edited 1 time in total.
Bladers
veteran
Posts: 92
Joined: Mon Aug 11, 2014 1:20 am
Has thanked: 15 times
Been thanked: 10 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by Bladers »

volfin wrote: Thu Sep 12, 2019 7:59 am I found another nifty feature I had to add support for, the 'breakable' items are defined as one solid mesh, however, there's a table with information about how they get broken up into parts when you 'break' them. So I figured it would be nice to have an option ("Split breakable meshes" in the plugin options, it's on by default) to break these meshes up as intended.
wow you have outdone yourself. can't wait to get home and get my hands on it.
User avatar
dennysbr
ultra-n00b
Posts: 6
Joined: Fri Sep 13, 2019 3:40 am
Has thanked: 11 times
Been thanked: 2 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by dennysbr »

Hey guys, I convert the jesse files but it's binfbx, I don't know this format, did one of you make any breakthrough? Will we ever have a jesse nude? :o
volfin
ultra-veteran
ultra-veteran
Posts: 452
Joined: Sun Jul 06, 2014 6:30 am
Has thanked: 110 times
Been thanked: 326 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by volfin »

characters are skinned meshes. like I said I haven't yet added support for skinned meshes. Give me time. :)

as for nude, I have no idea lol, I doubt that's in the game though.
tivac
ultra-n00b
Posts: 2
Joined: Fri Sep 13, 2019 7:31 am
Has thanked: 1 time
Been thanked: 2 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by tivac »

Anybody have any clues what the format is for the .ui files? I was able to get them all extracted from the .rmdp using the Northlight Tool (which was awesome and super-straightforward to use, thanks!) and while I can poke at the contents a bit in a hex editor it's been a long time since I wrote an unpacker script. If someone has already done the research I'd really appreciate it!
volfin
ultra-veteran
ultra-veteran
Posts: 452
Joined: Sun Jul 06, 2014 6:30 am
Has thanked: 110 times
Been thanked: 326 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by volfin »

well according to the splash screen, they are using Coherent Labs' UI system:
Image

https://coherent-labs.com/products/coherent-gameface/
According to their website, their UI system is based on HTML5 and javascript, which seems backed up by what I saw when I peeked in a UI file, it looks like it's a pretty simple package with HTML/XML/and JavaScript all bumdled in it. I have never heard of Coherent, so I don't know how often they have been used in games, I have a feeling not much, so I doubt anything already exists for their UI system.

But it looks pretty simple

Image
Looks like they use the signature 0xD34DB33F (good ol DEADBEEF) is a record block signifier (marked in green) and then it's just long integers and floats of data and strings with string length coming before. it's a great project for a beginner really.
tivac
ultra-n00b
Posts: 2
Joined: Fri Sep 13, 2019 7:31 am
Has thanked: 1 time
Been thanked: 2 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by tivac »

I'm actually interested specifically because it's using Coherent, as I'm a JS programmer :). I manually extracted a couple of files to poke at them but hopefully with your tip on the 0xD34DB33F identifier I can write a small unpacker and get a better sense of how it all fits together.

Thanks so much for the help!
Bladers
veteran
Posts: 92
Joined: Mon Aug 11, 2014 1:20 am
Has thanked: 15 times
Been thanked: 10 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by Bladers »

volfin wrote: Thu Sep 12, 2019 7:59 am Since it all seems to be fairly stable at this point i'm releasing 1.0 which works for blender 2.70 thru 2.79 (2.80 version *may* happen at a later date)

Remember this doesn't yet have support for skinned meshes, so trying those will probably result in failure. If you find any static/breakable meshes that won't load or don't load right, let me know the file's name and i'll try to fix it.

If you need help installing the plugin into Blender, check my tutorial here: viewtopic.php?p=103131#p103131
Looks like dead_letter_walls.binfbx and similar objects like walls floors are giving an error? These can't be skinned meshes.
Any ideas and updates?

I'm using blender 2.79b by the way
volfin
ultra-veteran
ultra-veteran
Posts: 452
Joined: Sun Jul 06, 2014 6:30 am
Has thanked: 110 times
Been thanked: 326 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by volfin »

Bladers wrote: Mon Sep 16, 2019 3:35 am
volfin wrote: Thu Sep 12, 2019 7:59 am Since it all seems to be fairly stable at this point i'm releasing 1.0 which works for blender 2.70 thru 2.79 (2.80 version *may* happen at a later date)

Remember this doesn't yet have support for skinned meshes, so trying those will probably result in failure. If you find any static/breakable meshes that won't load or don't load right, let me know the file's name and i'll try to fix it.

If you need help installing the plugin into Blender, check my tutorial here: viewtopic.php?p=103131#p103131
Looks like dead_letter_walls.binfbx and similar objects like walls floors are giving an error? These can't be skinned meshes.
Any ideas and updates?

I'm using blender 2.79b by the way
I'll give them a look, thanks.
volfin
ultra-veteran
ultra-veteran
Posts: 452
Joined: Sun Jul 06, 2014 6:30 am
Has thanked: 110 times
Been thanked: 326 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by volfin »

yeah about the walls, it's failing because of a new never-seen shader parameter:
Image

each parameter has to be added by name, because each has a possibly different number of arguments. First time seeing that one, so i'll have to add it to the plugin.

I'm making good progress on skinned models, skeleton decoding is done, and i think i see how vertice weights are stored. I just need a bit more time and I'll have it done.

Image
volfin
ultra-veteran
ultra-veteran
Posts: 452
Joined: Sun Jul 06, 2014 6:30 am
Has thanked: 110 times
Been thanked: 326 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by volfin »

Image

Ok, I think I have everything 'nearly' there. Weights and indexes are decoded and everything looks good. It's a bit insane that even NPCs have over 500 bones, but this is remedy I guess :P

I have a lot to do still, I found out the models were mirrored on the X axis, so be aware of that for version 1.0, it will be fixed in 1.10. As well I've vastly improved the shader parameter processing so there shouldn't be any more unknown shader param problems (unless it's really something wild you find).

As well I've noticed a peculiar issue with some models that have transparent sections/decals. the data is structured in such a way the transparent parts aren't defined in the model table. The data is there, it's just not drawn... Maybe some engine optimization or who knows. problem is if there's no table entry for the data, my plugin doesn't know to build it. So I need to figure out a way to detect this situation and insert a dummy entry or something.

Once I've resolved all these issues and have tested it across more models, I'll release 1.10
User avatar
dennysbr
ultra-n00b
Posts: 6
Joined: Fri Sep 13, 2019 3:40 am
Has thanked: 11 times
Been thanked: 2 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by dennysbr »

very nice :) congratulations
volfin
ultra-veteran
ultra-veteran
Posts: 452
Joined: Sun Jul 06, 2014 6:30 am
Has thanked: 110 times
Been thanked: 326 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by volfin »

Ok, what everyone wants finally :mrgreen:

Image
Image

I think i can safely say it is done, everything seems to work great.

Here's the change log from 1.0:
# 1.10 - Added initial support for rigged characters
# - mapped shader type table, now detects 8/16 bit bone index values, 16/32 bit face index values, and 4/8 weight per vertex usage
# - added skeleton building support. *.binskeleton file must be present (and if one exists it's always next to the model anyway)
# - Fixed issue with models being flipped on X axis
# - rewrote shader parameter parsing, should better handle unexpected parameters
# - worked around issue where transparent/decal sections don't have a model entry, and weren't created.
lot of parts i redid to make it work more reliably and i have to say it's looking good.
I checked every Jesse model, all of the costumes are there, even the preorder, PS/4 exclusive, etc.
And even one I think nobody has seen before called "Expedition":

Image
Image

Have to wonder if there is some challenge that hasn't been found yet, or for the upcoming DLCs...

Oh yeah and I finished converting it to Blender 2.80 also, if you're into that. I use the "collections" instead of layers but if you're using 2.80 you should already know about collections anyway. :P

As always if you find issues let me know.

and as Ahti would say:
Image
Nauttia (enjoy)



* New version posted on next page *
Last edited by volfin on Fri Sep 20, 2019 5:07 am, edited 6 times in total.
volfin
ultra-veteran
ultra-veteran
Posts: 452
Joined: Sun Jul 06, 2014 6:30 am
Has thanked: 110 times
Been thanked: 326 times

Re: Northlight Tool (also known as "The Alan Wake Engine")

Post by volfin »

For some reason it won't let me put a second attachment on the post, so here's the blender 2.80 version.

* new version posted on next page *
Last edited by volfin on Fri Sep 20, 2019 5:07 am, edited 1 time in total.
Post Reply