Havok binary files

[HunterAP]

How did you manage to convert from the HKX bone coordinates and rotations to SMD format?
If I could find a way to batch the process, I could get full DSIII animations out.

This is a project I started last year for Alien:Isolation, it's a HKX converter that outputs SMD skeleton. It works with 2012.2.0 and 2010.2.0, I had intended to take it all the way with support for many versions, and animation output, but I simply moved on to other projects. It's written in C#, a lot of the file structure is defined at the top of Program.cs, and maybe will help people, so I'm making the code public.

https://bitbucket.org/Volfin/hkx2smd/overview

I tried building your program as both x86 and x64, normally running the 2014.1.0-r1 HKX files give some errors in the header, but if I copy-paste the 2012.2.0-r1 header over the original, the program reads everything and gets up to "class hkaskeleton signature: 0xFEC1CEDB", then after a few seconds the program crashes with the "System.OutOfMemoryException" error. Looking it up says that I don;t have enough memory, even though I have 16GB of RAM, but I'm assuming that your program is looking for a big chunk of sequential memory that I may not have if my memory usage is fragmented.

What's interesting is that I've tried this with multiple skeletons, and it doesn't matter what size the HKX file is, I still get the error.

I would fix the issue myself, but I'm not versed in C#, so I don't know how to fix it in the code myself.

If you'd like, I could just send you a HKX sample of the skeleton and animations.

[volfin]

Well the point of me releasing it is that I have no time to mess with it anymore (haven't touched it since Alien:Isolation). I know it worked for Alien: Isolation and Skyrim, but don't guarantee anything else. The problem is HKX is very customizable so the format varies widely from game to game, and then on top of that many developers modify it in-house to do even more strange things, and some even obfuscate it purposely to prevent people reverse engineering their animations.

So likely the code will have to be tweaked for every particular game, no small job. That's why I gave the code out.

[philgrf]

As I understand, the spline compression is a technic that is animation specific.

See this article at riot game engineering
https://engineering.riotgames.com/news/ ... ation-data
So, it is likely there is no simple link between the hkx file and the animation keys.

Maybe the way hkx animation is encoded is similar to the one described in the article that is to say :
- rotation is encoded as quaternion (x,y,z,w),
- the highest value among (x,y,z,w) is discarded as x^2+y^2+z^2+w^2=1 and can thus be reconstructed from the 3 others,
- bytes need to be read according to a key (m bytes for discarded value, n bytes for remaining value1, n bytes for remaining value2, n bytes for remaining value3)

edit : this article at bitsquid describes also the technic
http://bitsquid.blogspot.fr/2009_11_01_archive.html

Best regards

Phil

[SergeantJoe]

The SDK also has a LOT of documentation files in it, including files for all the compression types instead of just Spline. It's not available through Intel anymore but I managed to save copies. Feel free to PM me for the link.

[philgrf]

Sergeant Joe,

I thank you for your reminding of the Intel documentation.
I have downloaded havok 2013 before Intel closed the public access.
At the time I downloaded, I couldn't make any sense of spline compression.

Now, I understand better :
After spending some time on xentax forum, I started to get acquainted with binary files ans lately with file compression.
Following your advice, I looked again at havok 2013 help files.

The game I'm interested in is using spline compression.
So, I restricted my field of research the spline compression.

There are havok default settings for spline compression :
Position : 16 bits
Rotation : 40 bits
Scale : 16 bits
Float (what for ? timeline ?) : 16 bits
Frames per block : 256

As the spline compression is highly customizable(*) in havok, this should only be considered as a reference.
At least, it gives the structure of the data.

I am a beginner at reading binary files but it looks like an interesting challenge :

- identify blocks,
- identify number of frame per blocks,
- guess bone ordering,
- guess quantization for each bone,
- translate into skeleton pose,
- translate into animation.

I need to evaluate the required time.

Best regards

Phil

*() : Even, a per bone customization is possible. The doc says :
"Judiciously setting compression parameters per bone can result in a significant reduction in output data size. One such use case is to create level of detail compression by grouping bones roughly by depth in the hierarchy; tight tolerances could be user for "major bones" such as spine and arm bones while more relaxed tolerances for "minor bones" such as hands and fingers could be used to gain additional savings. A similar use case might involve grouping bones as "upper body" and "lower body". Extremities such as hands often contain a significant number of bones though their visual impact may be limited."

[BloatheadSorcerer]

Does anyone have a working link for the script that flips the skeleton or model because when I open animation in 3dsmax the skeleton has the right animation but the model is all over the place.

[zweihard]

Would anyone mind helping me out with obtaining the animation data for Dark Souls III?

Regards

[SergeantJoe]

Would anyone mind helping me out with obtaining the animation data for Dark Souls III?

Sorry, you're out of luck. Havok formats are incredibly difficult, even with all the documentation nobody has been able to figure it out.

[zweihard]

Would anyone mind helping me out with obtaining the animation data for Dark Souls III?

Sorry, you're out of luck. Havok formats are incredibly difficult, even with all the documentation nobody has been able to figure it out.

Reverse engineering job huh?

[Ginsor]

Sorry for bringing up this topic again, but Im kinda stuck with a havok file here.

I got a havok binary with the hk_2012.2.0-r1 header. However, none of the converters seem to work with it (even when renaming the header) and the original havok preview tool says something like "wrong platform".

Anyone has an idea of what is so special/wrong about that file?

hk_2012.2.0-r1 binary

[SergeantJoe]

I honestly have no idea, and there's nobody else that can help. Sorry, you just have to give up.

[Ginsor]

I honestly have no idea, and there's nobody else that can help. Sorry, you just have to give up.

well, there are some people that made the converters, so there should be some other help out there...in theory haha
however, thanks for looking into it!

[Loomy]

Sorry for bringing up this topic again, but Im kinda stuck with a havok file here.

I got a havok binary with the hk_2012.2.0-r1 header. However, none of the converters seem to work with it (even when renaming the header) and the original havok preview tool says something like "wrong platform".

Anyone has an idea of what is so special/wrong about that file?

hk_2012.2.0-r1 binary

Been a while since i tried my luck figuring this out, but just if it helps: look out for the c# project shared by the awesome (if not mistaken) SeargentJoe, (hkx2smd, and there is another one, or similar project on github, with some fiddling managed to parse some basic 2013 structs, im looking to research splinecompressed anims & bone/weight/blendweight conversion. though quite difficult to find older/recent havok src. (looking for 2011r3 & 2013+)

Just off top of my head, and old snippet idk if works:- look for this flag in ur data which specifies platform type and modify to ur build platform. also thinking might be some endianness conversion needed ( ? )

public enum hkPlatform
{
HCL_PLATFORM_WIN32 = 0x0,
HCL_PLATFORM_X64 = 0x1,
HCL_PLATFORM_MACPPC = 0x2,
HCL_PLATFORM_IOS = 0x4,
HCL_PLATFORM_MAC386 = 0x8,
HCL_PLATFORM_PS3 = 0x10,
HCL_PLATFORM_XBOX360 = 0x20,
HCL_PLATFORM_WII = 0x40,
HCL_PLATFORM_LRB = 0x80,
HCL_PLATFORM_LINUX = 0x100,
HCL_PLATFORM_NGP = 0x400,
HCL_PLATFORM_ANDROID = 0x800,
HCL_PLATFORM_CTR = 0x1000//4096
}
if u look up my old posts, io posted 010 template for havok while back, somewhere on forum, including offset of Platform flag (i didnt have much joy parsing PS3 files on win32 build)

[HunterAP]

After all this time I'm also trying to get some newer formats of HKX working. Right now, Volfin's hkt2smd is crashing because newer formats report that for a variable used in the code, there is over 1.5 billion elements in an array (AKA way too much to work with), and I'm not sure how to go about patching it.

Also, interestingly, trying to use hkt2smd on Dark Souls 1 has a similar error but in a different section.

[Loomy]

After all this time I'm also trying to get some newer formats of HKX working. Right now, Volfin's hkt2smd is crashing because newer formats report that for a variable used in the code, there is over 1.5 billion elements in an array (AKA way too much to work with), and I'm not sure how to go about patching it.

Also, interestingly, trying to use hkt2smd on Dark Souls 1 has a similar error but in a different section.

can u post here sample of both assets ur trying to parse ? i can take a look.