Important information: this site is currently scheduled to go offline indefinitely by end of the year.

Black Desert Online File Formats (PAB, PAC, PAA)

Post questions about game models here, or help out others!
ArthainBaka
ultra-n00b
Posts: 3
Joined: Wed Aug 27, 2014 3:24 am
Has thanked: 2 times
Been thanked: 7 times

Black Desert Online File Formats (PAB, PAC, PAA)

Post by ArthainBaka »

Update:
I believe I've cracked pretty much all of it just need to determine the format of the keyframe values and we are done.
Explaination of converting the Animation's compressed scale, rotation & position can be found in my other post here.

I feel I've almost figured out the details for all of these formats but I may need some input for the last stretch. Huge credits go to Ekey for his PAZUnpacker_CBT2 (without which I'd have nothing to do!), and to chrrox for their Noesis script (which is pretty much the foundation of what I've got). There are still a few things that aren't quite complete, primarily with the PAA files which I believe have some form of compression which I talk about in their section any input here would be awesome.

For those not familiar with the files:
  1. PAC - This file represents the model format (chrrox mapped most of this out in their script)
  2. PAB - This file represents the skeleton format which various models depend on (chrrox mapped this out in their script)
  3. PAA - This file represents the animation format, one file per animation
Common Structures
Below are some of the common structures in each of these files, listing them here will save some space and typing. They are pretty self explainatory. If anyone knows more about the unknown data in the FileHeader I'd be interested to know what it is, it hasn't seemed to be required so I haven't really looked at it.

Code: Select all

typedef struct FileHeader
{
	char					 fileId[4];
	unsigned_int16		fileVersion;

	unsigned_int8		 unknown[10];
} FileHeader;

typedef struct Matrix4x4
{
	float 		data[4][4];
} Matrix4x4;

typedef struct Vector3
{
	float		x;
	float		y;
	float		z;
} Vector3;

typedef struct Quaternion
{
	float		x;
	float		y;
	float		z;
	float		w;
} Quaternion;

typedef struct CompressedVector3
{
	int16		x;
	int16		y;
	int16		z;
} CompressedVector3;

typedef struct CompressedQuaternion
{
	int16		x;
	int16		y;
	int16		z;
	int16		w;
} CompressedQuaternion;
PAB File Format
This file starts with the standard FileHeader structure with the main data starting at 0x00000010 which lists the number of bones in the file and is followed by the array of bones.
I believe the unknown data is just a padding or chunk end or something, either way it doesn't seem to be required.
Once again, credit to chrrox who's Noesis script is the framework for this.

Code: Select all

typedef struct Bone
{
	unsigned_int32		boneHash;
	unsigned_int8		 boneNameLength;
	char				    boneName[boneNameLength];
	unsigned_int32		boneParent;

	Matrix4x4			  boneMatrix;
	Matrix4x4			  boneMatrixInverse;
	Matrix4x4			  boneLocalMatrix;
	Matrix4x4			  boneLocalMatrixInverse;
	Vector3			    boneScale;
	Quaternion		    boneRotation;
	Vector3			    bonePosition;

	unsigned_int8		 unknown[2];
} Bone;

struct BoneFileFormat
{
	FileHeader		 fileHeader;

	unsigned_int16	boneCount;
	Bone 				bones[boneCount];
};
PAC File Format
Moving up in complexity slightly, this file has the mesh data. Each mesh has a texture name attached to it, I believe this is a base name so textures with other semantics (like normal map, ambient occlusion, specular power, etc) can be automatically added by searching for their relevant semantic id which is appended to the base texture name. I haven't really looked into determining what PAB file is supposed to be used for these yet, if it's not in the file then it may be a matter of searching the directory (and parent directories with a depth restriction) to find a PAB file which has the bone hashes that the PAC file contains. I've only included the version 259 of this, I haven't seen where the 513 version chrrox also uses is used (but there are a lot of models so easily possible I've missed it!)
Once again, credit to chrrox who's Noesis script is the framework for this.

Code: Select all

typedef struct VertexBuffer
{
	Vector3			   position;
	unsigned_int8		normals[4];			// Commented out by chrrox
	unsigned_int16     uv[2];
	unsigned_int8      colour[4];			// Commented out by chrrox
	unsigned_int8      boneIndices[4];
	unsigned_int8      boneWeights[4];
} VertexBuffer;


typedef struct LOD
{
	unsigned_int16			vertexCount;
	VertexBuffer           vertexBuffer[vertexCount];
	unsigned_int32			faceCount;
	unsigned_int16			faceIndices[faceCount];
} LOD;

typedef struct Mesh
{
	unsigned_int8			 textureNameLength;
	char			          textureName[textureNameLength];
	unsigned_int16			flag;

	LOD				        lodArray[3];
} Mesh;

struct ModelFile
{
	FileHeader 		      fileHeader;

	unsigned_int8			 boneCount;
	unsigned_int32			boneHashArray[boneCount];
	unsigned_int8			 usedBoneCount;
	int8			          usedBoneArray[usedBoneCount];

	unsigned_int32			totalVertices;
	unsigned_int32			totalFaces;
	unsigned_int16			meshCount;

	Mesh 			         meshes[meshCount];
};
PAA File Format
This file was a pain, it's easy knowing what data needs to be in an animation file yet people find so many different ways to do so. I believe there is some form of compression being used within this file for the scale, rotation and position (and possibly key frame / frame time).
The file starts the same as normal with the FileHeader followed by a count which for this file represents the number of bones in the animation, following this is 8 bytes of data (first 4 I believe are a float for the total animation time) and from here the bone animation data starts.
The animation data appears to compress all of the floats (e.g. time frame, scale, rotation and position components) into half's. I'm not so good at decryption so maybe someone can help out, at the least we can logically infer from the scale values that "E8 03" should equal 1.0F. Once again I such ad decompression so I'd love some input here.
With this knowledge the format for the animation file is as follows:

Code: Select all

typedef struct ScaleData
{
	half						keyframe;
	CompressedVector3	 keyframeScale;  // Multiply values by 0.001F to get float value
} ScaleData;

typedef struct RotationData
{
	half						 keyframe;
	CompressedQuaternion	keyframeRotation;  // Multiply values by max value of their sign to get float value
                                                                  // value == 0 : (0.0F)
                                                                  // value < 0 : (value * 32768.0F)
                                                                  // value > 0 : (value * 32767.0F)
} RotationData;

typedef struct PositionData
{
	half						keyframe;
	CompressedVector3	 keyframePosition;  // Multiply values by 0.1F to get float value
} PositionData;

typedef struct AnimBoneData
{
	unsigned_int32		boneHash;

	unsigned_int16		scaleKeyframeCount;
	ScaleData			  scaleData[scaleKeyframeCount];

	unsigned_int16		rotationKeyframeCount;
	RotationData		  rotationData[rotationKeyframeCount];

	unsigned_int16		positionKeyframeCount;
	PositionData		  positionData[positionKeyframeCount];
} BoneAnimationData;

struct Animation
{
	FileHeader			 fileHeader;
	unsigned_int16		boneCount;

	float				   animationDuration;	// ?
	unsigned_int8		 unknown[4];			 // ?

	AnimBoneData        	boneData[boneCount];
};
Last edited by ArthainBaka on Wed Aug 27, 2014 10:25 am, edited 2 times in total.
User avatar
BloodForce
ultra-n00b
Posts: 5
Joined: Wed Oct 30, 2013 3:49 pm
Has thanked: 2 times
Been thanked: 1 time

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by BloodForce »

You forget .pam (object files)
i think all our answers in \object\parc.exe
help for parc.exe, getting in korean windows
Pearl-Abyss Resource Compiler Ver1.0 01:11:13 08/25/14

Pearl-Abyss RC Command-line Interface:
-h [--help] Help
-i [--input_file] arg input file name
-o [--output_file] arg output file name
-I [--input_files] Arg input file name list
Save -p [--output_path] arg the output file folder
-b [--base_path] top as a reference when creating sub-folder structure path arg
create sub-folder structure when writing files -k [--keep_subpath] in the output folder,
If the file has -w [--overwrite] overwrite the same name exists
-l [--log_output_list] logging output file list
compilation -a [--all] for all files
performed recursively for -r [--recursive] in the sub-folder
create a texture reference list -t [--texture_list] mesh file
-g [--retargetting_animation] for generating animation retargeting
-v [--verbosity] detail message output
Doing a -s [--single_thread] single-threaded
Perform compile only files --bone_only pb
-m [--skinned_mesh_only] only pc file compilation
Compiled by -e [--check_coexist] Raw data files are guys and pair
Verify that the coexistence
-c [--change_mode] arg object to change compiler (c followed by n (General)
r (r3m) c (Curly Religion) h (House) l (LOD)
Put the flag should generate a file for the flag
k (key frame Animation) m (morphing) p (pam) text
Reads the file.
Do other files created in r3m -x [--change_readfile] is not the Text
Change the deviation -u [--reduce_deviation] the arg object must
If -d [--delete_readfile] the object mode and delete the file read.




Default: PARC.exe -cp -a

Object vertex to reduce levels (less if you are using does anhal.)
-u + a ~ f a = 0.5 (-ua)
b = 1.0 (-ub)
c = 1.5 (-uc)
d = 2.0 (-ud)
e = 2.5 (-ue)
f = 3.0 (-uf)
If the change to all sub-folders, you can simply put the -r.
ArthainBaka
ultra-n00b
Posts: 3
Joined: Wed Aug 27, 2014 3:24 am
Has thanked: 2 times
Been thanked: 7 times

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by ArthainBaka »

BloodForce wrote:You forget .pam (object files)
i think all our answers in \object\parc.exe
help for parc.exe, getting in korean windows
I had a look at it but for whatever reason my IDA chucks a fit when it tries to analyse the file and I can't get it to run for me (though maybe it's because I'm on an English install with Korean language pack). Either way I had no luck with it, or analysing the main .exe for the client.
I haven't really looked into the PAM files, so I'm not really sure what they are yet. I'm more interested in the model + animations. Once I've got those I might look at the PAM files.


Anyway I think I've cracked it!

Rotation Data
The rotation data it's being stored in four int16's and a bit of trickery to deserialize it into a float. Basically you just need to read the int16 value and and divide it by the largest value (for the given sign) to yeild the float data for the quaternion.
So if we look at some sample data:

Code: Select all

// Sample PAA Format Quaternion Data
7E A5 82 5A 00 00 00 00
If we break this into the format we know it's in (an array of four int16's) we get the following:
Break this into an array of four int16's

Code: Select all

Hex: [7E A5, 82 5A,  00 00, 00 00]
Decimal: [-23170, 23170, 0, 0]
Now we can convert it into float data using some simple math (my psuedo code always tends to be C++ for some reason >.>):

Code: Select all

#define FMAX_INT16    32767.0F
#define FMIN_INT16    32768.0F     // Positive value to preserve sign on division

// Unpacks int16 value (of a quaternion component) from the .PAA format into a float
float UnpackValue(int16 value)
{
    if (value < 0)
    {
        return ((float)value / FMIN_INT16);
    }
    else if (value > 0)
    {
        return ((float)value / FMAX_INT16);
    }

    // If we get this far then the value has to be 0
    return (0.0F);
}
So if we apply this to the data we get the following (which now surprisingly looked like real quaternion values):

Code: Select all

float: [-0.70709F, 0.70711F, 0,0F, 0.0F]
Scale Data
Every single file I've looked at has had the same E8 03 values, interestingly this nicely converts to an int16 as 1000.
Without spending time looking into it I'd say it's pretty safe to assume this is a fixed decimal point representation, so just mutliply by 0.001F (so people know we want a float, otherwise you could just divide by 1000) to get the data as a float (results with 1.0F which I think is obviously a good standard value for bone scale):

Code: Select all

int16: [1000, 1000, 1000]
// Multiply all values by 0.001F to yield float
float: [1.0F, 1.0F, 1.0F]
Position Data
This is where I may be wrong but it seems to be correct for all the models I'm cross-referencing with positions of exported in Maya (standard PAB bone matrices used).
It seems this is just another fixed decimal point value, which is shifted by 1, so multiply the int16 by 0.1 or divide by 10 (outcome will be the same so personal preference really, probably safer dividing by 0.1 as then people know the output is a float and not a decimal which would ignore the decimal values)
From data of the Windmill model, in Maya the "Bone_VerticalAxis" has the following position [-1371.124, -0.001, -0.000], now we know it's from different bone data so really if we can get one value close to this and have the process work for the rest of the bones similarly we can pretty much assume it's correct (or good enough).
Black Desert seems to not use a world scale of 1 = 1m (unless this windmill is almost 1.4km high >.>) so given that it's less than 1 unit = 1m then the single decimal resolution isn't really that bad (I'm guessing they use 1 unit = 1mm).
Anyway, data and results this one like scale is pretty simple:

Code: Select all

int16: [-13711, 0, 0]
// Multiply values by 0.1F for float
float: [-1371.1F, 0.0F, 0.0F]  // Pretty much identical to Maya target of [-1371.124, -0.001, -0.000]
So nice and easy!
That just leaves the keyframe values the be determined and we'll have a complete functional format. Hopefully I'll get enough time to finish it today ^.^
Last edited by ArthainBaka on Wed Aug 27, 2014 10:39 am, edited 1 time in total.
Rynage
n00b
Posts: 15
Joined: Thu Oct 31, 2013 2:55 pm
Has thanked: 5 times
Been thanked: 3 times

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by Rynage »

Just wow! You guys are incredible. If you will be able to unpack all those files - just no words.
ArthainBaka
ultra-n00b
Posts: 3
Joined: Wed Aug 27, 2014 3:24 am
Has thanked: 2 times
Been thanked: 7 times

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by ArthainBaka »

Rynage wrote:Just wow! You guys are incredible. If you will be able to unpack all those files - just no words.
I think I only did the easy part!
Ekey was the one that made the unpacker which makes all of this possible, I can guarantee without it I wouldn't have even looked into this!
And chrrox provided enough of a framework with his Noesis script that I could figure out what I have so quickly.
I can not thank those two enough.

I do a lot of game development so I think that helped me a lot since I work with these structures on pretty much a daily basis, I've never tried to unpack any file formats before (actually I recall that I datamined Fury when it came out but the files were essentially plain text) so this was actually really really fun (and oddly relaxing).

I don't think I have enough time to get a working script with Noesis today though as I need to go out for the rest of the night shortly. Tomorrow should be plenty of time to learn enough Python (for a programmer I often wonder why I know almost nothing about Python) and enough of the Noesis API to get something working.
Rynage
n00b
Posts: 15
Joined: Thu Oct 31, 2013 2:55 pm
Has thanked: 5 times
Been thanked: 3 times

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by Rynage »

ArthainBaka wrote:
Rynage wrote:Just wow! You guys are incredible. If you will be able to unpack all those files - just no words.
I think I only did the easy part!
Ekey was the one that made the unpacker which makes all of this possible, I can guarantee without it I wouldn't have even looked into this!
And chrrox provided enough of a framework with his Noesis script that I could figure out what I have so quickly.
I can not thank those two enough.

I do a lot of game development so I think that helped me a lot since I work with these structures on pretty much a daily basis, I've never tried to unpack any file formats before (actually I recall that I datamined Fury when it came out but the files were essentially plain text) so this was actually really really fun (and oddly relaxing).

I don't think I have enough time to get a working script with Noesis today though as I need to go out for the rest of the night shortly. Tomorrow should be plenty of time to learn enough Python (for a programmer I often wonder why I know almost nothing about Python) and enough of the Noesis API to get something working.
Yeah I see your point. Anyway I was aiming on .PAA files.
For now, it's not possible to unpack this to work.
Szkaradek123
mega-veteran
mega-veteran
Posts: 292
Joined: Wed May 05, 2010 8:21 pm
Location: Poland Głogów
Has thanked: 21 times
Been thanked: 742 times

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by Szkaradek123 »

Hi

Here is an animation importer for models from this game.
It requires Blender version 249 and Python 26.
I used chrrox script for Noesis viewtopic.php?f=16&t=10909 and ArthainBaka file format research .

Steps:
1.unpack importer
2.run Blender249.blend
3.in Blender Text Window press alt+p and select:
-pac file - for importing a skinned mesh
-pab file - for importing a skeleton
-paa file - for importing an animation
-pam file - for importing static objects

Updated 2015-01-10:
-added pam importer

Updated 2015-01-06:
- added new animation importer
You do not have the required permissions to view the files attached to this post.
Last edited by Szkaradek123 on Sat Jan 10, 2015 6:28 pm, edited 3 times in total.
raykingnihong
mega-veteran
mega-veteran
Posts: 179
Joined: Sun Apr 06, 2014 8:06 pm
Has thanked: 216 times
Been thanked: 6 times

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by raykingnihong »

Szkaradek123 wrote:Hi

Here is an animation importer for models from this game.
It requires Blender version 249 and Python 26.
I used chrrox script for Noesis viewtopic.php?f=16&t=10909 and ArthainBaka file format research .

Steps:
1.unpack importer
2.run Blender249.blend
3.in Blender Text Window press alt+p and select:
-pac file - for importing a skinned mesh
-pab file - for importing a skeleton
-paa file - for importing an animation
Hi Szkaradek123, my friend, the script runs perfectly, great work
koreago
ultra-n00b
Posts: 1
Joined: Wed Dec 30, 2009 5:41 pm
Has thanked: 1 time

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by koreago »

animation import error? help me.. :(
You do not have the required permissions to view the files attached to this post.
hiwap
ultra-n00b
Posts: 8
Joined: Sun Dec 11, 2011 10:07 pm
Has thanked: 6 times

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by hiwap »

Szkaradek123 wrote:Hi

Here is an animation importer for models from this game.
It requires Blender version 249 and Python 26.
I used chrrox script for Noesis viewtopic.php?f=16&t=10909 and ArthainBaka file format research .

Steps:
1.unpack importer
2.run Blender249.blend
3.in Blender Text Window press alt+p and select:
-pac file - for importing a skinned mesh
-pab file - for importing a skeleton
-paa file - for importing an animation
-pam file - for importing static objects

Updated 2015-01-10:
-added pam importer

Updated 2015-01-06:
- added new animation importer
Thanks for the importer!!
is any way to get the correct bone names?
ghgldk11
ultra-n00b
Posts: 2
Joined: Sun Aug 31, 2014 9:08 am

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by ghgldk11 »

anyone know that where is the eyebrow and eye iris or pupil image files?
recently I ported the Valkyrie model from the Korean Black Desert Client into SFM and Gmod.
but actually I couldn't find the texture files of eyebrows, pupil or iris. so i used the eyebrow image file from the Vindictus.
but you know, it is little bit awkward. I found the other things like body or face tattoo, eye line make-up, preview image of customization but
damn.. there is no eye brow, iris .dds file! I think it should be inside of PAT files.
PAT = Peal abyss Texture, isn't it? lol if you guys can rip those PAT files too. it must be wonderful! and i will appreciate about that!
ssringo
veteran
Posts: 98
Joined: Sat Apr 19, 2014 8:02 am
Has thanked: 46 times
Been thanked: 13 times

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by ssringo »

Would anyone that still has the BDO tools posted in viewtopic.php?f=10&t=10879 but later removed mind pm'ing me a download link? Not being able to extract the files makes it hard to use the tools posted here :D
User avatar
dantesai
beginner
Posts: 20
Joined: Sun Feb 09, 2014 3:42 am
Has thanked: 3 times
Been thanked: 5 times

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by dantesai »

@Szkaradek123 run step 3 alt+P just popup error window"python script error:check console"
:oops:
gfhjkm11
ultra-n00b
Posts: 2
Joined: Sun Jul 29, 2012 12:38 pm

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by gfhjkm11 »

Thx for scripts and Tools .. Work perfect as import Model and import Animation ( 100 % ) in Blender 2.49b
blackdesert_1.png
You do not have the required permissions to view the files attached to this post.
pewposterous
advanced
Posts: 41
Joined: Thu Jul 30, 2015 5:05 am
Has thanked: 13 times
Been thanked: 3 times

Re: Black Desert Online File Formats (PAB, PAC, PAA)

Post by pewposterous »

Do the tools used for extracting .paz exist anymore? if so, could someone please pm me?!
Post Reply