Deus EX: Mankind Divided .archive

[Vindis]

your command line is fine. But it would seem it's not finding any valid *.vap files in the 'input' folder. And just in case you think it will parse down subdirectories inside the 'input' folder, be assured it will not. It will only process *.vap files in 'input' folder itself. If your *.vap files are in input\someother directory, then you need to specify that directly.

Hi Volfin,

Tried to convert all txet (assuming those are the texture files) but I got an error, "unknown format -1" for each one. I renamed every file to vap.
I extracted the files with Sir Kanes' extractor, could be that problem? Should I try quickbms?

Thank you for your help!

[volfin]

your command line is fine. But it would seem it's not finding any valid *.vap files in the 'input' folder. And just in case you think it will parse down subdirectories inside the 'input' folder, be assured it will not. It will only process *.vap files in 'input' folder itself. If your *.vap files are in input\someother directory, then you need to specify that directly.

Hi Volfin,

Tried to convert all txet (assuming those are the texture files) but I got an error, "unknown format -1" for each one. I renamed every file to vap.
I extracted the files with Sir Kanes' extractor, could be that problem? Should I try quickbms?

Thank you for your help!

yes everything was written for the quickbms files. I don't know the differences involved with the other extractor.

[Vindis]

your command line is fine. But it would seem it's not finding any valid *.vap files in the 'input' folder. And just in case you think it will parse down subdirectories inside the 'input' folder, be assured it will not. It will only process *.vap files in 'input' folder itself. If your *.vap files are in input\someother directory, then you need to specify that directly.

Hi Volfin,

Tried to convert all txet (assuming those are the texture files) but I got an error, "unknown format -1" for each one. I renamed every file to vap.
I extracted the files with Sir Kanes' extractor, could be that problem? Should I try quickbms?

Thank you for your help!

yes everything was written for the quickbms files. I don't know the differences involved with the other extractor.

Is there any chance, that you share the file structure of the texture files, so I can write a program to work with Sir Kanes' extractor? Maybe there's not much difference...

[volfin]

Is there any chance, that you share the file structure of the texture files, so I can write a program to work with Sir Kanes' extractor? Maybe there's not much difference...

The header is as follows:

Code: Select all

struct HeaderVAP
{
	long Unknown0;
	short Unknown1;
	short Unknown2;
	short Unknown7;
	short Unknown8;
	short Width;
	short Height;
	short Format1; // MIP count maybe?
	short Format;
	long Unknown3;
	long Unknown4;
	long Unknown5;
	long Unknown6;
};

Directly after the header comes the compressed texture data. How that data is formatted is dependent on the "Format" variable. The known "Format" codes are:

Code: Select all

		TEXTURE AttemptedFormat = TEXTURE::NO_FORMAT;
		if (VAPHeader->Format == 0x4A)
			AttemptedFormat = TEXTURE::DXT1;
		else if (VAPHeader->Format == 0x50)
			AttemptedFormat = TEXTURE::DXT5;
		else if (VAPHeader->Format == 0x53)
			AttemptedFormat = TEXTURE::BC4;
		else if (VAPHeader->Format == 0x56)
			AttemptedFormat = TEXTURE::BC5;
		else if (VAPHeader->Format == 0x5B)
			AttemptedFormat = TEXTURE::BC7;
		else if (VAPHeader->Format == 0x43)
			AttemptedFormat = TEXTURE::U8bpp;
		else if (VAPHeader->Format == 0x1D)
			AttemptedFormat = TEXTURE::U32bpp;

		if (AttemptedFormat == TEXTURE::NO_FORMAT)
		{
			printf("Unknown Format %d\n", VAPHeader->Format);
			continue;
		}

[erik945]

Thank you for your tool
Do you planed update texture convertor?


And what does it mean when in a namemap in a line with a numeric file name there are several symbolic names?
as example:

71795574635566022 00FF11B1DA9913C6 [assembly:/_g2_assets/templates/aspectdummy.aspect]([assembly:/environment_assets/20_props/10_cluttering/17_industrial/bucket_laundry_garbage_a/bucket_laundry_garbage_a.template?/bucket_laundry_garbage_a bf7f7ac8-1407-421c-bb9a-faa85ce5fcdb.entitytemplate].entitytype,[modules:/runtime.streaming.moduleinfo?zstreamingroot.class].entitytype,[modules:/runtime.activation.moduleinfo?zactivationroot.class].entitytype).pc_entitytype

6031250991359965 00156D640CE427DD [[assembly:/material_class/2dnsr_pb.uberclass?/blendwithvertexcolor#!diffusemap1##tint,df~0,u~wrap,v~wrap,f~21,uv~uv0!!diffusemap2#bo~maskblend,bc~r,#tint,col!!usenormalmapfiltering#!nmf~1,normalmap1##tint,df~2,u~wrap,v~wrap,f~21,uv~uv0!!normalmap2#bo~add,#tint,df~3,u~wrap,v~wrap,f~21,uv~uv0!!specular1##tint,col!roughness#tint,df~5,u~wrap,v~wrap,f~21,uv~uv0!!specular2#bc~r,#tint,col!roughness#tint,df~7,u~wrap,v~wrap,f~21,uv~uv0,ch~g!!masktypesettings#!masktexture##tint,col!!maskchannelr#channelmap#tint,uf~7!!alphatypesettings#!blendwithvertexalpha#!vertexshadertype#!materialtype1#mt~dielectric!dbm~opaque.ubervariant].fx](dx11).pc_mate

[volfin]

as far as I know, there's no need to update the texture tool. it works with all valid textures. If that's not true, I need more information.

As for the number stuff, I have no idea what all that is.

[erik945]

Convertor don't work with textures extracted by Sir Kanes tool.

[volfin]

Convertor don't work with textures extracted by Sir Kanes tool.

yes everything was made for the original quickbms script. for that it all works.

[erik945]

It seems to me Sir Kane's extractor misses some files.
I can not find some models - for example, weapons and augmentations from the screen of equipment.

[Sir Kane]

https://sktest.aruarose.com/DXMDExtract_v002.7z

• Fixed a bug that caused files in containers whose names contain upper case characters to not be extracted.
• Speculative fix for files containing wrong data.
• Now generates a list of external resources and their file names.
• Now generates a binary file containing the data required to map resource IDs to asset names.

This is the format for the ResMap.bin file:

Code: Select all

	uint32_t	Count;
	struct {
		uint64_t	ResourceID;
		uint32_t	NameLength;
		char		Name[NameLength];
	} Entries[Count];

The name is zero terminated.

[Vindis]

I made a texture tool for Sir Kanes extractor (fortunately, the data is already there, uncompressed ).
It have basic functionality, and convert dxt1,5, bc4,5,7 textures. All you have to do is rename texture bin files to ".tex"
Just put the files into the same folder as the exe, and it converts all tex to DDS. (you can figure out what is texture from names list)


PS: Can't found files for these types, so it's not converting these tho, and you need intel's photoshop plugin to work with BCx dds.

Code: Select all

AttemptedFormat = TEXTURE::U8bpp;
      else if (VAPHeader->Format == 0x1D)
         AttemptedFormat = TEXTURE::U32bpp;

      if (AttemptedFormat == TEXTURE::NO_FORMAT)
      {
         printf("Unknown Format %d\n", VAPHeader->Format);
         continue;
      }

[erik945]

Join the topic - a couple of scripts on the python 2.

The first - copies and renames the extracted files. The result is a correct directory tree.
Fdir = "E: \\ DXMD_DLC_UNPACK" # base unpack folder with nameMap file
Fsrc = "E: \\ md_files_DLC2 \\" # folder with correct filenames


The second for the blender - batch export of 3d models to fbx
Folder to search (recursively)
Fdir = "E: \\ md_files_DLC \\ objects \\" - directory for search

[Vindis]

@Volfin

Hi Volfin,

may I ask you a big favor?
If you have time, could you please write down the file format of the model files?
I've tried to figure out, and write a template in 010 Editor, but it drives me nuts. Sometimes it works,
sometimes the values are offset by an unknown value (the tables for example, it points to an address, there's no vertex number).

I don't know nothing about python scripting, so your importer is not a good starting point for me

I would really appreciate your help!

[volfin]

@Volfin

Hi Volfin,

may I ask you a big favor?
If you have time, could you please write down the file format of the model files?
I've tried to figure out, and write a template in 010 Editor, but it drives me nuts. Sometimes it works,
sometimes the values are offset by an unknown value (the tables for example, it points to an address, there's no vertex number).

I don't know nothing about python scripting, so your importer is not a good starting point for me

I would really appreciate your help!

This is because there is no one format. There's 10 to 15 distinct vertex buffer formats. And they can have various sub blocks per format. so it's a very messy thing to decode. I have no real documentation, I figure a block format out in 010, then add it to the python script. then when I or someone else encounters another format, I add it to the script as well. The only real documentation is the python script. It's not hard to read really.

I have a library of data readers in the python script:

Code: Select all

def readcstr(): #Null Terminated text string
 def BB(n): #unsigned Char
 def Bb(n): # Signed Char
 def BH(n): # Unsigned Short
 def Bh(n): # Signed Short
 def Bf(n): # Float
 def BF(n): # HALF FLoat
 def Bi(n): # Signed Long Int
 def BI(n): # Unsigned Long Int
 def Bd(n): # Double
 def alignPosition(alignment): # align to given byte width (4/8/16 etc)

if you see BB(3) in the code, I read in 3 bytes, but didn't assign them to a variable, so I just skipped them
if you see new_var = BB(3), then 'new_var' is now an array of 3 bytes read in. new_var[0] would be the first new_var[1] would be the second etc.
if you see new_var = BB(1)[0], then I immediately assigned the value read in to 'new_var' instead of assigning it as an array.

filehandle.seek(next_offset,0) # SEEK_SET
filehandle.seek(hdr_size,1) #SEEK_CUR
filehandle.seek(0,2) # SEEK_END
These all do exactly what they do in C, seek on the file stream either from the start (SET), the current position (CUR), or from the end (END).

The rest should be self explanatory.

[Vindis]

@Volfin

Hi Volfin,

may I ask you a big favor?
If you have time, could you please write down the file format of the model files?
I've tried to figure out, and write a template in 010 Editor, but it drives me nuts. Sometimes it works,
sometimes the values are offset by an unknown value (the tables for example, it points to an address, there's no vertex number).

I don't know nothing about python scripting, so your importer is not a good starting point for me

I would really appreciate your help!

This is because there is no one format. There's 10 to 15 distinct vertex buffer formats. And they can have various sub blocks per format. so it's a very messy thing to decode. I have no real documentation, I figure a block format out in 010, then add it to the python script. then when I or someone else encounters another format, I add it to the script as well. The only real documentation is the python script. It's not hard to read really.

I have a library of data readers in the python script:

Code: Select all

def readcstr(): #Null Terminated text string
 def BB(n): #unsigned Char
 def Bb(n): # Signed Char
 def BH(n): # Unsigned Short
 def Bh(n): # Signed Short
 def Bf(n): # Float
 def BF(n): # HALF FLoat
 def Bi(n): # Signed Long Int
 def BI(n): # Unsigned Long Int
 def Bd(n): # Double
 def alignPosition(alignment): # align to given byte width (4/8/16 etc)

if you see BB(3) in the code, I read in 3 bytes, but didn't assign them to a variable, so I just skipped them
if you see new_var = BB(3), then 'new_var' is now an array of 3 bytes read in. new_var[0] would be the first new_var[1] would be the second etc.
if you see new_var = BB(1)[0], then I immediately assigned the value read in to 'new_var' instead of assigning it as an array.

filehandle.seek(next_offset,0) # SEEK_SET
filehandle.seek(hdr_size,1) #SEEK_CUR
filehandle.seek(0,2) # SEEK_END
These all do exactly what they do in C, seek on the file stream either from the start (SET), the current position (CUR), or from the end (END).

The rest should be self explanatory.

Oooh!

The binary reading part avoided my attention. Now it's really not that hard

Thank You very much!