XeNTaX

Posted: **Tue Aug 28, 2012 3:45 pm**

hehe, cool - will change the link in the topic starter.

Posted: **Tue Aug 28, 2012 5:27 pm**

Unity3D Engine *.assets extractor

Couldn't find an extractor anywhere, so I wrote my own.

Long overdue but wasn't motivated enough to go through the pain of the format.

If you experience any problems, pm me and I'll take a look. The outsorted file types are only the ones I found.
Else, if anyone can identify some of the missing extensions, I'd be grateful!

lateste updates:
- if type 49 contains png or xml, it is recognized and the size adjusted
- included variant with file tale at the end (some resources.assets use those)
- mp3 detection included

Code: Select all

# extract Unity3D engine *.assets files
# (c) 2012-10-15 by AlphaTwentyThree of XeNTaX

get EXT extension
if EXT == "resS" # no TOC -> scan
	cleanexit
endif
endian big
get FSIZE asize

goto 0xc
get BIAS long
if BIAS != 0 # var1
	endian little
	goto 0x28
	get FILES long
else # var2
	goto 0
	get INFOSIZE long
	get INFO long # actually file size
	math INFO -= INFOSIZE
	math INFO += 0x15
	goto INFO
	endian little
	get FILES long
endif

for i = 1 <= FILES
	set SKIP 0
	set EXT ""
	set FNAME ""
	set WNAME ""
	get FID long
	get OFFSET long
	math OFFSET += BIAS # zero in var2
	get SIZE long
	get TYPE long
	get DUMMY long # same as TYPE (?)
	savepos MYOFF
	callfunction distTYPE 1
	if SKIP == 0
		if EXT = ""
			set EXT "."
			string EXT += TYPE
		endif
		get FOLDERNAME basename
		string FOLDERNAME += "/"
		if FNAME == ""
			get FNAME basename
			string FNAME += "_"
			string FNAME += FID
		endif
		string FNAME += EXT
		set WNAME FOLDERNAME
		string WNAME += FNAME
		if FID == FILES # probably not needed, just a precaution
			get SIZE asize
			math SIZE -= OFFSET
		endif
		log WNAME OFFSET SIZE
	endif
	goto MYOFF
next i

startfunction distTYPE
	if TYPE == 1
		elif TYPE == 2
		elif TYPE == 3
		elif TYPE == 4
		elif TYPE == 5
		elif TYPE == 6
		elif TYPE == 7
		elif TYPE == 8
		elif TYPE == 9
		elif TYPE == 10
		elif TYPE == 11
		elif TYPE == 12
		elif TYPE == 13
		elif TYPE == 14
		elif TYPE == 15
		elif TYPE == 16
		elif TYPE == 17
		elif TYPE == 18
		elif TYPE == 19
		elif TYPE == 20
		elif TYPE == 21
			callfunction getname 1
			set EXT ".mat"
		elif TYPE == 23
		elif TYPE == 26
		elif TYPE == 28
			callfunction getname 1
			set EXT ".tex"
		elif TYPE == 33
		elif TYPE == 43
			callfunction getname 1
		elif TYPE == 48
			callfunction getname 1
			set EXT ".shader"
			goto OFFSET
			get SIZE long
			math OFFSET += 4
		elif TYPE == 49 # ingame movies/xml/backgrounds
			callfunction getname 1
			callfunction getType 1
		elif TYPE == 54
		elif TYPE == 64
		elif TYPE == 65
		elif TYPE == 74
			callfunction getname 1
			set EXT ".ani"
		elif TYPE == 82
		elif TYPE == 83 # can be empty (some pointer maybe)
			callfunction getname 1
			set TEST SIZE
			math TEST -= 0x18
			if TEST == 0
				set SKIP 1
			else
				math OFFSET += 0x10
				goto OFFSET
				get SIZE long
				savepos OFFSET
				getDstring TYPE2 3
				if TYPE2 == "RIF"
					set EXT ".wav"
				elif TYPE2 == "Ogg"
					set EXT ".ogg"
				elif TYPE2 == "ID3"
					set EXT ".mp3"
				else
					goto OFFSET
					get TYPE2 byte
					if TYPE2 == 0xff
						set EXT ".mp3"
					endif
				endif
			endif
		elif TYPE == 89
			callfunction getname 1
		elif TYPE == 96
		elif TYPE == 108
		elif TYPE == 111
		elif TYPE == 115
			callfunction getname 1
		elif TYPE == 123
		elif TYPE == 128
			callfunction getname 1
			set EXT ".ttf"
		elif TYPE == 131
		elif TYPE == 135
		elif TYPE == 150
		elif TYPE == 152
			callfunction getname 1
			math OFFSET += 0x10
			math SIZE -= 0x10
			set EXT ".ogm"
		elif TYPE == 159
			callfunction getname 1
		elif TYPE == 198
		elif TYPE == 199
		elif TYPE == 3584 # last file
		else
	endif
endfunction
	
startfunction getname
	goto OFFSET
	get NAMEL long
	set TEST NAMEL
	math TEST %= 4
	if TEST != 0 # round to next 4byte
		math NAMEL /= 4
		math NAMEL += 1
		math NAMEL *= 4
	endif
	getDstring FNAME NAMEL
	savepos NBIAS
	math NBIAS -= OFFSET
	math OFFSET += NBIAS
	math SIZE -= NBIAS
endfunction

startfunction getType
	goto OFFSET
	get DUMMY long
	get TYPE long
	if TYPE == 0x6d783f3c
		math OFFSET += 4
		set SIZE DUMMY
		set EXT ".xml"
	elif TYPE == 0x474e5089
		math OFFSET += 4
		set SIZE DUMMY
		set EXT ".png"
	endif
endfunction

Posted: **Tue Aug 28, 2012 6:34 pm**

XOR test

Just writes the first 2KB of a given file with 1 < XOR < 256 to determine the needed setting for a future parser more easily. You can change the setting if you want to.
Exremely simple script practically not really worth posting - but for those who are totally lost, get it here.

Code: Select all

for i = 1 <= 256
	filexor i
	get NAME basename
	string NAME += "_xor"
	string NAME += i
	log NAME 0 0x1000
next i

Posted: **Thu Aug 30, 2012 12:49 pm**

Dark Souls bdt/bhd5 extractor

This is the same type of archive than the one used in Armored Core V, just little endian. Somehow I got an error with the old script minus the "endian big", so I wrote a new one which works.

Be sure to grab the newest func_getTYPE.bms (viewtopic.php?f=13&p=69577#p69577) - I've added new formats and corrected one stupid error which prevented a bunch of formats to be recognized.

Code: Select all

# extracts the contents of bdt/bhd5 archive pairs
# tested on: Dark Souls: Prepare to Die Edition (PC, X360)
# NOTE: make sure both bdt and bht5 files have the same name before running the script!
#
# (c) 2012-10-31 by AlphaTwentyThree of XeNTaX

set X360 1
include "func_getTYPE.bms"
open FDDE bhd5 0
open FDDE bdt 1

if X360 == 1
	endian big
endif

idstring "BHD5" 0
get UNK long 0 # 00000000 (?)
get FOLDERS long 0
get NOF long 0 # number of files (not needed for loop)
get GROUPS long 0
get UNK long 0
for i = 1 <= GROUPS
	get FIG long 0 # files in group/folder
	get OFF_INFO long 0
	savepos MYOFF 0
	goto OFF_INFO 0
	for j = 1 <= FIG
		get NAME_CRC long 0
		get SIZE long 0
		get OFFSET long 0
		get ZERO long 0
		if X360 == 1
			set OFFSET ZERO
		endif
		get FOLDER basename 0
		string FOLDER += "/"
		set NAME i
		string NAME += "_"
		string NAME += j
		string NAME += "_"
		string NAME_CRC p= "0x%08x" NAME_CRC
		string NAME += NAME_CRC
		putVarChr MEMORY_FILE SIZE 0
		log MEMORY_FILE 0 0
		log MEMORY_FILE OFFSET SIZE 1
		callfunction getTYPE 1
		if X360 == 1
			endian big
		endif
		string NAME += EXT
		set WNAME FOLDER
		string WNAME += NAME
		log WNAME 0 SIZE MEMORY_FILE
	next j
	goto MYOFF 0
next i

Posted: **Mon Sep 03, 2012 7:17 am**

THANKS

Posted: **Mon Sep 03, 2012 7:59 pm**

file deinterleaver (new version)

Split files into alternating blocks of the same size. Used when manipulating audio formats with more than one channel. E.g. stereo wave streams have an interleave of 2 bytes, i.e. each two bytes the channel information switches 2xL,2xR,2xL and so on. The blocksize of this structure would be 4 (two times the interleave size). If the stream has four channels, the interleave would stay at 2, but the blocksize would be 8. So if you split the channels you need to deinterleave the file and this script does exactly that, according to the parameters you set.
Additionally you can specify a SKIP variable, which denotes an area at the start of each block not to process during the deinterleaving. A good example is the "disguised" Playstation ADPCM format in some Electronic Arts PS2 games, where some dummy marker is set each block. So, just set the layer count to one and adjust the SKIP variable.

This is an extremely powerful script! Play around with different files to get the idea what you can do with this. Most common use is channel splitting, but you can do all kinds of adjustments to your files. Easiest example would be to split audio formats into mono channels, splitting multichannel files into stereo pairs or deleting regularly occurring chunks of data inside a file.

This is a long overdue update of my old deinterleave script, which was rather cryptic and above all really really slow. This new one boosts the running time x300 and has some additional options:
- you can choose to add the original header to the deinterleaved files
- the header can be manipulated
- the variables are now better to understand (just header, blocksize, interleave size, layer and skip)

UPDATE 2016-05-16

The script now supports incomplete last blocks!

Here's a little example

Source file:
- header of 0x10 bytes
- four channels
- 4 byte dummy each 0x14 bytes
- you want to split the channels and know that one complete block is 0x14 bytes

Variables to set:
- HEADER 0x10
- SKIP 4
- LAYERS 4
- BLOCKSIZE 0x14 (alternatively BLOCKSIZE 0 and INTSIZE 4)
- to add the original header to each deinterleaved file, set PRESERVE to 1
- leave ADJUST at 0 if you're not familiar with QuickBMS

Let's say we have the same file, but the dummy occur for every channel interleave block. Then we have to run the script two times:
- first run, delete the dummy bytes: HEADER 0x10, PRESERVE 1, SKIP 4, LAYERS 1, BLOCKSIZE 0, INTSIZE 0x14 (or BLOCKSIZE 0x50) -> produces one smaller file
- second run, deinterleave channels: HEADER 0x10, PRESERVE 1, SKIP 0, LAYERS 4, BLOCKSIZE 0x40 (or 0 and INTSIZE 0x10) -> produces four files which are exactly 1/4th of the above file

Code: Select all

# deinterleave file by different parameters
# (c) 2016-05-16 by AlphaTwentyThree of XeNTaX

# notes:
# HEADER - area which is untouched by the deinterleaving
# PRESERVE 0/1			-	set to 1 to write the skipped area from 
#            				HEADER to each deinterleaved file
# ADJUST 0/1 			- 	if the header needs to be adjusted, set 
#              				this to 1 and write your code to the function 
#              				"adjustheader" at the end of this script
# SPLIT_LAST_BLOCK 0/1	-	set to 1 to split an incomplete last block equally 
#							between layers. If set to 0, the script will ter-
#							minate if it finds an incomplete last block.
#							HANDLE WITH CARE!
# BLOCKSIZE 			-	size of one complete interleave block 
# LAYERS				-	the numbers of (equal-sized) files you want 
#							to deinterleave to
# SKIP					-	number of bytes to skip at the start of each block
#
# examples:
# split channels of stereo wave file: # HEADER 0x2c, PRESERVE 1, ADJUST 1 (adjust function), BLOCKSIZE 4, LAYERS 2, SKIP 0
# leave out 2 bytes each 0x10 bytes: # BLOCKSIZE 0x10, LAYERS 1, SKIP 2
# split channels of Playstation ADPCM file: # HEADER 0x2c, PRESERVE 1, ADJUST 1 (adjust function), BLOCKSIZE 2*Interleave, SKIP 0

set HEADER 0xe0 # zero if skip is at end of block, otherwise = first skip at start
set PRESERVE 0
set ADJUST 0
set SPLIT_LAST_BLOCK 1
set BLOCKSIZE 0 # size of one complete block
set INTSIZE 0x5800 # if BLOCKSIZE is set to 0, the interleave size is taken to calculate the blocksize
set LAYERS 2
set SKIP_START 0 # area at start of each block to skip
set SKIP_END 0 # area at end of each block to skip

if BLOCKSIZE == 0
	xmath BLOCKSIZE "(INTSIZE * LAYERS) + SKIP_START + SKIP_END"
endif
if INTSIZE == 0
	xmath INTSIZE "BLOCKSIZE / LAYERS"
endif

xmath WSIZE "(BLOCKSIZE - SKIP_START - SKIP_END) / LAYERS" # data to write: (BLOCKSIZE - SKIP_START - SKIP_END)/LAYERS
get CYCLES asize
xmath CYCLES "(CYCLES - HEADER) / BLOCKSIZE"

get LASTBLOCK asize
xmath LASTBLOCK "(LASTBLOCK - HEADER) % BLOCKSIZE"
xmath LASTINT "LASTBLOCK / LAYERS"

callfunction testparameters 1
get EXT extension
callfunction deinterleave

startfunction testparameters
	# test 1: (BLOCKSIZE-SKIP_START-SKIP_END)/LAYERS must be integer (one block must be dividable into skip and layers)
	xmath TEST "BLOCKSIZE - SKIP_START - SKIP_END"
	math TEST %= LAYERS # single layer -> always ok
	if TEST != 0
		print "Error: blocksize minus skip isn't dividable by layer count! Aborting..."
		cleanexit
	endif
	
	# test 2: test for incomplete last block
	get TEST asize
	math TEST -= HEADER
	math TEST %= BLOCKSIZE
	if TEST != 0
		if SPLIT_LAST_BLOCK == 1
			print "Warning: file size minus header isn't dividable by blocksize! Last block will be equally split between layers!"
			print "Last chunks will have size of %LASTINT% bytes each."
		else
			print "ERROR: file size minus header isn't dividable by blocksize! Aborting..."
			cleanexit
	endif
	
	# test 3: ASIZE-HEADER-BLOCKSIZE must be greater null (at least one deinterleave cycle)
	get TEST asize
	xmath TEST "TEST - HEADER - BLOCKSIZE"
	if FSIZE <= 0
		print "Error: file too small to deinterleave! Aborting..."
		cleanexit
	endif
endfunction
	
startfunction deinterleave
	xmath PSIZE "CYCLES * WSIZE + LASTINT"
	if PRESERVE == 1
		math PSIZE += HEADER
		goto 0
		getDstring HDATA HEADER
	endif
	
	
	for i = 1 <= LAYERS
		putVarChr MEMORY_FILE PSIZE 0
		log MEMORY_FILE 0 0
		get NAME basename
		string NAME += "_"
		string NAME += i
		if PRESERVE == 1
			putDstring HDATA HEADER MEMORY_FILE
			string NAME += "."
			string NAME += EXT
		endif
		xmath BIAS_A "(i - 1) * WSIZE + SKIP_START"# bias at start
		xmath BIAS_B "BLOCKSIZE - BIAS_A - WSIZE" # bias at end
	
		xmath RES_BIAS_A "(i - 1) * LASTINT + SKIP_START"
		xmath RES_BIAS_B "LASTBLOCK - RES_BIAS_A - LASTINT"
		goto HEADER
		for k = 1 <= CYCLES
			getDstring DUMMY BIAS_A
			getDstring WDATA WSIZE
			putDstring WDATA WSIZE MEMORY_FILE
			getDstring DUMMY BIAS_B
		next k
		if LASTBLOCK != 0
			getDstring DUMMY RES_BIAS_A
			getDstring WDATA LASTINT
			putDstring WDATA LASTINT MEMORY_FILE
			getDstring DUMMY RES_BIAS_B
		endif
		get SIZE asize MEMORY_FILE
		if ADJUST == 1
			callfunction adjustheader 1
		endif
		log NAME 0 SIZE MEMORY_FILE
	next i
endfunction

startfunction adjustheader
	putVarChr MEMORY_FILE 0x10 1 long
	get SSIZE asize MEMORY_FILE
	math SSIZE -= HEADER
	putVarChr MEMORY_FILE 0x24 SSIZE long
endfunction

Posted: **Mon Sep 03, 2012 11:32 pm**

AlphaTwentyThree wrote:file deinterleaver (new version)

Also long overdue update of my deinterleave script, which was rather cryptic and above all really really slow. This new one boosts the running time x300 and has some additional options:
- you can choose to add the original header to the deinterleaved files
- the header can be manipulated
- the variables are now better to understand (just header, blocksize, layer and skip)
This is an extremely powerful script! Play around with different files to get the idea what you can do with this. Most common use is channel splitting, but you can do all kinds of adjustments to your files. Easiest example would be to split audio formats into mono channels, e.g. Playstation ADPCM files.

Perhaps link to your old one, for learning/comparison/case study purposes (assuming there's anyone interested in such a thing)?

Posted: **Tue Sep 04, 2012 12:55 am**

Done so in the text. Quite embarassing looking at my coding back then.

So many improvements now! =)

Posted: **Tue Sep 04, 2012 1:33 am**

If anyone is interested, I've added a link to my topic with tips how to speed up your scripts. It can be found here: viewtopic.php?f=13&t=9594

Hope that helps some people.

Posted: **Tue Sep 04, 2012 1:21 pm**

Just corrected the at3_to_aa3 script. Seemed to have some serious mistake... Also added pre-allocation although you'll probably only notice the difference when converting large amounts of at3 files.

Posted: **Tue Sep 04, 2012 5:53 pm**

Alpha pertaining to your updated deinterleaving script would there be a possible option to add a scanning function to determine if the file's interleave possibly changes to another interleave cycle and the option can also print out the offset where we would need to cut the file segment with the new interleave?

Posted: **Wed Sep 05, 2012 1:23 am**

OrangeC wrote:Alpha pertaining to your updated deinterleaving script would there be a possible option to add a scanning function to determine if the file's interleave possibly changes to another interleave cycle and the option can also print out the offset where we would need to cut the file segment with the new interleave?

How should that work? There's absolutely NO way to determine such an extremely generic thing automatically.

Posted: **Wed Sep 05, 2012 7:13 pm**

Your right.

Oh well maybe it can Print out the difference in size if its not divisible by the given blocksize/interleave?

Posted: **Wed Sep 05, 2012 8:20 pm**

OrangeC wrote:Your right.

Oh well maybe it can Print out the difference in size if its not divisible by the given blocksize/interleave?

Sure, just add %TEST% somwhere inside the two error print commands.

Posted: **Fri Sep 14, 2012 2:03 pm**

XIII Century *.pak extractor

Easy format that can be used to get aquainted to QuickBMS. =)

Code: Select all

# extract *.pak files from XIII Century (2008, PC)
# (c) 2012-09-14 by AlphaTwentyThree of XeNTaX

idstring "PACK"
get ZERO long
get INFO long
get ZERO long
get FILES long
goto INFO
for i = 1 <= FILES
	get OFFSET long
	get ZERO long
	get SIZE long
	get ZERO long
	get NAMEL long
	math NAMEL -= 1
	get UNK long # always 2?
	get UNK byte # always 0x2e?
	getDstring NAME NAMEL
	log NAME OFFSET SIZE
next i

XeNTaX

My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts

Re: My quickBMS scripts