Files extractors scripting

[aluigi]

the files are not compressed indeed it's enough the following script for extracting them:

Code: Select all

idstring "QPACK\0"
goto 10
get FILES long
goto 0x15
for i = 0 < FILES
    getdstring NAME 0x200
    get SIZE long
    get OFFSET long
    log NAME OFFSET SIZE
next i

the problem is that the DDS files have the first 508 bytes (skipping the first 2) encrypted with a non "visibly recognizable" algorithm

[szevvy]

Wombat (available at http://www.szevvy.com) lets you do some basic operations on data through the script:

Code: Select all

file name = "*1.gfx" is (
	folder: "Background Tiles"
	[16] (
		encryptedData SecretAgentXOR[8064] (

			data(3) unknown

			image EGA(
				width: 16
				height: 16
				dataSize: 8061
				tilesAcross: 4
			)
		)
	)
)

XORData: values (
	0x43, 0x6F, 0x70, 0x79, 0x72, 0x69, 0x67, 0x68, 0x74, 0x20, 0x31, 0x39, 0x39, 0x31,
        0x20, 0x50, 0x65, 0x64, 0x65, 0x72, 0x20, 0x4A, 0x75, 0x6E, 0x67, 0x63, 0x6B, 0x00)

script SecretAgentXOR (
	counter: 0

	[all] (

        read unsigned8 encryptedByte        

        #reverse the byte
        set encryptedByte: ((encryptedByte & 0x0F) << 4) | ((encryptedByte & 0xF0) >> 4)
        set encryptedByte: ((encryptedByte & 0x33) << 2) | ((encryptedByte & 0xCC) >> 2)
        set encryptedByte: ((encryptedByte & 0x55) << 1) | ((encryptedByte & 0xAA) >> 1)

        write unsigned8: encryptedByte ^ XORData[counter % 28]

        set counter: counter + 1
	)
)

This is for Secret Agent - decrypts and draws the tiles. It also lets you do packed file extraction and so forth:

Code: Select all

file name = "*.cmp" is (
	files [size > 0] (
		FixedString(12) name
		unsigned32 offset
		unsigned32 size
	)
)

will list all files in a Duke Nukem 2 archive.

[Rheini]

the problem is that the DDS files have the first 508 bytes (skipping the first 2) encrypted with a non "visibly recognizable" algorithm

Still wondering how that visual recognization works.

[aluigi]

yeah I don't know to what I was thinking in that moment :)
I guess I referred to the usual xor and rot13 checks

[Rheini]

Yeah but I can't imagine how to find out an encryption just by looking at the scrambled code

[aluigi]

usually the archive files have often 00 bytes (or a most used byte) inside them so if you see a sequence of b8 b8 b8 in various zones of the examined block you can guess it's a xor or rot13 which involves the byte 0xb8 (or -0x48 in the case of rot13 decryption).

if you see a pattern of 8 bytes which is used in at least 2 zones it can be an ECB algorithm like blowfish where the same sequence of bytes (which fits the block size of the algorithm, like 8 for blowfish) gives ever the same result.
obviously you can't retrieve the key but at least you can have an idea of the encryption used on that file.

while if it's used an ivec (a sequence of bytes used to xor the resulted block and updated at each cycle) there is nothing visually recognizable.

so, yes, it's possible to figure an algorithm (completely or partially) only using the eyes and a dose of luck :)

[Rheini]

while if it's used an ivec (a sequence of bytes used to xor the resulted block and updated at each cycle) there is nothing visually recognizable.

I like those ones
viewtopic.php?p=21276#p21276

[chrrox]

I can not figure out what I am doing wrong. The game is Dragon Nest Online.
I have extracted a single file no problem when I start before the first file and have it just extract that file but I can not figure out how to continue the pattern.
I set the files to 10 just for testing it should just go to the end of the file.
here is what I have.

Code: Select all

idstring "Nfs\0"
#get FILES long
math FILES += 11
goto 0xC
for i = 0 < FILES
get ZIP long
getdstring NAME 0x100
get ZSIZE long
get SIZE long
getdstring NULL 0x14
savepos OFFSET
if ZIP == 1
clog NAME OFFSET ZSIZE SIZE
math OFF += ZSIZE
goto OFF
next i
else
log NAME OFFSET SIZE
goto OFFSET
next i

here is the header and footer of the archive.
http://www.MegaShare.com/936627
Thanks in advance for any help

[aluigi]

the format of the archive is enough clear, it's practically like the "binary" version of a recursing directory scanning (including "." and "..").
the problem is that it's a job for a recursive function because the bms scripts can't support it.

... anyway the attached script although looks horrible seems to do the job correctly :)

*edit* updated script in the next post

[chrrox]

You sir are amazing.
It goes very far in the script but then I get this error.

Code: Select all

Cerberos\Darkelf\Evil Roots\Gargoyle\Ghoul\Goblin\Goblin.ani
  1b5e64a7 57368      Mapdata\Grid\Z_Test\Resource\Sound\Prop\Tile\Water\Trigger\Action\Condition\Ev
ent\Resource\Camera\Char\Monster\Basilisk\basilisk_shell\Bat\Beholder\Beholder_eye\BlackDragon\Broo\
Cerberos\Darkelf\Evil Roots\Gargoyle\Ghoul\Goblin\Goblin.msh
  1b5eadc9 174904     Mapdata\Grid\Z_Test\Resource\Sound\Prop\Tile\Water\Trigger\Action\Condition\Ev
ent\Resource\Camera\Char\Monster\Basilisk\basilisk_shell\Bat\Beholder\Beholder_eye\BlackDragon\Broo\
Cerberos\Darkelf\Evil Roots\Gargoyle\Ghoul\Goblin\Goblin01.dds
  1b609c8f 174904     Mapdata\Grid\Z_Test\Resource\Sound\Prop\Tile\Water\Trigger\Action\Condition\Ev
ent\Resource\Camera\Char\Monster\Basilisk\basilisk_shell\Bat\Beholder\Beholder_eye\BlackDragon\Broo\
Cerberos\Darkelf\Evil Roots\Gargoyle\Ghoul\Goblin\Goblin_Black.dds

- error in src\quickbms.c line 3053: dumpa()
Error: No such file or directory

I think this company liked folders a bit to much lol.
or is that error from name too long?
here is a section from where is stopped.
http://www.MegaShare.com/936988
if you need a different section let me know and I can upload it did the program get stuck in the Z_test folder?

I put the extraction destination to a folder called 1 on the root of my drive and it got this far.

Code: Select all

ent\Resource\Camera\Char\Monster\Basilisk\basilisk_shell\Bat\Beholder\Beholder_eye\BlackDragon\Broo\
Cerberos\Darkelf\Evil Roots\Gargoyle\Ghoul\Goblin\GoldDragon\DG_Gold.msh
  1b8f6b8e 12484      Mapdata\Grid\Z_Test\Resource\Sound\Prop\Tile\Water\Trigger\Action\Condition\Ev
ent\Resource\Camera\Char\Monster\Basilisk\basilisk_shell\Bat\Beholder\Beholder_eye\BlackDragon\Broo\
Cerberos\Darkelf\Evil Roots\Gargoyle\Ghoul\Goblin\GoldDragon\DG_Gold.skn
  1b8f6d9d 2796368    Mapdata\Grid\Z_Test\Resource\Sound\Prop\Tile\Water\Trigger\Action\Condition\Ev
ent\Resource\Camera\Char\Monster\Basilisk\basilisk_shell\Bat\Beholder\Beholder_eye\BlackDragon\Broo\
Cerberos\Darkelf\Evil Roots\Gargoyle\Ghoul\Goblin\GoldDragon\gold.dds
  1ba17fae 92881      Mapdata\Grid\Z_Test\Resource\Sound\Prop\Tile\Water\Trigger\Action\Condition\Ev
ent\Resource\Camera\Char\Monster\Basilisk\basilisk_shell\Bat\Beholder\Beholder_eye\BlackDragon\Broo\
Cerberos\Darkelf\Evil Roots\Gargoyle\Ghoul\Goblin\GoldDragon\Golem\Golem.act

- error in src\quickbms.c line 3053: dumpa()
Error: No such file or directory

Edit2 Here is the section where I think it is different it is attached

[aluigi]

I guess I have understood what's the problem (probably), I used max 8 subfolders in the script while seems that the game use more subfolders so now I have set them to 20.
I don't know if this solves the problem but I guess that any file in the archive should be extracted correctly.
so test the attached new version

[chrrox]

The script did go further this time
I think I see a pattern I believe when get NEXT_FOLDER long is == 0000 it goes backwards 2 directories
here is one part of the folder
F:\2\Mapdata\Grid\Z_Test\Resource\Sound\Prop\Tile\Water\Trigger\Event\Resource\Char\Monster\Wraith\NPC\Npc_Woman_Trader_May\Player\Archer\Parts\Cleric\Parts\
I think it should be
-----Mapdata
---Grid
--Z_Test
-----Resource
---Sound
--Prop
---Tile
--Water
---Trigger
--Event
-----Resource
---Char
--Monster
-Wraith
--NPC
-Npc_Woman_Trader_May
--Player
-Archer
Parts
-Cleric
Parts

There are other folders besides these in each directory and based on those folders and looking at some of the pattern I think this may be the case.
I have attached a section that should contain a few of these double back directories. I can upload the whole archive if need be just let me know.
Thanks again for all of your work
http://www.MegaShare.com/940686

[chrrox]

Here is a file extractor for the game cars online.
I am not used to working with separate header files so I
pasted the header file on top of the main file to write this script.

Code: Select all

get NULL long
math FILES += 0x28bc
for i = 0 < FILES
getdstring HASH 0x8
get OFFSET long
get SIZE long
math OFFSET += 0x28BD4
log NAME OFFSET SIZE
math NAME += 1
next i

This does not extract the file names as I can not figure out how they are stored. It appears they are stored every so many files in big edian format.
I will attach the header file and an index file if anyone is interested. the main file is called client.dat
website http://cars.wasabii.com.tw
client link http://dlcars.wasabii.com.tw/Cars_1.0.322.exe
sample file
http://upload.megashare.com/upload_succ ... 6388689087

[fatduck]

Hi Bugtest,

A little problem here, I have a archive with some data in it. But there is no index-table. So what I have to do is to export the resources in between Headers!

I use "findloc" to search for each header and it works fine!
The only problem is the last resource! Because I can find the header anymore!

According to BMS wiki(I know its a bit different).
If using "findloc" to search for a sting and can't find any. It will return 0
But QuickBMS just exit without any error!

I attached a example to make myself clear, sorry for my bad English!

In Fatduck.dat, there should have 3 resource! The header is "FatduckFile"
1 @ 0x0 length: 296
2 @ 0x128 length: 519
3 @ 0x32F length: 209 (failed to export this one)

The test.bms can only export the first two resource only!
No error, clean exit!

[aluigi]

I understand, practically you use the location of the next "FatduckFile" to calculate the size of the previous one... uhmmm yeah without knowing if there is another FatduckFile is available you can do nothing.

anyway the action in case findloc doesn't find the searched string is the termination, this has been confirmed in this thread.

basicly now we have "FindLoc <var> <datatype> <text/number> [filenumber]" so it's needed to specify somewhere something which says to the script engine "hey don't terminate if you don't find this string".
adding another option parameter after filenumber sux so it must be excluded, the other arguments can't be touched and by default findloc must exit.
now get these parameters and if you solve this rubik cube I implement it in quickbms on the fly :)