[REQ][PC]Sleeping Dogs .big files

[neburas]

It works perfectly. Thanks a lot

Unfortunately extracted files are all encrypted...

[Falo]

I don't believe they are encrypted (at least the XBOX version of the files don't seem to be), but they all have their own binary formats, which need to be examined and reverse-engineered on their own.

In other words, it obviously needs further research before anything meaningful can be changed in the BIG file. I probably won't do any of that research myself, but hopefully the extractor code allows others to get further results.

The problem was never extracting the files, 2 extractors scripts are on the first page and i coded a standalone .NET extractor, there were only 2 problems,
the hash algo and the compression, i didn't continue the work on the tool yet, because the hash functions does not work, i posted some filenames on the first page, but i can't find a matching hash with your algo in UI.bix.

Maybe my translated code is wrong or you reversed the wrong checksum, there are 3 crc32 & 3 crc32b precalculated tables in the exe.

Code: Select all

    Public Function CalcNameHash(ByVal Name As String) As Integer
        Dim vector As Integer = -1
        Dim idx As Byte = 0

        For i As Integer = 0 To Name.Length - 1
            idx = Val(Name(i)) Xor (vector >> 24)
            vector = cksum2_magic(idx) Xor (vector << 8) 
        Next i

        Return vector
    End Function

Code: Select all

CRC32 :: 00E35478 :: 01236078
	Referenced at 00D7B9B4, 00D7BA4D, 00D7BA75, 
	00D7BA9C, 00D7BAC4, 00D7BAEC, 00D7BB13,
	00D7BB3B, 00D7BB63, 00D7BB9F, 00D7BC18, 
	00D7BC94, 00D7BCED, 00D7BD47, 00D7BD9F, 
	00D7BDF8, 00D7BE52, 00D7BEAA, 00D7BF03, 
	00D7BF70, 00D7BFA6
CRC32 :: 00E57F78 :: 01258B78
	Referenced at 00D84F88, 00D84FA1, 00D84FBA, 
	00D84FDC, 00D84FED, 00D84FFE, 00D8500F, 
	00D85027, 00D85038, 00D85049, 00D8505D, 
	00D8508F, 00D850A8, 00D850C1, 00D850D6, 
	00D850E7, 00D850F8, 00D8510C, 00D85129, 
	00D85142, 00D8515B
CRC32 :: 00E77288 :: 01277E88
	Referenced at 00E3E93D, 00E3E989, 00E3E9BE, 
	00E3E9F3, 00E3EA28, 00E3EA5D, 00E3EA92, 
	00E3EACB, 00E3EB00, 00E3EB4D, 00E3EB78
CRC32b :: 00DD15F0 :: 011D21F0
	Referenced at 00416BED, 00716EF8, 00A1E33D
CRC32b :: 00FA2598 :: 013A3598
	Referenced at 004915FA, 0052BF60, 006CC10A, 00B0F1DD
CRC32b :: 00FC7090 :: 013C8090
	Referenced at 00A87D28, 00CC060F

[Falo]

Your translated code might be wrong, though. I don't know much about VB, but you seem to be using signed arithmetic while the checksum code should do everything unsigned. The string "testing" yields a 32-bit checksum of 0xdec270b7 and a 64-bit checksum of 0xfa9517a51a2f814b. If your translated code gives the same results, it's very likely to be correct.

I had an error, but it was not the signed arithmetic, somehow "Val(Name(i))" results always in 0x00, dunno why, it should return the integer value, but it works with "Asc(Name(i))", the 32-bit value is correct, the 64-bit not, maybe another small bug, but right know i only need the 32-bit name checksum.

signed or unsigned doesn't matter in calculations like this, it's only xor.

[vitoci]

I don't think the hashes are calculated from the actual file names.
Here's a list of a few of the hashes from the Game.bix, they're far too incremental to be calculated from a filename (unless they're all very similar file names but even then the algorithm must be pretty bad to produce hashes that similar)

No this is always the case for Square Enix Games, the hashes are sorted to be incremental, it's not sorted after the offsets.
I have seen this in KH1, KH2, KHBBS, Just Cause 2, it is correct.

i found some filenames in HKShip.exe, but not the function

Code: Select all

Data\UI\loading_generic3_tp.perm.bin
Data\UI\loading_generic4_tp.perm.bin
Data\UI\loading_generic7.tp.perm.bin

Data\UI\Icons_Weapon_QSZ92_TP.perm.bin
Data\UI\Icons_Weapon_45CAL_TP.perm.bin
Data\UI\Icons_Weapon_45CAL02_TP.perm.bin
Data\UI\Icons_Weapon_ACT02_TP.perm.bin
Data\UI\Icons_Weapon_ACT_TP.perm.bin
Data\UI\Icons_Weapon_ARGL_TP.perm.bin
Data\UI\Icons_Weapon_PUMPS_TP.perm.bin
Data\UI\Icons_Weapon_SMG_TP.perm.bin
Data\UI\Icons_Weapon_GOLD50_TP.perm.bin
Data\UI\Icons_Weapon_BATON_TP.perm.bin
Data\UI\Icons_Weapon_BROOM_TP.perm.bin
Data\UI\Icons_Weapon_CLEAVER_TP.perm.bin
Data\UI\Icons_Weapon_KNIFE_TP.perm.bin
Data\UI\Icons_Weapon_CROWBAR_TP.perm.bin
Data\UI\Icons_Weapon_STICK_TP.perm.bin
Data\UI\Icons_Weapon_SHOVEL_TP.perm.bin
Data\UI\Icons_Weapon_BRIEFCASE_TP.perm.bin
Data\UI\Icons_Weapon_HAMMER_TP.perm.bin
Data\UI\Icons_Weapon_CANEWALKING_TP.perm.bin
Data\UI\Icons_Weapon_HANDGRINDER_TP.perm.bin
Data\UI\Icons_Weapon_FISHWRAPPED_TP.perm.bin
Data\UI\Icons_Weapon_WOK.perm.bin
Data\UI\Icons_Weapon_FIREEXTINGUISHER.perm.bin
Data\UI\Icons_Weapon_COOKINGPAN.perm.bin
Data\UI\Icons_Weapon_SHOPPINGBAG.perm.bin
Data\UI\Icons_Weapon_DUFFLEBAG.perm.bin
Data\UI\Icons_Weapon_GROCERYBAG.perm.bin

Data\UI\SpyPC_texturepack.perm.bin
Data\UI\casecomplete_texturepack.perm.bin
Data\UI\unlockables_texturepack.perm.bin
Data\UI\garage_texturepack.perm.bin
Data\UI\gametracker_texturepack.perm.bin
Data\UI\camera_texturepack.perm.bin
Data\UI\camera_cctv_texturepack.perm.bin
Data\UI\options_common_texturepack.perm.bin
Data\UI\options_controllers_TexturePack.perm.bin
Data\UI\options_common_texturepack.perm.bin

data\world\game\dlc\%s\dlc%d.perm.bin
%s\Row_%d\TexturePack_%d_%d%s.perm.bin

Hello, what program you use to find these codes structure

[michalss]

Your translated code might be wrong, though. I don't know much about VB, but you seem to be using signed arithmetic while the checksum code should do everything unsigned. The string "testing" yields a 32-bit checksum of 0xdec270b7 and a 64-bit checksum of 0xfa9517a51a2f814b. If your translated code gives the same results, it's very likely to be correct.

I had an error, but it was not the signed arithmetic, somehow "Val(Name(i))" results always in 0x00, dunno why, it should return the integer value, but it works with "Asc(Name(i))", the 32-bit value is correct, the 64-bit not, maybe another small bug, but right know i only need the 32-bit name checksum.

signed or unsigned doesn't matter in calculations like this, it's only xor.

Hi Falo,

are you planning to do also repacker for X360 please ?

[onixer]

Soory im not programer, please tell me..is it posible to extract *.big file ? if yes please tell me how, in detail!
THANKS!

[cra0]

So no one has got the files extracted? in working order

[neburas]

Hmm... I think the progress will very slow untill UFG releases official tools or Rick works on it.
But It seems he's busy with Dark Souls's working

[Rick]

Hmm... I think the progress will very slow untill UFG releases official tools or Rick works on it.
But It seems he's busy with Dark Souls's working

And Borderlands 2, too many games!

[oveja]

Hmm... I think the progress will very slow untill UFG releases official tools or Rick works on it.
But It seems he's busy with Dark Souls's working

And Borderlands 2, too many games!

I'm looking forward to rick's bl2 save editor and profile editor.

[cra0]

any news?

[soulslayerzx]

Progresss? Or are we really waiting for ufg to release tools of their own. Also, I tried 3d ripper dx and it didn't work, i dont see a ready to capture text, and i tried forcing sleeping dogs to direct x 9 to force it, but it just doesnt launch. lol

[Ekey]

As Rick told there the modified CRC32. Function used also in the scripts for hashing ID's and ect.

Here modified table

Code: Select all

static unsigned long SDTable[256] = {
0x00000000, 0x04C11DB7, 0x09823B6E, 0x0D4326D9, 0x130476DC,
0x17C56B6B, 0x1A864DB2, 0x1E475005, 0x2608EDB8, 0x22C9F00F,
0x2F8AD6D6, 0x2B4BCB61, 0x350C9B64, 0x31CD86D3, 0x3C8EA00A,
0x384FBDBD, 0x4C11DB70, 0x48D0C6C7, 0x4593E01E, 0x4152FDA9,
0x5F15ADAC, 0x5BD4B01B, 0x569796C2, 0x52568B75, 0x6A1936C8,
0x6ED82B7F, 0x639B0DA6, 0x675A1011, 0x791D4014, 0x7DDC5DA3,
0x709F7B7A, 0x745E66CD, 0x9823B6E0, 0x9CE2AB57, 0x91A18D8E,
0x95609039, 0x8B27C03C, 0x8FE6DD8B, 0x82A5FB52, 0x8664E6E5,
0xBE2B5B58, 0xBAEA46EF, 0xB7A96036, 0xB3687D81, 0xAD2F2D84,
0xA9EE3033, 0xA4AD16EA, 0xA06C0B5D, 0xD4326D90, 0xD0F37027,
0xDDB056FE, 0xD9714B49, 0xC7361B4C, 0xC3F706FB, 0xCEB42022,
0xCA753D95, 0xF23A8028, 0xF6FB9D9F, 0xFBB8BB46, 0xFF79A6F1,
0xE13EF6F4, 0xE5FFEB43, 0xE8BCCD9A, 0xEC7DD02D, 0x34867077,
0x30476DC0, 0x3D044B19, 0x39C556AE, 0x278206AB, 0x23431B1C,
0x2E003DC5, 0x2AC12072, 0x128E9DCF, 0x164F8078, 0x1B0CA6A1,
0x1FCDBB16, 0x018AEB13, 0x054BF6A4, 0x0808D07D, 0x0CC9CDCA,
0x7897AB07, 0x7C56B6B0, 0x71159069, 0x75D48DDE, 0x6B93DDDB,
0x6F52C06C, 0x6211E6B5, 0x66D0FB02, 0x5E9F46BF, 0x5A5E5B08,
0x571D7DD1, 0x53DC6066, 0x4D9B3063, 0x495A2DD4, 0x44190B0D,
0x40D816BA, 0xACA5C697, 0xA864DB20, 0xA527FDF9, 0xA1E6E04E,
0xBFA1B04B, 0xBB60ADFC, 0xB6238B25, 0xB2E29692, 0x8AAD2B2F,
0x8E6C3698, 0x832F1041, 0x87EE0DF6, 0x99A95DF3, 0x9D684044,
0x902B669D, 0x94EA7B2A, 0xE0B41DE7, 0xE4750050, 0xE9362689,
0xEDF73B3E, 0xF3B06B3B, 0xF771768C, 0xFA325055, 0xFEF34DE2,
0xC6BCF05F, 0xC27DEDE8, 0xCF3ECB31, 0xCBFFD686, 0xD5B88683,
0xD1799B34, 0xDC3ABDED, 0xD8FBA05A, 0x690CE0EE, 0x6DCDFD59,
0x608EDB80, 0x644FC637, 0x7A089632, 0x7EC98B85, 0x738AAD5C,
0x774BB0EB, 0x4F040D56, 0x4BC510E1, 0x46863638, 0x42472B8F,
0x5C007B8A, 0x58C1663D, 0x558240E4, 0x51435D53, 0x251D3B9E,
0x21DC2629, 0x2C9F00F0, 0x285E1D47, 0x36194D42, 0x32D850F5,
0x3F9B762C, 0x3B5A6B9B, 0x0315D626, 0x07D4CB91, 0x0A97ED48,
0x0E56F0FF, 0x1011A0FA, 0x14D0BD4D, 0x19939B94, 0x1D528623,
0xF12F560E, 0xF5EE4BB9, 0xF8AD6D60, 0xFC6C70D7, 0xE22B20D2,
0xE6EA3D65, 0xEBA91BBC, 0xEF68060B, 0xD727BBB6, 0xD3E6A601,
0xDEA580D8, 0xDA649D6F, 0xC423CD6A, 0xC0E2D0DD, 0xCDA1F604,
0xC960EBB3, 0xBD3E8D7E, 0xB9FF90C9, 0xB4BCB610, 0xB07DABA7,
0xAE3AFBA2, 0xAAFBE615, 0xA7B8C0CC, 0xA379DD7B, 0x9B3660C6,
0x9FF77D71, 0x92B45BA8, 0x9675461F, 0x8832161A, 0x8CF30BAD,
0x81B02D74, 0x857130C3, 0x5D8A9099, 0x594B8D2E, 0x5408ABF7,
0x50C9B640, 0x4E8EE645, 0x4A4FFBF2, 0x470CDD2B, 0x43CDC09C,
0x7B827D21, 0x7F436096, 0x7200464F, 0x76C15BF8, 0x68860BFD,
0x6C47164A, 0x61043093, 0x65C52D24, 0x119B4BE9, 0x155A565E,
0x18197087, 0x1CD86D30, 0x029F3D35, 0x065E2082, 0x0B1D065B,
0x0FDC1BEC, 0x3793A651, 0x3352BBE6, 0x3E119D3F, 0x3AD08088,
0x2497D08D, 0x2056CD3A, 0x2D15EBE3, 0x29D4F654, 0xC5A92679,
0xC1683BCE, 0xCC2B1D17, 0xC8EA00A0, 0xD6AD50A5, 0xD26C4D12,
0xDF2F6BCB, 0xDBEE767C, 0xE3A1CBC1, 0xE760D676, 0xEA23F0AF,
0xEEE2ED18, 0xF0A5BD1D, 0xF464A0AA, 0xF9278673, 0xFDE69BC4,
0x89B8FD09, 0x8D79E0BE, 0x803AC667, 0x84FBDBD0, 0x9ABC8BD5,
0x9E7D9662, 0x933EB0BB, 0x97FFAD0C, 0xAFB010B1, 0xAB710D06,
0xA6322BDF, 0xA2F33668, 0xBCB4666D, 0xB8757BDA, 0xB5365D03,
0xB1F740B4}; 

Script with new formatting hash ID

Code: Select all

open FDDE bix 1
goto 0x58 1
get numFiles long 1
goto 0x90 1

for i = 0 < numFiles
    get hash long 1
    get Offset long 1
    get unk1 long 1
    get CSize long 1
    get unk2 long 1
    get USize long 1
   
    set SizeEmpty unk1
    math SizeEmpty &= 0xFFF
    math Offset *= 4
    math Offset += SizeEmpty

    string NAME p= "%08X" hash
   
    if CSize == 0
      log NAME Offset USize
    else
      log NAME Offset CSize
    endif
next i

I write tool for generate hash, maybe useful for someone

Code: Select all

[Usage]
        SDHasher <Mode> <Text>

[Modes]
        f - Generate Hash for FileName
        s - Generate Hash for Scripts Labels, ID ect.

[Examples]
        SDHasher -f Data\UI\loading_generic3_tp.perm.bin
        SDHasher -s TransformNodeComponent::sIdentifyNode

Here Example

GL in ripping (some founded filenames you can found on Rick's SVN here)

PS: Hate you all because had to install Win7

[Ekey]

Decompress function (sub_8B1760 - example used by game)

[soulslayerzx]

Lol. I'm kind a noob at this, but I have no idea what to do with these programs. I'll see what happens, one seems like a list of the models and one seems to be a converter?