hey man do you have a compiled version? it wont compile for me for some reason in visual studioCarLuver69 wrote:Posting my research here in case anyone can put it to good use. I'll probably make my own tools eventually, but I don't have an ETA on that
Thanks to aluigi for his Mafia .SDS (PS3) script, it helped point me in the right direction.
NOTE: I'm probably wrong on a lot of this, so please feel free to make corrections. I'm most likely off on the chunkCount being used to determine how many SDSChunkData's there are.
Code: Select all
struct SDSFile { int magic; // 0x534453 ('SDS\0') int version; // 0x14 (v20 / v1.4?) enum SDSType : int { PC = 0x4350, /* These are ASSUMED, I only have the PC version */ PS4 = 0x345350, XB1 = 0x314258, } type; int unk_0C; // 0x5DE53FDE int dataInfoOffset; int dataOffset; int unk_18; // always zero? int unk_1C; // offset / size of something? int unk_20; // always zero? int unk_24; // offset / size of something? int unk_28; // always zero? int unk_2C; // 1 int unk_30; // always zero? int unk_34; // always zero? int unk_38; // always zero? int unk_3C; // always zero? int chunkCount; // how many data chunks? (educated guess) int reserved; // checksum, hash, or key of some sort? /* @ dataInfoOffset */ struct SDSDataInfo { int count; // how many data types there are struct SDSDataType { int typeIndex; // local index to reference this type int strLen; char typeName[strLen]; // no null-terminator int reserved; // possibly a guard? (always zero?) } dataTypes[count]; } dataInfo; /* @ dataOffset */ struct SDSData { int magic; // 0x6C7A4555 ('UEzl') int unk_04; // 0x10000? (never seems to change) char unk_05; // 4? struct SDSChunkData { int dataSize; // size of SDSChunkData char dataType; // reference to SDSDataInfo.dataType struct SDSChunkData { int memorySize; // size in memory? (e.g. uncompressed size) int bufferOffset; // offset to start of data int memorySize_2; // SDSData.unk_04 again? short unk_0A; // number of something? (this format is confusing) short unk_0C; // 0x80F? short bufferSize; // this might be an int, but unlikely /* <<< Data padding >>> */ /* @ bufferOffset */ char buffer[bufferSize]; } data; } chunks[chunkCount]; }; };
Important information: this site is currently scheduled to go offline indefinitely by end of the year.
Mafia 3 - .SDS
Re: Mafia 3 - .SDS
-
- ultra-n00b
- Posts: 3
- Joined: Mon Jun 23, 2014 10:06 am
- Has thanked: 2 times
- Been thanked: 3 times
Re: Mafia 3 - .SDS
Ok the thing with zlib, I get it.Ekey wrote:They not encrypted, chunks are compressed with Zlib. About XML toc > not exist anymore. Content filenames hashed (fnv64a).BitEmE wrote: Also the content seems to be encrypted, I've tried to open an encrypted DDS file -> exception.
Some namesCode: Select all
uint64 m3_gethash(unsigned char *m_String, int dwLength) { uint64 dwHash = 0xCBF29CE484222325; for(int i = 0; i < dwLength; i++) { dwHash *= 0x00000100000001B3; dwHash ^= m_String[i]; } return dwHash; }
For DDS you need skip additional header with 0x2F bytes after decompress. You can do it here in TextureViewer:Code: Select all
/gui/pop/garage.dds
Try set to 0x2F for code (memory.Position = 0L;) and this will be enought i guessCode: Select all
private void UpdatePreview() { ... }
But i have the following problem at the moment:
My .SDS File has 9 chunks, the alignment is 0x10000 (65335)
The first file has a size of 699239 bytes (682kb)
Of course I can't read the file, because there are too less chunks (at least I'm getting exceptions thrown in my face)
9 Chunks a. 65335 bytes alignment = 588015 bytes
So I will always fail to read that file.
Am I doing something with zlib wrong?
I think the TypeID and Version are correct? Maybe the file sizes too?
I am confused right now. Would be great if someone could tell me what I'm doing wrong or at least giving me a clue
Thanks!
-
- ultra-n00b
- Posts: 4
- Joined: Sun Dec 19, 2010 10:15 pm
- Has thanked: 4 times
- Been thanked: 5 times
Re: Mafia 3 - .SDS
Zlib decompression is exactly the same as was in the M2BitEmE wrote:...
About files.. It looks like now the file header has bigger size than was before (36 bytes instead of 30, I have such assumption because there was extra 11 bytes between file header and DDS file magic in M2, if we follow the same logic - the header size has increased by 6 bytes)
here is my 010 Editor Templates for sds files and decompressed memory blocks https://github.com/Chipsman/mafia3-010ETemplates
may be it will help you somehow
p.s.: anybody knows where radio music is located? here Mafia III\sds_retail\audio or here Mafia III\edit\audio\packages ?
-
- veteran
- Posts: 92
- Joined: Thu Jul 12, 2012 12:24 pm
- Has thanked: 12 times
- Been thanked: 32 times
-
- ultra-n00b
- Posts: 3
- Joined: Mon Jun 23, 2014 10:06 am
- Has thanked: 2 times
- Been thanked: 3 times
Re: Mafia 3 - .SDS
Alright, thanks to your help guys, really apprechiate it!
I'm now able to extract/show DDS files. Unfortunately DDs file in Mafia 3 can have the compression types ATI and DXT10, I'm not quite sure how to fix those decompressions with the texture viewer, but that's just a minor problem. (Also extracting is kind of a problem as there are +4 bytes infront of the magic DDs identifier of the file, so external DDs viewer will fail to load the image)
Also I've modified the XML reading and get now 'readable' XML content. Again, unfortunately, the format has changed in Mafia 3. The Problem right now is that I'm not able to say that the next node in memory is an attribute of a node, is a value, a child, or another node.
For example, with this file it's working because the values are only numeric (I'm using a hack at the moment to detect values inside nodes -> TryDouble)
With this file, it's not working because there are node attributes (see xsi:type is a node instead of attribute)
The structure is weird because it looks like this:
node type = 4
6 bytes = 0
String (length is not written in memory, you have to check for null terminator)
String Terminator
node type = 4
6 bytes = 0
String (length is not written in memory, you have to check for null terminator)
... and so on and so on..
So basically there's always a 4 between all nodes (maybe seperator? idk)
I don't know how I should figure out what's a child node, what's an attribute etc..
Again, here's the source (Just based of Rick's Mafia 2 tool) https://github.com/MartinJK/Mafia3SDSExplorer (in case someone of you wants to try it out etc.)
Thanks for you help guys, really apprechiate it. I'm used to reverse engineer and mod games, this is the first time I'm trying to 'reverse' the files format and create a tool for it
I'm now able to extract/show DDS files. Unfortunately DDs file in Mafia 3 can have the compression types ATI and DXT10, I'm not quite sure how to fix those decompressions with the texture viewer, but that's just a minor problem. (Also extracting is kind of a problem as there are +4 bytes infront of the magic DDs identifier of the file, so external DDs viewer will fail to load the image)
Also I've modified the XML reading and get now 'readable' XML content. Again, unfortunately, the format has changed in Mafia 3. The Problem right now is that I'm not able to say that the next node in memory is an attribute of a node, is a value, a child, or another node.
For example, with this file it's working because the values are only numeric (I'm using a hack at the moment to detect values inside nodes -> TryDouble)
With this file, it's not working because there are node attributes (see xsi:type is a node instead of attribute)
The structure is weird because it looks like this:
node type = 4
6 bytes = 0
String (length is not written in memory, you have to check for null terminator)
String Terminator
node type = 4
6 bytes = 0
String (length is not written in memory, you have to check for null terminator)
... and so on and so on..
So basically there's always a 4 between all nodes (maybe seperator? idk)
I don't know how I should figure out what's a child node, what's an attribute etc..
Again, here's the source (Just based of Rick's Mafia 2 tool) https://github.com/MartinJK/Mafia3SDSExplorer (in case someone of you wants to try it out etc.)
Thanks for you help guys, really apprechiate it. I'm used to reverse engineer and mod games, this is the first time I'm trying to 'reverse' the files format and create a tool for it
-
- M-M-M-Monster veteran
- Posts: 1823
- Joined: Wed Mar 31, 2010 6:54 am
- Has thanked: 92 times
- Been thanked: 1058 times
Re: Mafia 3 - .SDS
Some new names
Code: Select all
/gui/pop/missions.dds
/gui/pop/womaniser_voodoo_dancer.dds
/gui/pop/alternative_shop.dds
/gui/pop/classy_shop.dds
/gui/pop/cloth_shop.dds
/gui/pop/fashion_shop.dds
/gui/pop/gun_shop.dds
/gui/pop/item_shop.dds
/gui/pop/shades_shop.dds
/gui/bc_map_pcp_lab_a.gfx
/gui/config/maps/bc_map_pcp_lab_a
/gui/hideout_district_assignment.swf
/gui/images/icons/button_a.dds
/gui/images/icons/button_b.dds
/gui/images/icons/button_x.dds
/gui/images/icons/other_icons0031.dds
/gui/images/icons/other_icons0037.dds
/gui/images/icons/other_icons0099.dds
/gui/images/icons/other_icons0101.dds
/gui/images/icons/other_icons0103.dds
/gui/images/icons/other_icons0105.dds
/gui/images/icons/other_icons0106.dds
/gui/interact_collectable.swf
/gui/interact_object.swf
/gui/interact_weapon_pickup.swf
/gui/in_world_menu.swf
/gui/pop/garage.dds
/gui/pop/home.dds
/gui/scripted_test.swf
/xbin/cities.xlaybin
/xbin/plotlines.xlaybin
/xbin/missions.xlaybin
/xbin/quests.xlaybin
/xbin/game_events.xlaybin
/xbin/persistent_characters.xlaybin
/xbin/businesses.xlaybin
/xbin/speeches.xlaybin
/xbin/shops.xlaybin
/xbin/cities_game_group.xbin
/xbin/plotlines_group.xbin
/xbin/missions_group.xbin
/xbin/quests_group.xbin
/xbin/game_events_group.xbin
/xbin/persistent_characters_group.xbin
/xbin/businesses_group.xbin
/xbin/streammap_gameinit_group.xbin
/xbin/speeches_game_group.xbin
/xbin/generic_speech_mp.xlaybin
/xbin/generic_speech_mp_group.xbin
/xbin/generic_speech_situations_group.xbin
/xbin/generic_speech_variations_group.xbin
/xbin/shops_group.xbin
/xbin/game_main.xlaybin
/xbin/game_main_group.xbin
/xbin/expressiondata.xlaybin
/xbin/game_expressions_group.xbin
/xbin/charactertable.xlaybin
/xbin/charactertable_cinematics_group.xbin
/sds/maps/mapa_bourbon2.sds
/sds/maps/mapa_testscene.sds
/root/Situations
/root/Variations
/root/CharacterTableMP
/root/StreamMapTableMP
/root/GameEventsTableMP
/root/QuestsTableMP
/root/MissionsTable
/root/SpeechesTableMP
/root/PlotlinesTableMP
/root/BusinessesTableMP
/root/PersistentCharactersTableMP
/fsm/persistent_characters/alvarez_active
/fsm/persistent_characters/alvarez_inactive
/fsm/persistent_characters/biaggio_active
/fsm/persistent_characters/biaggio_inactive
/fsm/persistent_characters/cesar_active
/fsm/persistent_characters/cesar_inactive
/fsm/persistent_characters/clayton_active
/fsm/persistent_characters/clayton_inactive
/fsm/persistent_characters/creed_active
/fsm/persistent_characters/creed_inactive
/fsm/persistent_characters/danny_quest_active
/fsm/persistent_characters/danny_quest_inactive
/fsm/persistent_characters/doctor_active
/fsm/persistent_characters/doctor_inactive
/fsm/persistent_characters/donovan_active
/fsm/persistent_characters/donovan_inactive
/fsm/persistent_characters/mack_quest_active
/fsm/persistent_characters/mack_quest_inactive
/fsm/persistent_characters/marcano_active
/fsm/persistent_characters/marcano_inactive
/fsm/persistent_characters/maria_active
/fsm/persistent_characters/maria_inactive
/fsm/persistent_characters/martinez_active
/fsm/persistent_characters/martinez_inactive
/fsm/persistent_characters/professor_active
/fsm/persistent_characters/professor_inactive
/fsm/persistent_characters/sixto_active
/fsm/persistent_characters/sixto_inactive
/scripts/gamelogic/bc_openworld
/scripts/gamelogic/fpm/fpm_plotline
/scripts/gamelogic/milestones/milestone27/drug_farm_main
/scripts/gamelogic/milestones/milestone27/gameplay_bricks/shooting_from_boat
/scripts/gamelogic/milestones/milestone27/gameplay_bricks/shooting_from_car
/scripts/gamelogic/milestones/milestone28/steal_rum
/scripts/gamelogic/plotline_act2
/scripts/gamelogic/testscenes/bourbon_test_mission/bourbon_test_mission
/scripts/gamelogic/testscenes/districts/west_marais_district_structure_quest
/scripts/gamelogic/testscenes/districts/wharf_district_structure_quest
/scripts/gamelogic/testscenes/game_structure_init
/scripts/gamelogic/testscenes/game_structure_quest
/scripts/gamelogic/testscenes/open_world/main
/scripts/gamelogic/testscenes/racing_focustest/main
/scripts/gamelogic/testscenes/test_bartek/simple
/scripts/gamelogic/testscenes/test_racing_pod/main
My Github repo
Re: Mafia 3 - .SDS
We owe you bro, thanks. Can you share compiled version of this script? And i want a question, this tool does it work to subtitles.sds files, can you extract and repack clearly?BitEmE wrote:Alright, thanks to your help guys, really apprechiate it!
Thanks for you help guys, really apprechiate it. I'm used to reverse engineer and mod games, this is the first time I'm trying to 'reverse' the files format and create a tool for it
-
- M-M-M-Monster veteran
- Posts: 1823
- Joined: Wed Mar 31, 2010 6:54 am
- Has thanked: 92 times
- Been thanked: 1058 times
Re: Mafia 3 - .SDS
Info for PCKG archives
Code: Select all
struct PCKGHeader
{
uint32_t dwID; // Always PCKG (0x50434B47)
uint32_t dwVersion; // 1
uint32_t dwTotalFiles;
uint32_t dwFlag; // 0xFFFFFFFF
};
struct PCKGEntry
{
uint64_t dwHash; // Hash of filename?
uint32_t dwOffset;
uint32_t dwSize;
uint32_t dwZSize;
};
My Github repo
Re: Mafia 3 - .SDS
Can you please post here builded version of your tool? ThxBitEmE wrote:Alright, thanks to your help guys, really apprechiate it!
I'm now able to extract/show DDS files. Unfortunately DDs file in Mafia 3 can have the compression types ATI and DXT10, I'm not quite sure how to fix those decompressions with the texture viewer, but that's just a minor problem. (Also extracting is kind of a problem as there are +4 bytes infront of the magic DDs identifier of the file, so external DDs viewer will fail to load the image)
Also I've modified the XML reading and get now 'readable' XML content. Again, unfortunately, the format has changed in Mafia 3. The Problem right now is that I'm not able to say that the next node in memory is an attribute of a node, is a value, a child, or another node.
For example, with this file it's working because the values are only numeric (I'm using a hack at the moment to detect values inside nodes -> TryDouble)
With this file, it's not working because there are node attributes (see xsi:type is a node instead of attribute)
The structure is weird because it looks like this:
node type = 4
6 bytes = 0
String (length is not written in memory, you have to check for null terminator)
String Terminator
node type = 4
6 bytes = 0
String (length is not written in memory, you have to check for null terminator)
... and so on and so on..
So basically there's always a 4 between all nodes (maybe seperator? idk)
I don't know how I should figure out what's a child node, what's an attribute etc..
Again, here's the source (Just based of Rick's Mafia 2 tool) https://github.com/MartinJK/Mafia3SDSExplorer (in case someone of you wants to try it out etc.)
Thanks for you help guys, really apprechiate it. I'm used to reverse engineer and mod games, this is the first time I'm trying to 'reverse' the files format and create a tool for it
-
- ultra-n00b
- Posts: 4
- Joined: Sun Dec 19, 2010 10:15 pm
- Has thanked: 4 times
- Been thanked: 5 times
Re: Mafia 3 - .SDS
Audio packages format info (AKPK)
Audio is stored in .wav container but the format/codec inside is unknown
here is 010 template https://github.com/Chipsman/mafia3-010E ... /m3-pck.bt
Code: Select all
struct Header
{
char dwMagic[4]; // AKPK (0x4B504B41h)
DWORD dwHeaderSize; // +0x4
DWORD _f8;
DWORD _fC;
DWORD _f10;
DWORD _f14;
DWORD _f18;
DWORD _f1C;
DWORD _f20;
DWORD _f24;
DWORD _f28;
DWORD _f2C;
DWORD _f30;
DWORD dwFileCount; //+0x34
};
struct FileEntry
{
DWORD dwHash; // +0x0
DWORD _f4; // always 16, block size ? or aligment ?
DWORD dwSize; // +0x8
DWORD dwOffsetInBlocks; //+0xC // block size == 16
DWORD _f10; // always 0, // or dwOffsetInBlocks is QWORD
};
here is 010 template https://github.com/Chipsman/mafia3-010E ... /m3-pck.bt
Re: Mafia 3 - .SDS
Skrillex wrote:We owe you bro, thanks. Can you share compiled version of this script? And i want a question, this tool does it work to subtitles.sds files, can you extract and repack clearly?BitEmE wrote:Alright, thanks to your help guys, really apprechiate it!
Thanks for you help guys, really apprechiate it. I'm used to reverse engineer and mod games, this is the first time I'm trying to 'reverse' the files format and create a tool for it
-
- veteran
- Posts: 113
- Joined: Mon Oct 13, 2014 6:54 am
- Has thanked: 52 times
- Been thanked: 19 times
Re: Mafia 3 - .SDS
I just wanted to add my support for anyone thinking of putting tools together for this game! Love it! Great character models, and vehicles, too! Would love to be able to get in and take a proper look at some of them...!
-
- MEGAVETERAN
- Posts: 2647
- Joined: Tue Mar 24, 2015 8:12 pm
- Has thanked: 65 times
- Been thanked: 2871 times
Re: Mafia 3 - .SDS
yeah when decrypted with the tool the results from the sds are dat files with random names and they are encrypted. so no actual assetsdaemon1 wrote:they are not encryptediambosh wrote:any progress on decrypting the dat files?