idTech 5 Audio (.streamed)

[Researchman]

Currently solved with: RAGE, Wolfenstein: The New Order, The Evil Within.

=======================================
Format, which can be a container or archive with compression.

Currently used in three games - Rage, Wolfenstein: New Order, The Evil Within(aka Psychobreak).
All information down is about PC versions.

Rage:
Single files for audio and voices. Uses different format that .resource files.
Nova Extractor dont find anything.
Possibly uses chunk structure and byte ordering, no header.

streamed.resources
First bytes:

Code: Select all

...¤.]..пьaвфю. ...г..:р.б..Ъ.."р‘ `..ят°О/СG+.е.с..'....ва....G..@буа.B.3.t.;"рРа.у~ .яЖб.,и.1.

Last bytes:

Code: Select all

.......ыяюя............р.....................................................чяшя...............................................................

english.streamed
First bytes:

Code: Select all

...ќяДя.>Ф....ь0г.р.С0.я...вMт.р. РPР..<ф.я..п0Т

Last bytes:

Code: Select all

...а....рр.....р..р..........цячя......................р.р........................р.............

Wolfenstein: New Order:
Single files for audio and voices. Uses different format that .resource files.
Nova Extractor find oggs, but only small audio files is extractable. Also detects 987MB(18474 oggs, 48000 Hz, stereo, 128kbps) from 1.07GB streamed.resource file.
Uses chunk sctructure(several empty big blocks), no header.

streamed.resources
First bytes:

Code: Select all

OggS..........)........ Ёя...vorbis.....Ђ»..яяяя.ф..яяяяё.OggS..

Last bytes:

Code: Select all

ќj.wћФЏmL¦.8a·ЖМґ...бтћ\зп?/П5–—u±Клї.Ч..ыс |Юk.

english.streamed:
First bytes:

Code: Select all

OggS..........)........…l1...vorbis.....Ђ»..яяяя.w..яяяяё.OggS..

Last bytes:

Code: Select all

8и БLВ5..,р.t.ђCѓ..®-ъЯYx©.Ё_Ё).................

Evil Within(aka Psychobreak):
Separate files for every level. Example: a_planted_seed_will_grow.streamed, a_planted_seed_will_grow.tangoresource, a_planted_seed_will_grow_en.streamed. Has the same format as .resource files, but can use another compression method.
Nova Extractor dont find anything.
Extractable with bms script(can be find in aluigi's quickbms page), but dont sure about correcting extraction.
Uses byte ordering(table in the end of file), have header(4 bytes, #”«Н as 23 94 AB CD).

a_planted_seed_will_grow.streamed
First bytes:

Code: Select all

#”«Н.P®Т..Јk.M.·юияпЊЪЫрV*О..E!QП.ъбQ..Ю..®а....

Last bytes:

Code: Select all

R.OTФ..|”..|”...S.OСh..rў..rў...T.PD...jИ..jИ

a_planted_seed_will_grow_en.streamed
First bytes:

Code: Select all

#”«Н.ВW(..Rэ.....!..а.[!б/у-4с1N.Гщ.юЮ.р.~3 -аЬ.

Last bytes:

Code: Select all

ш.Gћш...Я.Ў:њ...„...„...а.¬D ...„...„...б.·M¤...„...„

a_planted_seed_will_grow.tangoresource:
First bytes:

Code: Select all

#”«Н..3л<...textures/uitng/img/chapter_select_pic/chapter_load_1

Last bytes:

Code: Select all

€A....9¤..€B....9Є..€C....9°..€D....9¶..€E....9ј

Files for researching can be transfered throught PM or request for public download link in thread.

[Researchman]

Few extractable examples(Wolfenstein: New Order, The Evil Within):
http://dropmefiles.com/GPLmA
https://mega.co.nz/#!lFkFWaoB!k1PdxtSU8 ... LIX24tbuEY

[Researchman]

Information about console versions(Xbox360 and PS3 can be researched currently):
Rage:
Uses the same formats as in PC version.

Wolfenstein: New Order:
Uses the same formats as in PC version.

Evil Within:
.tpr+.str files. Same format as in PC version.
.tpr contains .tangoresource data, .str contains .streamed data
PS3 version uses .MSF format for audio(MP3). Just rename .MSF to .MP3. (tested in Media Player Classic)
Xbox360 version uses .XMA format for audio.
Extractable with bms script(can be find in aluigi's quickbms page).


Additional info about audio in PC versions - uses WAV with MS ADPCM.
Currently see two problems:
- container unpacking(streamed.resource) in Rage and Wolfenstein: New Order. (all versions)
- MS ADPCM decoding. Currently can be researched in Evil Within. (PC version)

[Pepper]

Information about console versions(Xbox360 and PS3 can be researched currently):
Rage:
Uses the same formats as in PC version.

Wolfenstein: New Order:
Uses the same formats as in PC version.

Evil Within:
.tpr+.str files. Same format as in PC version.
.tpr contains .tangoresource data, .str contains .streamed data
PS3 version uses .MSF format for audio(MP3). Just rename .MSF to .MP3. (tested in Media Player Classic)
Xbox360 version uses .XMA format for audio.
Extractable with bms script(can be find in aluigi's quickbms page).


Additional info about audio in PC versions - uses WAV with MS ADPCM.
Currently see two problems:
- container unpacking(streamed.resource) in Rage and Wolfenstein: New Order. (all versions)
- MS ADPCM decoding. Currently can be researched in Evil Within. (PC version)

if you have RAGE on steam, the container unpacking is not an issue. install the "RAGE Tool Kit" found under "Tools" category of library. (and now steam downloads 20gb or so) all sounds are now available in the folder structure as ms adpcm. The bigger issue with RAGE was that the .streamed and .resources used some kind of link for the header in one and the raw data in the other. I know the ps3 version of wolfenstein the new order uses mp3 files, if you scan it for them you'll get then all. albeit nameless.

[Researchman]

Sorry, but I dont understand your point.

I think, possible to unpack .streamed data without a header in another file, because .streamed files of Evil Within can be unpacked in any version(PC, PS3, Xbox360).

Also:

- .streamed files of PS3 version of Wolfenstein: New Order contains .msf files variant, which is mp3. If cut it as samples throught Nova Extractor, so begin and end of files will be cuted(has place with Killzone 3 as minimum).

- I dont find MS ADPCM decoder for idTech5 PC versions of games on him. They dont supported in Media Player Classic, VLC, XMPlay.

[Pepper]

Sorry, but I dont understand your point.

I think, possible to unpack .streamed data without a header in another file, because .streamed files of Evil Within can be unpacked in any version(PC, PS3, Xbox360).

Also:

- .streamed files of PS3 version of Wolfenstein: New Order contains .msf files variant, which is mp3. If cut it as samples throught Nova Extractor, so begin and end of files will be cuted(has place with Killzone 3 as minimum).

- I dont find MS ADPCM decoder for idTech5 PC versions of games on him. They dont supported in Media Player Classic, VLC, XMPlay.

I was just informing you that the files in the rage toolkit are all the sounds in ms adpcm. decoding worked for me via the xbox adpcm codec (only way to decode it on windows 8 with goldwave, default ms adpcm codecs have been gimped for a long time as they are .acm which is no longer supported) also the exact frequency of each file is semi-random. all 48khz files are a random number +- 2000 within the range of 48000, which messes up alot of decoding, or makes playback impossible without first doing a re encode on certain audio systems. (realtek HD handles adpcm at 47329khz with a noticable 2 seconds of lag before even being able to start playback, other drivers fail altogether.) the xbox adpcm codec installer that is stored in ekszbox abx's sourceforge repository handles a wide array of ms/xbox adpcm very well, better than vlc or mpc does by default. the strange somewhat random khz ranges is an issue for not just latency but also decoding depending on your system.

[spider91]

It will restore headers to get playable adpcm .wav's

Code: Select all

# The Evil Within adpcm script (PC)
# Put .bsnd (extracted from .tangoresource) and .msadpcm (extracted from .streamed) files in the same dir and use .bsnd as input
# Written by spider91
# script for QuickBMS http://quickbms.aluigi.org
idstring "bsnf"
get BSND_NAME FILENAME
get WAV_NAME BASENAME
string WAV_NAME -= "_msadpcm"
string WAV_NAME += ".msadpcm"
open FDSE BSND_NAME 0
open FDSE WAV_NAME 1
get RIFFSIZE ASIZE 1
math RIFFSIZE += 70
set FMTSIZE long 50
get DATA_SIZE ASIZE 1
Goto -50 0
get CODEC_ID short
get CHANNELS short
get FREQUENCY long
get BPS long
get ALIGN short
get UNKNOWN1 long
get UNKNOWN2 long
get UNKNOWN3 long
get UNKNOWN4 long
get UNKNOWN5 long
get UNKNOWN6 long
get UNKNOWN7 long
get UNKNOWN8 long
get UNKNOWN9 long
putVarChr MEMORY_FILE 78 0
log MEMORY_FILE 0 0
set MEMORY_FILE binary "\x52\x49\x46\x46\x00\x00\x00\x00\x57\x41\x56\x45\x66\x6d\x74\x20\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x64\x61\x74\x61\x00\x00\x00\x00"
putVarChr MEMORY_FILE 4 RIFFSIZE long
putVarChr MEMORY_FILE 16 FMTSIZE long
putVarChr MEMORY_FILE 20 CODEC_ID short
putVarChr MEMORY_FILE 22 CHANNELS short
putVarChr MEMORY_FILE 24 FREQUENCY long
putVarChr MEMORY_FILE 28 BPS long
putVarChr MEMORY_FILE 32 ALIGN short
putVarChr MEMORY_FILE 34 UNKNOWN1 long
putVarChr MEMORY_FILE 38 UNKNOWN2 long
putVarChr MEMORY_FILE 42 UNKNOWN3 long
putVarChr MEMORY_FILE 46 UNKNOWN4 long
putVarChr MEMORY_FILE 50 UNKNOWN5 long
putVarChr MEMORY_FILE 54 UNKNOWN6 long
putVarChr MEMORY_FILE 58 UNKNOWN7 long
putVarChr MEMORY_FILE 62 UNKNOWN8 long
putVarChr MEMORY_FILE 66 UNKNOWN9 long
putVarChr MEMORY_FILE 74 DATA_SIZE long
append
log MEMORY_FILE 0 DATA_SIZE 1
get NAME BASENAME
string NAME -= "_msadpcm"
string NAME += ".wav"
set FULLSIZE long RIFFSIZE
math FULLSIZE += 8
log NAME 0 FULLSIZE MEMORY_FILE

[Researchman]

It will restore headers to get playable adpcm .wav's

Works. Thanks.

[spider91]

Some .bsnd files have bad "data" size. Updated script to avoid that problem.

[Researchman]

Additional information:
- Wolfenstein: The New Order dont use .index file for streamed.resources, english.streamed and other separate audio archives.
- File headers stores separated in gameresources.resources(Rage), chunk0-13(Wolfenstein: The New Order).
- Rage probably uses WAV ADPCM without headers(they stores separated)(.idmsa).
- Wolfenstein: The New Order uses vorbis or wwise ogg (PC version), MSF MP3(PS3 version) for audio in .resources/.streamed files.

Headers for music files in attach. (Rage and Wolfenstein: The New Order PC versions)

[OrangeC]

Thanks!!

This is what i needed. So will test.

EDIT: Is there any automated way to match the headers witht he audio files themselves.

[Researchman]

No, because no one made .streamed files unpacker and find a way to get filenames.

[spider91]

-File headers stores separated in gameresources.resources(Rage), chunk0-13(Wolfenstein: The New Order).
- Wolfenstein: The New Order uses vorbis or wwise ogg (PC version), MSF MP3(PS3 version) for audio in .resources/.streamed files.

I'm not good with ogg format (and in WTNO files are simple ogg vorbis), but i'll try to find some way to get filenames for WTNO sounds.

- Rage probably uses WAV ADPCM without headers(they stores separated)(.idmsa).

As i remember there is wav adpcm with headers, but i can be wrong. If they are really without headers i can take a look, it should be similiar to Evil Within.

[spider91]

I've fidured out how to locate WTNO filenames for sounds. They are stored in .index files. I'm pretty busy now, so i don't know when i will write a script, but i'll try to make it as soon as i can.

Upd.
Rage seems to be the same (names stored in gameresources.resources), exept files are headerless adpcm.

[OrangeC]

GREAT!!

Can't wait.