NieR Replicant ver.1.22 .arc

[LinkOFF]

Can someone help with new NieR Replicant game? There is a few arc archives.
A few examples - https://disk.yandex.ru/d/da5YuGsCEZZyIw

[vvampii]

using Zstandard (.arc > .arc.zst) requires password

[vvampii]

more examples (.arc & .arc.zst) + .exe: https://disk.yandex.ru/d/azCHDeM2gBLrkQ?w=1
might need to check .exe file for password ? common.arc has 'library (in game room)' associated files/lines inside.

/a_center_library_01_base/mtl_book01_1nier00_village01_tjblock_002_a6stivy_001chandelierktpicturewoodwalltjmetalsttablewa1ktmarbl4carpet0kt43glass2ceilingwall_2multi_pg0out
/bg/a_center_library_01_base/tex__base

[infogram]

using Zstandard (.arc > .arc.zst) requires password

Hmm, Zstd doesn't seem to use passwords though, at least the dev headers don't seem to have a mention of them (https://github.com/facebook/zstd/blob/dev/lib/zstd.h)

Using "zstd.exe -d info.arc -o info.dec" seemed to run fine for the dlc\dlc01\info.arc file, decompressed file format is very weird though, not sure if it's actually decompressed properly.

(zstd.exe from https://github.com/facebook/zstd/releas ... -win64.zip)

Fails with every other .arc I tried with though, 'File "dlc01.arc" not compressed by zstd' or 'zstd: dlc01.arc: unsupported format', but it has the same header as the info.arc that worked so dunno what's going on there. Is there a Zstd variant that uses extra algos or something dumb like that?

E: oh, zstd.exe seems to work with the common.arc too, decompresses to ~30MB file that seems to make more sense than the info.arc I tried. Still no luck with larger files though.

[DKDave]

Yeah, there's no password protection, it's just pure ZSTD from the few sample files I've looked at that people have posted.

The files info.arc and common.arc are both 1 single block of compressed data, whereas others like lang1.arc and lang2.arc are made up of small blocks of compressed data that have to be read separately.

info.arc contains the master list for all archives. Some of them, like common.arc need to be decompressed first as the file table in info.arc references the decompressed offsets, but for others like lang1.arc it references the individual compressed sections.

Once the files from common.arc are extracted, they contain what seems like thousands of file tables that refer to other data archives.

At least that's what it seems like!

[DaniKH]

The sample files they shared arent really important if we want just the models. They are inside of stream.arc, but its too heavy, impossible to upload somewhere.

So, anyone had any idea about how to extract the assets files inside stream.arc? I tried with many arc unpacker tools but nothing works

[DKDave]

Like I mentioned, the sample files *are* important because info.arc contains the file table which has several thousand entries for stream.arc. So you need that to get the offsets and sizes for all of the individual files in stream.arc, easier than ripping them manually.

I've managed to parse the info.arc file table, but I don't have stream.arc to test it with at the moment (as it's probably about 13 GB), so here's an example of some entries that reference stream.arc (offset/size/filename). They aren't in any particular order in the file table. I assume these are meshes by the filnames.

0x0000000000580b00 0x0000000000041c5c bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_house06
0x00000000004aa7d0 0x00000000000249ad bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_house04
0x00000000004cf180 0x00000000000b197d bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_house05
0x0000000000408070 0x0000000000060cad bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_house02
0x0000000000468d20 0x0000000000041ab0 bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_house03
0x00000000003cef60 0x0000000000039105 bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_house01
0x0000000000385da0 0x00000000000491b2 bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_entrance01
0x000000000081bd40 0x000000000001a72f bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_pg001
0x00000000007b7c90 0x00000000000640aa bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_pg000
0x0000000000106690 0x00000000001632ba bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_cliff01
0x0000000000269950 0x000000000004bbaf bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_cliff02
0x00000000002b5500 0x00000000000d0894 bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_cliff03
0x0000000000027820 0x00000000000dee6c bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_bridge_cliff_01
0x000000000084f010 0x00000000000047f0 bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_shadow
0x00000000005c2760 0x00000000001f390f bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_obj01
0x00000000007b6070 0x0000000000001c1e bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_obj02
0x0000000000836470 0x00000000000133b7 bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_scaff01
0x0000000000849830 0x00000000000057db bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_scaff02
0x0000000000000000 0x0000000000009ff6 bg/a_cliff_village_01/a_cliff_village_01_base/msh_a_cliff_village_01_base
0x000000000000a000 0x000000000001d820 bg/a_cliff_village_01/a_cliff_village_01_base/msh_spl_bridge_bag_01
0x0000000000853800 0x000000000920b240 bg/a_cliff_village_01/a_cliff_village_01_base/tex_a_cliff_village_01_base

[infogram]

Like I mentioned, the sample files *are* important because info.arc contains the file table which has several thousand entries for stream.arc. So you need that to get the offsets and sizes for all of the individual files in stream.arc, easier than ripping them manually.

I've managed to parse the info.arc file table, but I don't have stream.arc to test it with at the moment (as it's probably about 13 GB), so here's an example of some entries that reference stream.arc (offset/size/filename). They aren't in any particular order in the file table. I assume these are meshes by the filnames.

Just checked stream.arc and looks like you're on the money Offsets point to zstd chunks, all seem to decompress fine with zstd.exe into some "PACK" format, upped the zstd chunks here if you want to look into them: https://bayfiles.com/vcoeU9s6uf/stream_arc_samples_zip

E: If anyone is interested in the audio/video files, NSACloud has made a tool for extracting/repacking those here: https://github.com/NSACloud/NieR-Audio-Tools
Sadly seems the audio/video pcks are much different to the main ones we've been discussing here, nice to see such quick progress though, guess we won't have to wait long for soundtrack mods

[vvampii]

I've managed to parse the info.arc file table, but I don't have stream.arc to test it with at the moment (as it's probably about 13 GB)

stream.arc is about 12 gb. also there is init.arc which is about 8 gb (and it might be somewhat similar to stream or maybe common/info)
i will try to upload some ofthese somewhere along with dlc01.arc & its info.arc

upd - init + dlc01: https://disk.yandex.ru/d/fMGTicYB14Q6Iw?w=1

[LinkOFF]

info.arc:
0x48 - number of files (4)
0x138 - table start offset (offset, size and etc.)

single block size is 28 bytes:
4 - unknown
4 - name offset
4 - offset*16
4 - compressed size
4 - uncompressed size
4 - unknown (usually zero)
1 - arc ID (0 - lang1.arc, 1 - lang2.arc, 2 - common.arc, 3 - param.arc, 4 - init.arc, 5 - stream.arc)
1 - flag (Compression 0 or 1)
2 - skip (40 40)

filenames...

[DKDave]

@LinkOFF, some extra info that might be useful for info.arc:

* All offsets are relative to the address where that offset is stored *

Main header - 0x14 bytes:

0 - Text - "BXON"
4 - ?
8 - ?
12 - Int - Offset to archive name/type ("tpArchiveFileParam" in this case - other types seem to have different data layouts)
16 - Int - Offset to archive info header

Archive info header - 0x10 bytes:

0 - Int - Number of main archives
4 - Int - Offset to archive table
8 - Int - Total number of files in the file table
12 - Int - Offset to file table

Archive table - 0xc bytes per entry:

0 - Int - Offset to archive filename
4 - Int - ?
8 - Byte - Flags: 0 = offsets refer to uncompressed archive, 1 = offsets refer to separate ZSTD compressed files, 2 = ?)
9 -
10 -
11 -

File table - 0x1c bytes per entry:

0 - Int - ?
4 - Int - Offset to filename
8 - Int - File offset (*16 as previously mentioned)
12 - Int - File size
16 - Int - ?
20 - Int - ?
24 - Byte - archive index number
25 - Byte - Flags
26 - Byte
27 - Byte

[infogram]

010 template for PACK containers, doesn't seem like there's any compression used (besides the zstd covering the whole PACK), but haven't checked with many files yet.

Code: Select all

struct PACK_Header
{
    DWORD Magic; // 'PACK'
    DWORD Version; // 4 in Nier
    DWORD PackSize;
    DWORD UnkData_Offset; // Offset to some unknown/encrypted data, sometimes looks like floats - offset from base of header

    DWORD Unk10; // value seems to scale with PackSize

    DWORD NameCount;
    DWORD NameTableOffset;

    DWORD Table1_FileCount;
    DWORD Table1_Offset;

    DWORD Table2_FileCount;
    DWORD Table2_Offset;
};

typedef struct 
{
    local long pos = FTell();

    DWORD NameHash;
    DWORD NameOffset;
    DWORD Unk8;

    local long endPos = FTell();

    FSeek(pos + 0x4 + NameOffset); // 0x4 = offset of NameOffset field
    string FileName;
    FSeek(endPos);
} PACK_NameEntry<read=ReadPACK_NameEntry>;

typedef struct
{
    local long pos = FTell();

    DWORD NameHash;
    DWORD NameOffset;
    DWORD DataLength;
    DWORD DataOffset;
    DWORD ErrorOffset; // offset to some weird error string, maybe is set to something besides error on some files...

    local long endPos = FTell();

    FSeek(pos + 0x4 + NameOffset); // 0x4 = offset of NameOffset field
    string FileName;

    FSeek(pos + 0xC + DataOffset);
    BYTE Data[DataLength] <optimize=false>;

    // ErrorOffset is sometimes set to 0x80XXXXXX, no idea what purpose of it is yet
    if(ErrorOffset < 0x80000000) {
        FSeek(pos + 0x10 + ErrorOffset);
        BYTE StringLength;
        char String[StringLength] <optimize=false>;
    }

    FSeek(endPos);
} PACK_FileEntry<read=ReadPACK_FileEntry>;

// funcs to let template window show names next to entry
string ReadPACK_NameEntry(PACK_NameEntry& entry)
{
    return entry.FileName;
}

string ReadPACK_FileEntry(PACK_FileEntry& entry)
{
    return entry.FileName;
}

// Seperate struct so files can be collapsed in template window
struct PACK_NameTable(int NameCount)
{
    PACK_NameEntry Entries[NameCount] <optimize=false>;
};

struct PACK_FileTable(int FileCount)
{
    PACK_FileEntry Entries[FileCount] <optimize=false>;
};

PACK_Header Header;

FSeek(Header.NameTableOffset + 0x18); // 0x18 = offset of NameTableOffset field
if(Header.NameCount > 0)
    PACK_NameTable Names(Header.NameCount);

// Table1/Table2 seem to share same struct
FSeek(Header.Table1_Offset + 0x20);
if (Header.Table1_FileCount > 0)
    PACK_FileTable Table1(Header.Table1_FileCount);

FSeek(Header.Table2_Offset + 0x28);
if (Header.Table2_FileCount > 0)
    PACK_FileTable Table2(Header.Table2_FileCount);

Data field of each entry should contain the full BXON for that entry, seems some PACKs contain some extra data that isn't handled by this though, not sure why that is yet, looks like it might be something to do with the UnkData_Offset field.
Maybe that data is pointed at by the BXON themselves, similar to the info.arc BXON pointing to offsets in other arc files? (my guess is maybe this is data shared by multiple files in the PACK?)

E2: If anyone wants to dig in the game EXE for any info, it seems the EXE uses hashes instead of struct names, eg. EXE has no mentions of "tpXonAssetHeader" anywhere, but the bytes nearby (that I guess are a hash) "F0 29 FB 38" do have a mention in there.

E3: also made 010 editor template for parsing decompressed info.arc, thanks to LinkOFF & DKDave's info:

Code: Select all

struct BXON_Header
{
    local long header_pos = FTell();

    DWORD Magic;
    DWORD Version; // 3 in Nier, 1 in DQXIS

    DWORD StructNameHash;
    DWORD StructNameOffset;
    DWORD StructDataOffset;

    FSeek(header_pos + 0xC + StructNameOffset);
    string StructName;
};

enum<BYTE> CompressionType
{
    Solid = 0,
    SeperateBlobs1 = 1,
    SeperateBlobs2 = 2, // dunno what difference is
};

typedef struct
{
    local long archive_pos = FTell();

    DWORD NameOffset;
    DWORD Unk4; // always 4?
    CompressionType Flags;

    // pad bytes always set to 0x40?
    BYTE Pad9;
    BYTE PadA;
    BYTE PadB;

    local long archive_end_pos = FTell();

    FSeek(archive_pos + 0 + NameOffset);
    string Name;

    FSeek(archive_end_pos);
} BXON_tpArchiveFileParam_Archive<read=ReadBXON_tpArchiveFileParam_Archive>;

typedef struct
{
    local long file_pos = FTell();

    DWORD Unk0;
    DWORD NameOffset;
    DWORD FileOffset<read=ReadFileOffset>; // offset needs to be multiplied by 0x10 always!

    DWORD CompressedLength;
    DWORD UncompressedHeadersLength; // size of PACK headers/data
    DWORD UncompressedDataLength; // seems to be the size of UnkData part of PACK?

    BYTE ArchiveIdx;
    BYTE Flags;

    // pad bytes always set to 0x40?
    BYTE Pad1A;
    BYTE Pad1B;

    local long file_end_pos = FTell();

    FSeek(file_pos + 0x4 + NameOffset);
    string Name;

    FSeek(file_end_pos);
} BXON_tpArchiveFileParam_File<read=ReadBXON_tpArchiveFileParam_File>;

// multiplies offset by 0x10 for template window view
string ReadFileOffset(DWORD offset)
{
    local uint64 offs = offset;
    offs = offs * 0x10;
    string s;
    SPrintf(s, "%Lu", offs);
    return s;
}

string ReadBXON_tpArchiveFileParam_Archive(BXON_tpArchiveFileParam_Archive& entry)
{
    return entry.Name;
}

string ReadBXON_tpArchiveFileParam_File(BXON_tpArchiveFileParam_File& entry)
{
    string s;
    SPrintf(s, "%s:/%s", Archive.Archives[entry.ArchiveIdx].Name, entry.Name);
    return s;
}

// Seperate struct so files can be collapsed in template window...
struct BXON_tpArchiveFileParam_FileTable(int FileCount)
{
    BXON_tpArchiveFileParam_File Files[FileCount] <optimize=false>;
};

struct BXON_tpArchiveFileParam
{
    local long archive_param_pos = FTell();

    DWORD ArchiveCount;
    DWORD ArchiveTableOffset;
    DWORD FileCount;
    DWORD FileTableOffset;

    FSeek(archive_param_pos + 0x4 + ArchiveTableOffset);
    BXON_tpArchiveFileParam_Archive Archives[ArchiveCount] <optimize=false>;

    FSeek(archive_param_pos + 0xC + FileTableOffset);
    BXON_tpArchiveFileParam_FileTable Files(FileCount);
};

local long pos = FTell();

BXON_Header Header;

if(Header.StructName == "tpArchiveFileParam")
{
    FSeek(pos + 0x10 + Header.StructDataOffset);
    BXON_tpArchiveFileParam Archive;
}

[Ekey]

https://github.com/yretenai/kaine

[LinkOFF]

Unpacker - https://disk.yandex.ru/d/TqBu-IUq7Ku88Q
Drag&drop info.arc

[DaniKH]

https://github.com/yretenai/kaine

That's great! Works perfectly fine! Any idea about when the meshes can be exportable?