Spider Man 3 XBOX 360

[PaHaNchickT]

Hi everyone!

There are different topics that solved two problems: how to extract PCPACK archives from Spider Man 3 and Spider Man: Web Of Shadows on PC. But there are no topics with description of how to extract XEPACK files. It's the same files but from XBOX360 game directory. Also these files got similar structure.

I got some scripts from the internet and this one is working scipt for Noesis (made by Acewell):

Code: Select all

from inc_noesis import *

def registerNoesisTypes():
    handle = noesis.register("Spiderman 3 (PC)", ".PCPACK")
    noesis.setHandlerExtractArc(handle, pacExtractArc)
    #noesis.logPopup()
    return 1
    
def pacExtractArc(fileName, fileLen, justChecking):
    with open(fileName, "rb") as file:
        bs = NoeBitStream(file.read(fileLen))
    if justChecking:
        return 1
    bs.seek(0x54, NOESEEK_ABS)
    fileCount = bs.readUInt()
    tableLen = fileCount * 16
    TMP = bs.tell()
    myString = bs.readBytes(4500) 
    myIndex = myString.find(b"\xA1\xA1\xA1\xA1")
    bs.seek((TMP + myIndex) - tableLen, NOESEEK_ABS)
    for i in range(fileCount):
        basename = bs.readUInt()
        extension = bs.readUInt()
        offset = bs.readUInt()
        size = bs.readUInt()
        TMP2 = bs.tell()
        fileName = hex(basename) + "." + hex(extension)
        fileName = fileName.replace("0x", "") 
        bs.seek(offset, NOESEEK_ABS)
        rapi.exportArchiveFile(fileName, bs.readBytes(size))
        bs.seek(TMP2, NOESEEK_ABS)
    return 1

This script extracts uncompressed PCPACK files. I understood that in bs.seek it needs to be started from 0x57 in XEPACK files. But Noesis always gives me an error:

Code: Select all

Traceback (most recent call last):
  File "C:\Users\terno_000\Desktop\TOOLS\Noesis\plugins\python\extract_Spiderman3_XEPACK.py", line 30, in pacExtractArc
    bs.seek(offset, NOESEEK_ABS)
  File "C:\Users\terno_000\Desktop\TOOLS\Noesis\plugins\python\inc_noesis.py", line 152, in seek
    self.toUnpacker(); r = noeSuper(self).seek(addr, isRelative); self.fromUnpacker(); return r
  File "C:\Users\terno_000\Desktop\TOOLS\Noesis\plugins\python\inc_noesis.py", line 148, in fromUnpacker
    self.setOffset(self.byteOfs)
  File "C:\Users\terno_000\Desktop\TOOLS\Noesis\plugins\python\inc_noesis.py", line 77, in setOffset
    noesis.bsSetOfs(self.h, ofs)
OverflowError: Python int too large to convert to C long
Detected file type: Unknown
Nothing was exported!
Cleaned 22.82MB (in 2 pools).

I understand that the problem is in this line:

Code: Select all

myString = bs.readBytes(4500)

I need to calculate how many bytes should be here. But I can't understand how.. Could anyone help me please?

p.s. there are samples of the same files in different formats (*PCPACK and *XEPACK)
https://mega.nz/folder/pUF01JjZ#yJKvR3S45Qk1yB3X4Hkg5A

[DKDave]

The reason it doesn't work is because the XBox 360 files use Big Endian format for storing numbers, so the script as it is will only work on PC.

Assuming the file structure is identical, you should be able to change the following line and see if it works:

bs = NoeBitStream(file.read(fileLen))

Change to: bs = NoeBitStream(file.read(fileLen), NOE_BIGENDIAN)

[PaHaNchickT]

The reason it doesn't work is because the XBox 360 files use Big Endian format for storing numbers, so the script as it is will only work on PC.

Assuming the file structure is identical, you should be able to change the following line and see if it works:

bs = NoeBitStream(file.read(fileLen))

Change to: bs = NoeBitStream(file.read(fileLen), NOE_BIGENDIAN)

Thanks for the answer!

A tried to change this line and now I got another error:

Code: Select all

Traceback (most recent call last):
  File "C:\Users\terno_000\Desktop\TOOLS\Noesis\plugins\python\extract_Spiderman3_XEPACK.py", line 21, in pacExtractArc
    bs.seek((TMP + myIndex) - tableLen, NOESEEK_ABS)
  File "C:\Users\terno_000\Desktop\TOOLS\Noesis\plugins\python\inc_noesis.py", line 152, in seek
    self.toUnpacker(); r = noeSuper(self).seek(addr, isRelative); self.fromUnpacker(); return r
  File "C:\Users\terno_000\Desktop\TOOLS\Noesis\plugins\python\inc_noesis.py", line 148, in fromUnpacker
    self.setOffset(self.byteOfs)
  File "C:\Users\terno_000\Desktop\TOOLS\Noesis\plugins\python\inc_noesis.py", line 77, in setOffset
    noesis.bsSetOfs(self.h, ofs)
OverflowError: Python int too large to convert to C long
Detected file type: Unknown
Nothing was exported!
Cleaned 22.82MB (in 2 pools).

Got any ideas what the problem is?

[DKDave]

I'm not sure from looking at the script. I've rewrote a QuickBMS script using the same logic that should work for both file versions, if you want to give this a try:

s3.zip

[PaHaNchickT]

I'm not sure from looking at the script. I've rewrote a QuickBMS script using the same logic that should work for both file versions, if you want to give this a try:

s3.zip

OMG MAN YOU'RE AMAZING! It's really works!

Now we got important file with extension *54 that contains textures, meshes and other. When we extracting *PCPACK we got *36 important file. So these files (*36 from PC and *54 from XBOX) also similar. There is also script that works with *36 and doesn't works with *54.

Could you please try to upgrade this script too?

Here is samples of *36 file, *54 file and script for exracting *36 files:
https://mega.nz/folder/AQEiibwQ#HJmJwXHVOScsmdm9BncX5w

I'll be glad if you could help again

[DKDave]

Yeah, it looks like the original Python script had the extension in hex, so just a quick modification to my script:

s3.zip

[PaHaNchickT]

Yeah, it looks like the original Python script had the extension in hex, so just a quick modification to my script:


s3.zip

Hmm. I really got now *36 file instead *54, thanks! But classic 36-extractor script still doesn't extract this file. It's so odd

[PaHaNchickT]

Yeah, it looks like the original Python script had the extension in hex, so just a quick modification to my script:


s3.zip

So interesting.. *36 files from PC and XBOX seem to be similar, but all words and strings in XBOX file looks like.. Mirrored?
Do you know how to fix it?

[DKDave]

Yes, everything is mirrored because the XBox format is Big Endian, so numbers and strings are the other way around compared to the PC version. Any scripts that reads those files need to take into account the difference and read them accordingly.

If the format is identical, then it should be a case of just setting the endianness in Noesis (or whatever else is used).

[PaHaNchickT]

Yes, everything is mirrored because the XBox format is Big Endian, so numbers and strings are the other way around compared to the PC version. Any scripts that reads those files need to take into account the difference and read them accordingly.

If the format is identical, then it should be a case of just setting the endianness in Noesis (or whatever else is used).

Hmm.. How I understand I need to find script that transform big-endian to little-endian, isn't it?

[DKDave]

Yes... If the format is identical, then it might be enough just to set any scripts to read Big Endian instead of Little Endian on the PC version.

[PaHaNchickT]

Yes... If the format is identical, then it might be enough just to set any scripts to read Big Endian instead of Little Endian on the PC version.

Hello again!
So I found the way to swap endianness from big- to small-. But now *36 script-extractor says that offset if wrong:

Code: Select all

Wrong offset, so we try again!

Could you please check what the problem is?
Here are samples of swapped *36 file, unswapped *36 file and script for extraction:

https://mega.nz/folder/VElECQrC#qQMCsOjmP-iB1GYZ9bI5sQ

[DKDave]

You shouldn't need to change the file data at all - it looks like you've swapped every 4 bytes in the whole file, as all of the texts etc. are the wrong way round. This definitely won't work as Big Endian only applies when reading numbers; any strings should remain the same, although there are expections, such as some 4-byte strings may be read as numbers. And not all numbers are on 4-byte boundaries or consist of 4 bytes - e.g some are single bytes, some are 2 bytes, so any swapping will completely change the file.

It's only the script to extract them that should be changed. I think the script probably needs to be amended as it seems to be searching for only data in Little Endian format. And it looks like that script is only designed to extract textures too. So it probably needs a complete rewrite to get everything, unfortunately.

[PaHaNchickT]

You shouldn't need to change the file data at all - it looks like you've swapped every 4 bytes in the whole file, as all of the texts etc. are the wrong way round. This definitely won't work as Big Endian only applies when reading numbers; any strings should remain the same, although there are expections, such as some 4-byte strings may be read as numbers. And not all numbers are on 4-byte boundaries or consist of 4 bytes - e.g some are single bytes, some are 2 bytes, so any swapping will completely change the file.

It's only the script to extract them that should be changed. I think the script probably needs to be amended as it seems to be searching for only data in Little Endian format. And it looks like that script is only designed to extract textures too. So it probably needs a complete rewrite to get everything, unfortunately.

Oh, yeah, I see.. I swapped words also. Sorry, I'm little bit a nub in this, ahah
Okay, thanks for your tips in this theme and your patience!! Will try to change script for reading Big Endian instead of Little Endian.

Got one more problem. Tried again your script for extracting another *XEPACK file and it doesn't work... I will be so glad if you could check it

More different samples here:
https://mega.nz/folder/BEFUDDRQ#V_htxyUrzcBctpG2inCbWg

[DKDave]

Having looked at the other samples, some of them don't fit the logic of my script. Some of them have the file table starting in a slightly different place. There's no obvious offset to the start of this table because the logic used doesn't always work, so I'm stumped on how to calculate it accurately for every file.