XeNTaX

I have a tool called 'Rebuild File Lists' which takes all file lists and writes new ones out based on the archives. So yes, it can do that.

Good to hear that, Rick!
I uploaded some more filenames to Pastebin: http://pastebin.com/JqNyX3tA
Now _bnk_audio_1.tor, _bnk_audiodata_1.tor and _bnk_location_1.tor are complete.

SCPT file specification (version 5)
I was looking at the script files (header: SCPT, path: /resources/systemgenerated/compilednative/) and what a surprise, the format has completely changed since I posted the specification on page 1 of this thread. The current version is now 5, compared to the old version 4 I wrote about.

It seems like the ELF script files have been abandoned and a custom script format is now used. The new script files start with D1 03 but I have not looked into them very much. Anyway, here's the format of the new SCPT files, but without the last part that contains the script files:

File header:
0-3: 4 bytes, file header: SCPT (53 43 50 54)
4-7: file version as two 16-bit integers: 05 00 05 00 → version 5
8-11: Unknown 32-bit integer (little-endian), always 1: 01 00 00 00
12-15: four zero bytes: 00 00 00 00
16-21: six bytes, possibly a CRC checksum, different for each file
22-23: two bytes, end of header: 00 D0

Script Name:
24: Variable-length integer specifying length of script name (all version 5 files no longer contain a script name, i.e. this byte is always 00)
string with specified length, if it exists
25-32: eight zero bytes
33-36: 32-bit integer, length of file, offset starting after this length, excluding the final 00

Strings without id:
37: Variable-length integer specifying number of strings without an id
LOOP (for each string) {
→ Variable-length integer specifying string length
→ String with the specified length
} END LOOP

Strings with id:
Variable-length integer specifying number of strings with an id
LOOP (for each string) {
→ Variable-length integer specifying string id
→ Variable-length integer specifying string length
→ String with the specified length
} END LOOP

Script file:
Beginning of script file, starting with D1 03
Final byte: 00

Example
For example, the script file /resources/systemgenerated/compilednative/14988250124449474902 contains the following strings:

Strings without id: datecreated, datelastmodified, lastread, ticketstatus, summary, fulltickettext
Strings with id: MAX_UPDATE_LENGTH, SubmitNewTicket, UpdateTicketText, UpdateTicketRead, CloseTicket, OnRequestTicketsCB, _ParseTicketFields, OnRequestTicketCB, TrackLine, ConstructLookupListIterator, LookupListIteratorNext, PopUntilTemporaries, ConstructString, GetLookupListCurrentValueLookupList, PrepareLocalCall, UnprepareCall, RefLookupListElementROByStringString, ConvertFromStringInt, LookupListHasStringCopyValueString

It seems like the strings without id are just strings used by the script, while strings with id contain the function and variable names. However, I do not know enough about the script files to say this for sure.

Script file
As I said, I have not yet analysed the format of the script file; here are the starting bytes of one script files.
It seems like D1, D3 and bytes 00 to 06 are used very often.
Does anyone recognize this format by chance, or is it really a custom format?

D1 03 01 D1 00 02 D3 C9 09 C5 D3 05 D1 00 02 D3
D1 00 00 D3 D1 00 06 D3 D1 00 08 D1 00 02 D3 D1
00 08 D1 00 06 D3 D1 00 06 D3 D3 D3 D1 00 08 D1
00 06 D3 D1 00 06 D3 D3 06 02 00 01 04 02 03 00
01 04 00 04 02 02 00 01 04 00 02 00 01 04 03 08
00 01 04 04 03 00 03 00 03 00 03 02 03 02 03 02
03 00 01 04 00 04 04 01 06 04 01 00 00 00 00 00
00 C9 06 0F 01 00 03 01 10 C9 06 0F C3 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 83 EC 0C 8B
44 24 14 8B 00 89 44 24 04 8B 44 24 10 89 04 24
E8 FC FF FF FF 83 C4 0C C3 00 00 00 C3 00 00 00

Ñ..Ñ..ÓÉ.ÅÓ.Ñ..ÓÑ..ÓÑ..ÓÑ..Ñ..ÓÑ..Ñ..ÓÑ..ÓÓÓÑ..Ñ..ÓÑ..ÓÓ.........................................................É.......É..Ã...............ƒì.‹D$.‹.‰D$.‹D$.‰.$èüÿÿÿƒÄ.Ã...Ã...............ƒì.‹D$.‹.‹P.‹@.‰D$.‰T$.‰L$.‹D$.‰.$èüÿÿÿƒÄ.Ã.....Ã...............ƒì.‹D$.‹.‹@.‰D$.‰L$.‹D$.‰.$èüÿÿÿƒÄ.Ã............Ã...............ƒì.‹D$.‹.‹@.‰D$.‰L$.‹D$.‰.$èüÿÿÿƒÄ.Ã............USWVƒ

Here's the biggest of my file name updates so far, with more than 30,000 new file names.
Most of them are from swtor_main_gfx_1.tor (the item icons) but the archive is still not even near completion.

What percentage are we at now, Rick? Unfortunately, I have not built any statistics into my program yet.

SWTOR fan wrote:Here's the biggest of my file name updates so far, with more than 30,000 new file names.
Most of them are from swtor_main_gfx_1.tor (the item icons) but the archive is still not even near completion.

What percentage are we at now, Rick? Unfortunately, I have not built any statistics into my program yet.

Now at 299288/364794 (82%).

(Your list added 30876 new names to my lists.)

Rick wrote:(Your list added 30876 new names to my lists.)

That's strange, maybe you have a different version of the asset files than me. I am sure we will have that figured out when we got closer to 100%.

For now, here are nearly 15,000 new file names.

SWTOR fan wrote:That's strange, maybe you have a different version of the asset files than me. I am sure we will have that figured out when we got closer to 100%.

For now, here are nearly 15,000 new file names.

I'm adding new files all the time, it's possible that I added some that were in your lists since I posted the last update.

Hate to make a request of my first post, but I'm sure others will be asking soon. Slap me if I'm out of line here, but I haven't been able to find the latest EasyMYP that isn't on megaupload (currently locked down in legal proceedings).

Can we get a new link?

There is a working link in this post, the archive contains a special TOR build of EasyMYP along with a hash dictionary which has 44.5% of file names.

EDIT
New working link to the SWTOR build of EasyMYP 3.6 in this post.
You can get the latest compatible hash list in this post.
The latest *.gr2 model import plugin for noesis can be found in this post.

AceWell wrote:There is a working link in this post, the archive contains a special TOR build of EasyMYP along with a hash dictionary which has 44.5% of file names.

I must've read past that couple of times and not seen it. Very sneaky. Many thanks.

I am new to this type of stuff, but have been really interested in all that is being said in this forum.

I see these lists of file names that are being posted fairly regularly. I saw a post saying that they are in a custom format that will not work with easyMYP. When I open the list of file names, it is just a list - how do I match those up to the actual files I have extracted? Specifically, I extracted all the .dds files in swtor_main_gfx_1.tor - now I would like to get those file names translated from just the numbers to something meaningful so I can start to match them up with items and abilities.

If anyone would be willing to point me in the right direction, I'd appreciate it!

This may not be how it was meant to go, but it works:

Download and unzip the 299288.zip posted by Rick (above), rename the .filelist file you would like to access to .txt then open the .tor archive with the corresponding name in EasyMYP. Go to the dictionary tab and do test full filename list and open the file you just gave the extension txt. That doesn't make any immediate change, but when you close MYP the hash list is saved.

Reopen that .tor file and it will have good names in the archive file list view.

Thank you! That worked.

It seems like many of you are veterans at doing this sort of stuff. Is there anyone here that would be willing to explain (or give me a beginner's understanding) HOW you figured out these file name hashes? I would love to be able to figure some of this stuff out on my own when updates to the game come out.

I still can't seem to figure out how to get the ability data from these .tor files using EasyMYP. I believe it's swtor_main_global_1.tor in the bucket files but I'm not sure how to read them. Any advice?

The context of this is I'm doing a school project and would like to have the ability data to create a database for myself.

BTW Totally appreciate the work you guys have done!

Any progress on completing the list? Mind you that Update 1.1 was released