Important information: this site is currently scheduled to go offline indefinitely by end of the year.

2022 - Tom Clancy's Rainbow Six: Siege Asset Extraction Tools

Post questions about game models here, or help out others!
Custard
advanced
Posts: 65
Joined: Fri Oct 25, 2019 12:33 am
Has thanked: 11 times
Been thanked: 14 times

Re: Rainbow Six: Siege Models Thread

Post by Custard »

I'm not making any more progress with this. I spent a while researching the subject and doing tests, but even Intel themselves say that the program doesn't store UV data (albeit information a few years old). I'm going to put up the method I used here as it's been requested, maybe someone else can take it further, and sometimes just getting meshes alone could be helpful (I've seen people that want to 3d print Siege miscellania, etc).

I use the latest version of Intel GPA directly from the official source. There is very little information about using it for ripping assets out there that I've found, but the few write-ups I've come across refer to using an old version of the program, just to clear that up to begin with.

Battleye seems to complicate launching so I use the "Auto-detect launched applications" setting to attach the GPA monitor to the game. The game has never connected to the online service when I've been attaching GPA to it, again I think Battleye having a fuss and not allowing it, though it won't incur penalty my account has been fine since. Maybe that was just Ubi's servers being crap.

Take a frame in game, and go open it in the frame analyser. Along the top is a graph showing the various calls that's going on in the frame, that's where I navigate and browse for what I'm looking for. I sort it by primitive count I think (I'm writing this by memory), geometry tends to be heaviest thing so the bigger the register on the graph the more likely it's a proper detailed mesh object, but actually some things like scopes have been quite small ones. There's a lot of calls that just aren't going to be of interest, for lighting and crap I don't really know. In the lower center is the specific display, and at the top of that section you should be able to change between viewing different types of assets for the particular call that you are viewing; e.g. texture, mesh.

There seems to be a bug in the display of geometry, that prevents freely browsing all that is there in a frame without reloading it. If I go through looking at the geometry items I'll see glitchy broken stuff that seems to be mainly the same after I've clicked on a few draw calls — just a couple of disjointed planes — but the textures hint at what the asset might actually be. So I browse by looking at textures instead to find the thing I want — a scope or gun or whatnot will have a recognizable texture — then I remember what draw call number that was, close the frame, the re-open it and click on that specific draw call first and it will show the right geometry. It seems to be the act of clicking through different draw calls that breaks geometry display, the first one you click on works right. Literally a bug with the program it seems, unless I'm missing something.

Also, so far any frame I take doesn't come close to containing draw calls for all the assets that are present in the actual viewed render. Like if I stand in a map looking at walls and props, I'll never find draw calls for any of that stuff. If I try taking a frame in the menu where a character or weapon is being centrally displayed then I'll never find draw calls for most of it. If I load into the game and take a frame then nearly all the provisions of that frame will be focused something specific, which tends to be what was in the characters hands, which is how I'm able to get equipment meshes. The architecture of frame capturing is beyond me, there is probably some analog of baking occuring for static objects and physics objects that means they exist only as a reference call in the frame.. or something, just a guess.

So that's how I can get some meshes, anything I can hold in the character's hands will show up reliably in the frame capture. There's a button right there in the window to export meshes, textures can be got the same way. It will be a distorted mesh and I then attach a cube to it with a mesh deform modifier in Blender and correct the shape of the mesh using orthographic views, doesn't take much time but requires guesswork as to setting the dimensions. Look for parts of the object that are suppose to be circular (such as the end of the barrel or screws and bolts, then push and pull until those are circular and that does most of the work for you.

Better than nothing at all but yeah.. :(
banlu
n00b
Posts: 11
Joined: Fri Nov 20, 2015 10:15 pm
Has thanked: 14 times

Re: Rainbow Six: Siege Models Thread

Post by banlu »

Custard wrote: Tue Dec 31, 2019 4:40 pm I'm not making any more progress with this. I spent a while researching the subject and doing tests, but even Intel themselves say that the program doesn't store UV data (albeit information a few years old). I'm going to put up the method I used here as it's been requested, maybe someone else can take it further, and sometimes just getting meshes alone could be helpful (I've seen people that want to 3d print Siege miscellania, etc).

I use the latest version of Intel GPA directly from the official source. There is very little information about using it for ripping assets out there that I've found, but the few write-ups I've come across refer to using an old version of the program, just to clear that up to begin with.

Battleye seems to complicate launching so I use the "Auto-detect launched applications" setting to attach the GPA monitor to the game. The game has never connected to the online service when I've been attaching GPA to it, again I think Battleye having a fuss and not allowing it, though it won't incur penalty my account has been fine since. Maybe that was just Ubi's servers being crap.

Take a frame in game, and go open it in the frame analyser. Along the top is a graph showing the various calls that's going on in the frame, that's where I navigate and browse for what I'm looking for. I sort it by primitive count I think (I'm writing this by memory), geometry tends to be heaviest thing so the bigger the register on the graph the more likely it's a proper detailed mesh object, but actually some things like scopes have been quite small ones. There's a lot of calls that just aren't going to be of interest, for lighting and crap I don't really know. In the lower center is the specific display, and at the top of that section you should be able to change between viewing different types of assets for the particular call that you are viewing; e.g. texture, mesh.

There seems to be a bug in the display of geometry, that prevents freely browsing all that is there in a frame without reloading it. If I go through looking at the geometry items I'll see glitchy broken stuff that seems to be mainly the same after I've clicked on a few draw calls — just a couple of disjointed planes — but the textures hint at what the asset might actually be. So I browse by looking at textures instead to find the thing I want — a scope or gun or whatnot will have a recognizable texture — then I remember what draw call number that was, close the frame, the re-open it and click on that specific draw call first and it will show the right geometry. It seems to be the act of clicking through different draw calls that breaks geometry display, the first one you click on works right. Literally a bug with the program it seems, unless I'm missing something.

Also, so far any frame I take doesn't come close to containing draw calls for all the assets that are present in the actual viewed render. Like if I stand in a map looking at walls and props, I'll never find draw calls for any of that stuff. If I try taking a frame in the menu where a character or weapon is being centrally displayed then I'll never find draw calls for most of it. If I load into the game and take a frame then nearly all the provisions of that frame will be focused something specific, which tends to be what was in the characters hands, which is how I'm able to get equipment meshes. The architecture of frame capturing is beyond me, there is probably some analog of baking occuring for static objects and physics objects that means they exist only as a reference call in the frame.. or something, just a guess.

So that's how I can get some meshes, anything I can hold in the character's hands will show up reliably in the frame capture. There's a button right there in the window to export meshes, textures can be got the same way. It will be a distorted mesh and I then attach a cube to it with a mesh deform modifier in Blender and correct the shape of the mesh using orthographic views, doesn't take much time but requires guesswork as to setting the dimensions. Look for parts of the object that are suppose to be circular (such as the end of the barrel or screws and bolts, then push and pull until those are circular and that does most of the work for you.

Better than nothing at all but yeah.. :(
thank you very much for sharing it custard !
User avatar
MaZTeR
mega-veteran
mega-veteran
Posts: 248
Joined: Sat Jul 09, 2016 4:06 pm
Location: Finland
Has thanked: 6 times
Been thanked: 12 times

Re: Rainbow Six: Siege Models Thread

Post by MaZTeR »

Tushkan wrote: Sat Dec 07, 2019 2:22 am Oh, yeah, about your question. At this stage I can export all meshes, all textures, but there are literally tons of them. I can also sort them into folders containing the stuff that is linked into one asset. I still can't get the skeletons linked to those assets though.
Wait, so you can get everything but skeletons and skin data? Is the tool you use quite simple to use (like id-daemon's GR: Wildlands tool) or does it require hex editing or some advanced knowledge? I really don't care about the skeletons for now because you can always make new ones.
Tushkan
veteran
Posts: 106
Joined: Mon Dec 18, 2017 1:47 pm
Has thanked: 2 times
Been thanked: 39 times

Re: Rainbow Six: Siege Models Thread

Post by Tushkan »

The tool I use is a python 3 module written to deal with forge archives. It has tools to unpack .forge, to convert textures and to encode mesh data (partially). All this stuff has a bunch of dependencies.
1. Basic .forge manipulation:
zstandard to unpack compressed data
2. textures:
RawTex do assemble proper dds from .forge (they use custom headers that I need to repack into a common dds)
texconv to convert file to png
3. meshes:
blender

3rd module is not polished and can't be used easily. I have a bunch of dirty scripts embedded into a blend file to convert mesh's generic output to a proper blender model. Plus I tendo to unpack forges directly from blender so I had to import zstandard binary into blender's libs folder.
My main goal is to get localization data and to get how assets are formed. From that I can start getting full model packages and investigate the data mode in-depth.
Rainbow 6 Siege forge unpacking tool:
https://www.dropbox.com/sh/b2cuse4hp90y ... qTfja?dl=0
Ronininja
ultra-n00b
Posts: 2
Joined: Wed Jan 29, 2020 8:50 am

Re: Rainbow Six: Siege Models Thread

Post by Ronininja »

Tushkan wrote: Tue Jan 28, 2020 11:54 pm The tool I use is a python 3 module written to deal with forge archives. It has tools to unpack .forge, to convert textures and to encode mesh data (partially). All this stuff has a bunch of dependencies.
1. Basic .forge manipulation:
zstandard to unpack compressed data
2. textures:
RawTex do assemble proper dds from .forge (they use custom headers that I need to repack into a common dds)
texconv to convert file to png
3. meshes:
blender

3rd module is not polished and can't be used easily. I have a bunch of dirty scripts embedded into a blend file to convert mesh's generic output to a proper blender model. Plus I tendo to unpack forges directly from blender so I had to import zstandard binary into blender's libs folder.
My main goal is to get localization data and to get how assets are formed. From that I can start getting full model packages and investigate the data mode in-depth.
Do you plan on uploading the tool?
Tushkan
veteran
Posts: 106
Joined: Mon Dec 18, 2017 1:47 pm
Has thanked: 2 times
Been thanked: 39 times

Re: Rainbow Six: Siege Models Thread

Post by Tushkan »

Fml... All this time I was wondering where do they keep strings to identify models. Turns out they where stored in metadata header which I was mindlessly skipping in meshes. And same goes for texture data.

Yes, Ronininja, I do plan on releasing it. However, it's not going to be a tidy and easy to use tool. I will try to provide some tutorials plus a documentation that explains all the structures and pieces of format so anyone who is versed in writing standalone apps can write a complete tool to grab any needed data. I also suspect that it might get banned by ubisoft, or my account on github\whatever will get suspended as they did with luxox or (even whorse) they will change their file format again which will render my tool mostly useless. So it should be released as a more or less complete working package at once so everyone can rip the stuff they need while they can.
Also, can anyone suggest a place to host it's code where it's unlikely to get suspended?
Rainbow 6 Siege forge unpacking tool:
https://www.dropbox.com/sh/b2cuse4hp90y ... qTfja?dl=0
Ronininja
ultra-n00b
Posts: 2
Joined: Wed Jan 29, 2020 8:50 am

Re: Rainbow Six: Siege Models Thread

Post by Ronininja »

Maybe you can create a torrent and upload it to a tracker?
Custard
advanced
Posts: 65
Joined: Fri Oct 25, 2019 12:33 am
Has thanked: 11 times
Been thanked: 14 times

Re: Rainbow Six: Siege Models Thread

Post by Custard »

Tushkan wrote: Thu Jan 30, 2020 1:17 am Fml... All this time I was wondering where do they keep strings to identify models. Turns out they where stored in metadata header which I was mindlessly skipping in meshes. And same goes for texture data.
Sounds like a major breakthrough, will that be enough to allow indexing assets?
Yes, Ronininja, I do plan on releasing it. However, it's not going to be a tidy and easy to use tool. I will try to provide some tutorials plus a documentation that explains all the structures and pieces of format so anyone who is versed in writing standalone apps can write a complete tool to grab any needed data. I also suspect that it might get banned by ubisoft, or my account on github\whatever will get suspended as they did with luxox or (even whorse) they will change their file format again which will render my tool mostly useless. So it should be released as a more or less complete working package at once so everyone can rip the stuff they need while they can.
Also, can anyone suggest a place to host it's code where it's unlikely to get suspended?
I've noticed with some other ripping tools they only distribute privately among people that seem involved and helpful, who will make the assets available in bulk / take requests / help develop the tool. That way the game gets ripped but the tool is less likely to end up under a developer's eye, improving the chance it doesn't get bricked. Siege is riding high on the whole games-as-a-service thing right now. Even if a bulk rip gets pulled off if the methods don't last we'll miss out on juicy future content; New ops, skins, weapons, etc.

If you can make a torrent it won't get taken down but it'll be reliant on seeding, which for something this niche may likely be an issue. I've seen a lot of blatantly-illegal stuff hosted on Mega.nz. I expect that site still does takedowns, but if you make a throwaway account your files may slip through the cracks for ages there. If a complete tool gets made then p3dm.ru hosts such programs with pretty much no chance of a takedown.
comword
n00b
Posts: 12
Joined: Thu Nov 22, 2018 10:38 am

Re: Rainbow Six: Siege Models Thread

Post by comword »

Hello,
I am new to this forum and feel really happy to see a breakthrough here.
You can upload your scripts to IPFS. It's distributed and thus can't be censored.
I want some zombie models from the operation Chimera, but it was ended in Apr 2018. I have got an older version game, and related forge files downloaded, but I don't know how did you make breakthrough guessing file structures. Thanks in advance.
User avatar
MaZTeR
mega-veteran
mega-veteran
Posts: 248
Joined: Sat Jul 09, 2016 4:06 pm
Location: Finland
Has thanked: 6 times
Been thanked: 12 times

Re: Rainbow Six: Siege Models Thread

Post by MaZTeR »

Tushkan wrote: Thu Jan 30, 2020 1:17 am Fml... All this time I was wondering where do they keep strings to identify models. Turns out they where stored in metadata header which I was mindlessly skipping in meshes. And same goes for texture data.

Yes, Ronininja, I do plan on releasing it. However, it's not going to be a tidy and easy to use tool. I will try to provide some tutorials plus a documentation that explains all the structures and pieces of format so anyone who is versed in writing standalone apps can write a complete tool to grab any needed data. I also suspect that it might get banned by ubisoft, or my account on github\whatever will get suspended as they did with luxox or (even whorse) they will change their file format again which will render my tool mostly useless. So it should be released as a more or less complete working package at once so everyone can rip the stuff they need while they can.
Also, can anyone suggest a place to host it's code where it's unlikely to get suspended?
The only reason Luxos really has been banned is because he was (or is) accepting paid commissions to get stuff out of the game. If I remember correctly, some Ubisoft employe stated they don't care that their models are used as long as they're not directly profited from, like reselling them on some chinese pirate site.
User avatar
zaramot
double-veteran
double-veteran
Posts: 783
Joined: Wed Jan 05, 2011 12:41 pm
Has thanked: 39 times
Been thanked: 855 times

Re: Rainbow Six: Siege Models Thread

Post by zaramot »

Making model-import scripts, PM
Tushkan
veteran
Posts: 106
Joined: Mon Dec 18, 2017 1:47 pm
Has thanked: 2 times
Been thanked: 39 times

Re: Rainbow Six: Siege Models Thread

Post by Tushkan »

zaramot, did you get .dat files from forge or did you grab them from game memory? If first, how did you locate all Ash related pieces?
Rainbow 6 Siege forge unpacking tool:
https://www.dropbox.com/sh/b2cuse4hp90y ... qTfja?dl=0
User avatar
zaramot
double-veteran
double-veteran
Posts: 783
Joined: Wed Jan 05, 2011 12:41 pm
Has thanked: 39 times
Been thanked: 855 times

Re: Rainbow Six: Siege Models Thread

Post by zaramot »

Tushkan wrote: Mon Feb 03, 2020 9:44 pm zaramot, did you get .dat files from forge or did you grab them from game memory? If first, how did you locate all Ash related pieces?
I got .dat from .forge archives, with quickbms script. Just a luck, I don't know name of this character, I never played the game (just some random female) - parts were in same .forge.
Making model-import scripts, PM
Tushkan
veteran
Posts: 106
Joined: Mon Dec 18, 2017 1:47 pm
Has thanked: 2 times
Been thanked: 39 times

Re: Rainbow Six: Siege Models Thread

Post by Tushkan »

O_o could you elaborate a bit more on your process? That seems very odd, considering the fact that you managed to get all parts of a character that should be scattered randomly among an entire .forge archive if not even a couple of them. Did you gather a list of needed pieces from ondemand.forge? What was vertex, normals and skinning weights format? (they differ from model to model, there are about 3 levels of compression plus 2 storage types: array vertex structs or bunch of buffer arrays per-pos, per-norm etc)? I'm particularly interested in skinning weights, I couldn't yet locate that data. Also, maybe you were lucky enough to find her skeleton data?
Rainbow 6 Siege forge unpacking tool:
https://www.dropbox.com/sh/b2cuse4hp90y ... qTfja?dl=0
User avatar
zaramot
double-veteran
double-veteran
Posts: 783
Joined: Wed Jan 05, 2011 12:41 pm
Has thanked: 39 times
Been thanked: 855 times

Re: Rainbow Six: Siege Models Thread

Post by zaramot »

Tushkan wrote: Tue Feb 04, 2020 3:44 pm O_o could you elaborate a bit more on your process? That seems very odd, considering the fact that you managed to get all parts of a character that should be scattered randomly among an entire .forge archive if not even a couple of them. Did you gather a list of needed pieces from ondemand.forge? What was vertex, normals and skinning weights format? (they differ from model to model, there are about 3 levels of compression plus 2 storage types: array vertex structs or bunch of buffer arrays per-pos, per-norm etc)? I'm particularly interested in skinning weights, I couldn't yet locate that data. Also, maybe you were lucky enough to find her skeleton data?
Don't know, just a luck - there was only 8 skinned meshes (and ton of static ones) in that particular .forge, all referenced to her (as you said, I never saw this character before). My process of extraction is grouping files by type (meshes, textures etc). So, I know, that all meshes from certain .forge are in model folder, then I'm running another script (just for more faster process, it's not really necessary) which is telling me which one is skeletal and which one is static, then I just importing (checking) all skeletal and that's basically all lol. Skinning was pretty simple 4 bone weights - 4 bone indices, I haven't found skeleton - but it should't be hard, honestly. I assume they are using two sets of base skeleton for male and female like a lot of Ubisoft games. Like here for For Honor (a bit similar to Rainbow Six btw, wildlands + all of AC games are quite different)

https://www.youtube.com/watch?v=GergFXnfy2g
Making model-import scripts, PM
Post Reply