Page 9 of 15

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Wed Oct 16, 2013 12:49 pm
by deepshit
redman wrote:i hope road not ends here because no more updates and without mip maps texture editor is only good for menu screens and hud.
would love to help but i'm no hacker.
Dageron's working on mipmaps. Be patient.

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Thu Oct 17, 2013 10:21 am
by Dageron
Now GTA V Console Texture Editor has support for PS3 mipmaps, working on Xbox360.
New version is still in development stage.
http://dageron.com/?p=5432&lang=en

Image

MipMaps won't be the only new feature in the new version. Friends, be patient waiting for it, please.
Updates for the next GTA V Console Texture Editor (version 1.4) will be serious, but there won’t be much updates after it’s release.

============

I have spent a lot of time for RAGE-games resource research. Soon I will publish some interesting articles on my site, they will clarify much RSC techniques. While my goal is only textures yet, that will help others, who have plans working with models and other RSC data.

Since I haven't got much time working on other resources, my current objective is to make a complete texture builder. I mean, a console application, which will create texture dictionary resource (*.xtd/*.ctd) from input *.dds files. That is completly different level in comparsion with simple texture replacement (which is available in GTA V Console Texture Editor).

Small problem is that GTAIV, MC:LA, RDR and GTA V *.xtd/*.ctd have a bit different structure (and resource container also), so we will need separate builders for each game, or just it will be necessary to define work mode manually.

============

Btw, just ignore "OpenIV developers" and their henchmen. Since they don't support console modding and never help anybody, what are the reasons of helping them and listening what they say? I think that no such. We should just ignore them. Let them work on the PC tools.

I hadn't mentioned previously, but there is yet another (much easier) way to create a simple texture convertor from any PC version of the game: IV/EFLC, MP3, or future GTA V PC version (I mean, *.wtd to *.xtd/*.ctd). That can be useful for easy mod conversion from PC to consoles. And, it possible to do with other resources - convert them from PC, because all memory paging relocation is already done, and RSC flags are calculated.

============

More info will be published on my site.
Stay tuned and follow the news.

Sorry for a lot of technical terminology, everything will be explained in articles.

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Thu Oct 17, 2013 11:03 pm
by kornto
I finally implemented the saving feature (but it rewrite the whole file each time) and made some GUI improvements.
You can edit a rpf inside a rpf, but just don't forget to save it before you exit.

There are still many GUI improvements that should be done. I try to balance the developing of new core features/improvement of existing features. But since I am the only one who works on this, and I don't have a lot of time, I prefer to focus on new features.

Now, as I said, the next thing that I am going to work on is resources.

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Fri Oct 18, 2013 9:54 am
by michalss
kornto wrote:I finally implemented the saving feature (but it rewrite the whole file each time) and made some GUI improvements.
You can edit a rpf inside a rpf, but just don't forget to save it before you exit.

There are still many GUI improvements that should be done. I try to balance the developing of new core features/improvement of existing features. But since I am the only one who works on this, and I don't have a lot of time, I prefer to focus on new features.

Now, as I said, the next thing that I am going to work on is resources.

Thank you mate, looks perfect :) I guess if you focus to new features it will much better then rewrite the whole core or gui in my opinion :)

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Sun Oct 20, 2013 8:14 am
by Dageron
I have written an article:
RSC resources in RAGE games - the introduction.
http://dageron.com/?page_id=5446&lang=en

It describes the general principles of RSC-resources internal structure and logic.
Information is mainly theoretical, but should help those who tried reverse engineering RAGE-based games in IDA Pro, and tool developers.

Most info is about GTA IV/MC:LA/MP3 RSC-resources.
There is also some about RDR and GTA V, but I don't know all details.

kornto
Hope that will help understanding some RSC-resource logic, especially about the memory paging. We need to define all bitfields in RSC7 flags.
Next it will be possible to describe some structures (for example, those which had been taken from previous games with minor changes), and make a builder :) .
I may describe structures, but need explanation with bitfields.

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Sun Oct 20, 2013 11:24 am
by redman
i'm very thankful that you two still on it. spaziwa :keke:

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Sun Oct 20, 2013 2:14 pm
by listener
@Dageron
Looks like, you forgot to add link to the original post:
http://sannybuilder.com/forums/viewtopi ... 5850#p5850

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Sun Oct 20, 2013 4:39 pm
by Dageron
Nope, only parts of info you have published on SB3 forum in 2008/2009 were taken into account. All info we have access to, will be used. If you don't like that any researches from SB3 forum (ie yours) are used in someone else research/developments, then you may delete it (and it would be never used again).

And I can tell you the reasons why such info collecting have been done: if someone is really interested in spehere, he will have a headache because of necessity of reading about hundreds of pages on forums.gtamodding.ru, sannybuilder.com and gtaforums.com, cathing useful info from pieces (and, сonsidering that the first two ones are Russian-speaking, this attempt will fail). You don't want to document data and give people ability to research something themselves using ready info, but I want.

Now only lazy will be unable to extract AES keys, read RPF archives, uncompress RSC, untile textures, etc.
You know better than me: everything this is 1% in comparison with the whole panorama of RSC-resource structures.

p.s. Btw, I can't understand your (and others from OpenIV) interest for this thread, since .black project and related developments are closed for public (ie useless, let's call a spade a spade). Everything discussed here is guided by a different approach.

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Sun Oct 20, 2013 5:54 pm
by kornto
Thanks for that info!

I just want to fix a little mistake that I made. I saw in the binary a reference to RSC5 ("52 53 43 35") and not RSC7.
If in GTA IV/.. it is 52 53 43 05 and in RDR it is 52 53 43 85, so it is probably only by coincidence a printable "5".
So it is better to call it RSC35. (I will change it in my tool soon). I will also change the endianity of the magic. (Which is wrong right now)

Right now I am working on the system part of the resources. I am working on a library that will be able to parse/rebuild the structures. (So it will be easier to write things that will handle and edit those structures)
And since I switched to git now, I can work on multiple features at once. So soon there will be a branch for resources.

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Sun Oct 20, 2013 6:33 pm
by ThanatosSelisen
thx for all the work you guys put into all your tools =)

just 2 quick questions.

will there be any chance to be able to open .cdr files in the near future? or is there allready something to view. cant get it to work somehow ^^

is there another location to dl Zone (DAT) / Train Paths (IPL) Importer (3DMax) / tank?

thx in advance =)

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Sun Oct 20, 2013 6:54 pm
by Dageron
ThanatosSelisen
Don't know about others, but I haven't looked on that things yet.
(busy in yet another sphere...)

kornto
Good news!) Hope that flags bitfield documentation won't be hard.

And, also, just one thing... I think that we have been mistaken a bit when thought about RSC exporter/importer from *.rpf. So, GTA V RSC-resources exported "as is" haven't got header and the decision was to make header ourselves, just write signature and flag values from RPF directory. But (important!) we shouldn't write this header before uncompressed data block, only before compressed. Writing RSC header before uncompressed data is wrong from a conceptual point of view, because flags has special bit which defines is resource compressed or not.

I think that only two modes are be suitable:
  • RSC 7 (or 5/35): header (signature + flags) + compressed data
  • *.sys + *.gfx + *.hdr: CPU uncompressed segment + GPU uncompressed segment + header (ie signature and flags)
Import implementation for both modes won't be hard.
I think there is no necessity of RSC-resource exporting with "broken" header as it is stored in *.rpf, while exporting with normal header is quite useful (but data should be compressed).

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Sun Oct 20, 2013 7:04 pm
by kornto
Dageron wrote:ThanatosSelisen
Don't know about others, but I haven't looked on that things yet.
(busy in yet another sphere...)

kornto
Good news!) Hope that flags bitfield documentation won't be hard.

And, also, just one thing... I think that we have been mistaken a but when thought about RSC exporter/importer from *.rpf. So, GTA V RSC-resources exported "as is" hasn't got header and the decision was to make header ourselves, just write signature and flag values from RPF directory. But (important!) we shouldn't write this header before uncompressed data block, only before compressed. Writing RSC header before uncompressed data is wrong from a conceptual point of view, because flags has special bit which define is resource compressed or not.

I think that only two modes are be suitable:
  • RSC 7 (or 5/35): header (signature + flags) + compressed data
  • *.sys + *.gfx + *.hdr: CPU uncompressed segment + GPU uncompressed segment + header (ie signature and flags)
Import implementation for both modes won't be hard.
I think there is no necessity of RSC-resource exporting with "broken" header as it is stored in *.rpf, while exporting with normal header is quite useful (but data should be compressed).
I don't think that it is the case here. All the resources are compressed. Some of them are encrypted, and that is decided by the code that request that resource. (At least that is the way that I understood it, but maybe I am wrong). All the bits of the flag are related to size calculation. (If you take a look at the calculation, there are 28 bits are used for system/graphics size, and 8 more bits for the version). I don't think that there are more missed flags.

So you don't really have a way to know if the resource is encrypted or not only from the container. (I decide it by the extension)

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Sun Oct 20, 2013 7:16 pm
by Dageron
Well, let it be as in the current LibertyV. If there is no compression bit in flags, there is no the case.
I just thought with the comparsion how it was in previous R* games.

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Sun Oct 20, 2013 10:49 pm
by kilian277
Hi,

a friend has also been reasearching the files of gta 5 and already did some extracting of some text based files.
You can find some research here http://gtaforums.com/topic/594021-gta5- ... 1063409020

He found a file that could possibly be a object placement file , he wrote a small 3ds max importer for it and got this

Image


If anyone is interested you can contact me via pm and i will further contact him.

Kilian.

Re: [PS3/ X360] GTA V resource research & software developme

Posted: Mon Oct 21, 2013 11:37 am
by listener
Dageron wrote:Nope, only parts of info you have published on SB3 forum in 2008/2009 were taken into account. All info we have access to, will be used. If you don't like that any researches from SB3 forum (ie yours) are used in someone else research/developments, then you may delete it (and it would be never used again).
Let's take it step-by-step.
1) You says: "Since they don't support console modding and never help anybody, what are the reasons of helping them and listening what they say? I think that no such. We should just ignore them. Let them work on the PC tools."
2) You take my texts (and code), remove all credits, add some bugs and post it as yours
3) You suggest that I should remove my posts

Man, you are really funny clown ...
Dageron wrote: ...
Now only lazy will be unable to extract AES keys, read RPF archives, uncompress RSC, untile textures, etc.
You know better than me: everything this is 1% in comparison with the whole panorama of RSC-resource structures.
I'm doubting in a usefulness of this approach. Ok, everyone can read RPF and view textures. But, who will edit clothes and wrinkles, make blendshapes and pose transitions? A.S.Pushkin ?

Digging thousands of pages, searching hundreds of books, collecting potentially useful info bit-by-bit - gives a momentum for making steps further.
Dageron wrote:p.s. Btw, I can't understand your (and others from OpenIV) interest for this thread, since .black project and related developments are closed for public (ie useless, let's call a spade a spade). Everything discussed here is guided by a different approach.
It's simple: working on the map editor for V is hard, so, sometimes, I need more motivation. Reading this topic, give me a feeling of supremacy enough to move on.