WillowTree 2 / Discussion / Development: Save Game File Format (Hints)

da_fileserver - 2012-09-29

I have so far done some extensive Borderlands 2 save game format reverse engineering. However, I do not wish to release this information publicly at present, since I feel that most people will use it to cheat (and I do not want this while the game is still so new). I will give some hints though:

The whole file is hashed with SHA-1.

They do use LZO to compress it.

They are using huffman coding. (This is probably the most complex part of getting the raw data.)

The "raw" data is checked with CRC-32.

If you can get past that, they are using protobuf... (this isn't so easy to analyze, though the wire formats are easy to get)

The weapon and item data seems to be using an interesting packing method, so it's looking like it won't be as easy to modify as the first one. (I haven't done too much work reversing this right now.)

That should be enough for someone well versed in all these topics to figure it out from there.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Kyle873 - 2012-09-30

https://docs.google.com/spreadsheet/ccc?key=0Av9Yb4vQhMoTdEI3NFdZbjVkdllVcVRMSFF4V2dFQWc#gid=0

WIP Google doc of the save file format structure.

Info from Gibbed:

first layer is compressed LZO data
20 bytes sha1 of compressed data
uncompressed size, uint32, big endian
LZO compressed data

inner layer is
length, uint32, big endian
3 bytes, magic, "WSG"
version, uint32, little endian, 2
crc32 hash of uncompressed data, uint32, little endian
huffman compressed data

http://pastebin.com/jjtUkhS6
dump of decompressed huffman data

Last edit: Kyle873 2012-10-03

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Isthiriel - 2012-10-05

This is great! I've been writing some prototype code with the goal of helping to rebuild the .proto definitions but I'm stuck on the Huffman encoding :(

I can't work out how (where? if?) they've stored the Huffman tree in the (LZO-)decompressed data.

Any tips/source to share? :)

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

We've fully unpacked the weapon/item data, allowing us to associate part names to the various weapon fields - sample: http://pastebin.com/rFzEnRwE. A slightly out of date protocol specification is attached to the end of this message. Important subroutines used in various unpacking steps are identified with a signature.

.save file format

.sav:
 uint8_t[20]: Decompressed SHA1 hash
 uint32_t: LZO decompressed size (big endian)
 <variable>: LZO compressed data

LZO decompressed data:
 uint32_t: Payload size (LZO decompressed size - 4) (big endian)
 uint8_t[3]: Magic number ("WSG")
 uint32_t: Version (big endian)

 // Reference:
 //  FF D2 8B 06 8B 50 04 6A 04 8D 4D F0 51 8B CE FF D2
 uint32_t: CRC32 checksum (big endian)

 uint32_t: Huffman decoded length (little endian)

 // Reference:
 //  8D 34 96 57 8B 7D 0C C7 46 08 FF FF FF FF C7 06 00 00 00 00
 <variable>: Huffman dictionary

 // Reference:
 //  89 45 FC 83 7D 10 01 8B 45 08 56 8B 75 0C 0F 82 D0 00 00 00
 <variable>: Huffman encoded data
 uint32_t: Unknown

Huffman decoded data:
 <variable>: Google protocol buffer data

Google protocol buffers:
 Field 0x35 (length-delimited)
  Field 0x1 (length-delimited)
   <variable>: Item data
 Field 0x36 (length-delimited)
  Field 0x1 (length-delimited)
   <variable>: Weapon data

Item/Weapon data:
 uint8_t: type (version/type ID)
 uint32_t: seed (big endian)

 // Reference:
 //  0F B6 46 01 C1 E0 08 0B C1 0F B6 4E 04 C1 E0 08
 uint8_t[34]: Obfuscated data

Raw weapon data:
 // Reference:
 //  53 8B 5D 08 56 8B 75 10 89 45 F8 89 45 FC 83 FE 03
 32 bits: Seed (same as weapon data)
 16 bits: Hash (set to 0xFFFF here)
 8 bits: Set ID
 13 bits: Weapon type
 20 bits: Balance
 11 bits: Manufacturer
 7 bits: Weapon level (manufacturer grade)
 7 bits: Weapon level (game stage)
 17 bits: Body
 17 bits: Grip
 17 bits: Barrel
 17 bits: Sight
 17 bits: Stock
 17 bits: Elemental capacitor
 17 bits: Accessory
 17 bits: Accessory 2
 17 bits: Material
 17 bits: Prefix
 17 bits: Title

Raw item data:
 // Reference:
 //  53 8B 5D 08 56 8B 75 10 89 45 FC 89 45 F8 83 FE 03
 32 bits: Seed (same as weapon data)
 16 bits: Hash (set to 0xFFFF here)
 8 bits: Set ID
 17 bits: Item type
 20 bits: Balance
 11 bits: Manufacturer
 7 bits: Item level (manufacturer grade)
 7 bits: Item level (game stage)
 16 bits:
 16 bits:
 16 bits:
 16 bits:
 16 bits:
 16 bits:
 16 bits:
 16 bits:
 16 bits:
 16 bits:
 16 bits:

michaelncguy - 2012-11-18

Question here......

Any idea how the weapon data in the save file is used in relation to the weapon model? I can dump out the weapon files with umodel, but there seems to be 1 model for each type of gun with EVERYTHING on it, and they just remove parts they don't need to display in the game.

I'm trying to find a way to export a gun model and try 3d-printing it for fun.... maybe even add LEDs like the in-game guns glow.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

hi-ban - 2012-10-06

Great news!

will you make some kind of unpacker/packer tool to allow us begin testing?

that way we could begin making a complete gun part list with properties for each part, and also we could test compatibility between different gun parts.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

HeyW - 2012-10-06

No, the code that we're using to extract the save files at the moment is proof of concept only. No interface exists to edit the data and the compression routines were not written. I'm afraid you'll have to wait for a working WillowTree 2 release.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

matt911 - 2012-10-06

They sure did go through a lot of effort to obscure the weapon data especially.

Its good to see that the item level is only 7 bits so there won't be any level 0x7fffffffffffffff shotguns in this game that kill everything in one hit.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Zyreal - 2012-10-06
  
  Matt, I had made you an admin on this project, and it seemed you de-admind yourself...do you not have any interest?
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
  - matt911 - 2012-10-07
    
    Matt, I had made you an admin on this project, and it seemed you de-admind yourself...do you not have any interest?
    
    Is there actually a project here? I don't see any code. If there was a project here, I would be interested in it but probably still not participate in it at this time.
    
    I already had said that I didn't want to administrate a Borderlands 2 project right now before you ever added me as administrator. Administrator has certain duties and responsibilities that I'm not ready to take on for another project yet. I have also stated that I don't have any intention of releasing any Borderlands 2 code until the game is at least a month old which is still almost two weeks from now.
    
    This is not to say that I am ignoring Borderlands 2. I've done a bit of research myself and started writing some alpha stage code related to basic savegame loading and saving, but I don't have anything finished and I'm not going to join or create any projects yet. My own research is not as far along as the work other people have reported in this forum.
    
    A project of this magnitude takes a lot of work and while several people have offered to help, that's no guarantee that in the end I wouldn't still be left to do the lion's share of it myself. There were bugs in WillowTree# that existed for over a year that nobody had ever even reported, even though WT#2.2.1.102 was downloaded over half a million times. That requires far less effort than actually writing any lines of code and people couldn't be bothered with it. I was naive about the amount of effort required to write full-featured Windows applications for public distribution when I started working on WillowTree#, but now I have experienced the Windows nightmare firsthand and it has made me tired. I don't know if I am ready to do that again yet. For now I am doing research and writing the parts of the code that interest me and it remains to be seen whether inspiration will hit me and give me the will to deal with the rest or not.
    
    If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

hi-ban - 2012-10-06

Anyway, it's great news that you managed to extract the code. Also great news that the uncompressed weapon data is similar than the one from BL1.
Looking forward to more news!

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Kyle873 - 2012-10-06

The item data still needs some work. Currently I'm putting that together in the spreadsheet. Grenade Mods are already covered, I just need to inspect some Shield, Relic and Class Mods and then I can finish those up too.

edit:
There, that should cover almost everything.

Last edit: Kyle873 2012-10-06

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Kyle873 - 2012-10-07

We are no longer working on the file format.

Gibbed has linked us to his code, which is much more complete than ours. We also have his permission to use his code in WT2.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

matt911 - 2012-10-07

Gibbed's code is quite detailed and impressive. He's a very skilled programmer and reverse engineer. I took a peek at his editor and it is almost complete for a basic editor. It seemed to mainly be lacking name resolution for weapons, items, skills, etc.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

strikerx3 - 2012-10-09

I would like to point out that the save file also seems to contain some sort of unlock flag that enables the character to enter the Natural Selection Annex (the Creature Slaughter Dome area). This does not seem to be connected to your profile, account or anything but the character save. In fact, I have a save from a friend who has access to that place and I can enter it with his save, even though I don't have the Gamestop pre-order bonus. My own characters do not have access to the place.

It would be great to be able to change that flag for those unlucky people who don't have the Gamestop pre-order bonus, at least until the DLC is released.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Kyle873 - 2012-10-09
  
  It's in Gibbed's code, so I see no reason why not.
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
  - strikerx3 - 2012-10-09
    
    OK, cool.
    
    Is there anything I could help you guys with? I may not be able to put much time into the project, but I'm sure I could give a hand. It may not be evident, but I'm a skilled Java programmer with some knowledge of C# and very strong OOP skills.
    
    If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Kyle873 - 2012-10-09

I'm personally not going to do much until I see how far Gibbed takes his editor, and I believe the others who I've been working with also feel the same. When his editor is in a mature/stable state, or when he stops working on it, this will likely be the time that actual development on Willowtree 2 begins. For now, I would just advise everybody to sit tight and wait around a bit to see what happens.

On another note, I've also sent off a message to duncanfogg to see what his interest may be in developing an awesome Gear Calculator alongside (or even INSIDE) Willowtree 2, or at least making the data he has into a format we could bring into Willowtree 2 to give users a ballpark idea of what kind of stats/benefits an item/weapon may have as you build it.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Racoon SilverMoon - 2012-10-10

Like @Kyle873, I'll be busy for a while. I prefer to watch Gigged continue his great work until he had enough or we have time to invest in WT2.

Don't expect anything from me before Christmas !

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Mik - 2012-10-19

Is it possible for one of you guys to pass me Gibbs code. I can give this a go during the weekend. Just a little background on who I am and why I am asking. I have been programming for the past 15 years or so as a professional. I have primarily been focused on Java and C but I have come to like/admire C#. As it relates to developing on a Windows based platform of course. Mono is nice but there is a plethora of development capability that is better served utilizing C when it comes to Linux but I digress. On any note, if you guys can pass me the source code I can dedicate at least 18+ hours completely and solely towards building on what he has. I will be more than happy to share anything that I find, discover, and/or enhance with regards to his efforts. PEACE OUT! P34(3 0U7!

For the chosen few who understand Hack culture:

15 17 p0551BL3 Ph0R 0||3 0Ph j00Z 9U'/5 70 p455 /\/\3 91BB5 (0D3. 1 (4|| 91\/3 7|-|15 4 90 dUR1||9 7|-|3 \/\/33|<3||D. jU57 4 L177L3 b4(|<9R0U||D 0|| \/\/|-|0 1 4/\/\ 4||D \/\/|-|'/ 1 4/\/\ 45|<1||9. 1 |-|4\/3 b33|| pR09R4/\/\/\/\1||9 Ph0R 7|-|3 p457 15 '/34R5 0R 50 45 4 pR0Ph35510||4L. 1 |-|4\/3 pR1/\/\4R1L'/ b33|| Ph0(U53D 0|| j4\/4 4||D ( bU7 1 |-|4\/3 (0/\/\3 70 L1|<3/4D/\/\1R3 (#. 45 17 r3L4735 70 d3\/3L0P1||9 0|| 4 \/\/1||D0\/\/5 b453D pL47Ph0R/\/\ 0Ph (0UR53. /\/\0||0 15 ||1(3 bU7 7|-|3R3 15 4 pL37|-|0R4 0Ph d3\/3L0P/\/\3||7 (4P4B1L17'/ 7|-|@ 15 b3773R 53R\/3D U71L1Z1||9 ( \/\/|-|3|| 17 (0/\/\35 70 L1||U>< bU7 1 d19R355. 0|| 4||'/ ||073, 1Ph j00Z 9U'/5 (4|| p455 /\/\3 7|-|3 50UR(3 (0D3 1 (4|| d3D1(473 @ L3457 18+ |-|0UR5 (0/\/\PL373L'/ 4||D 50L3L'/ 70\/\/4RD5 bU1LD1||9 0|| \/\/|-|@ |-|3 |-|45. 1 \/\/1LL b3 /\/\0R3 7|-|4|| |-|4PP'/ 70 5|-|4R3 4||'/7|-|1||9 7|-|@ 1 Ph1||D, d15(0\/3R, 4||D/0R 3|||-|4||(3 \/\/17|-| r394RD5 70 |-|15 3PhPh0R75. P34(3 0U7!

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Racoon SilverMoon - 2012-10-19
  
  http://svn.gib.me/public/borderlands2/trunk/
  
  2012/10/18 Mik mickillah@users.sf.net
  
  Is it possible for one of you guys to pass me Gibbs code. I can give this
  a go during the weekend. Just a little background on who I am and why I am
  asking. I have been programming for the past 15 years or so as a
  professional. I have primarily been focused on Java and C but I have come
  to like/admire C#. As it relates to developing on a Windows based platform
  of course. Mono is nice but there is a plethora of development capability
  that is better served utilizing C when it comes to Linux but I digress. On
  any note, if you guys can pass me the source code I can dedicate at least
  18+ hours completely and solely towards building on what he has. I will be
  more than happy to share anything that I find, discover, and/or enhance
  with regards to his efforts. PEACE OUT! P34(3 0U7!
  
  For the chosen few who understand Hack culture:
  
  15 17 p0551BL3 Ph0R 0||3 0Ph j00Z 9U'/5 70 p455 //3 91BB5 (0D3. 1 (4||
  91/3 7|-|15 4 90 dUR1||9 7|-|3 //33|<3||D. jU57 4 L177L3 b4(|<9R0U||D 0||
  //|-|0 1 4// 4||D //|-|'/ 1 4// 45|<1||9. 1 |-|4/3 b33|| pR09R4////1||9
  Ph0R 7|-|3 p457 15 '/34R5 0R 50 45 4 pR0Ph35510||4L. 1 |-|4/3 pR1//4R1L'/
  b33|| Ph0(U53D 0|| j4/4 4||D ( bU7 1 |-|4/3 (0//3 70 L1|<3/4D//1R3 (#. 45
  17 r3L4735 70 d3/3L0P1||9 0|| 4 //1||D0//5 b453D pL47Ph0R// 0Ph (0UR53.
  //0||0 15 ||1(3 bU7 7|-|3R3 15 4 pL37|-|0R4 0Ph d3/3L0P//3||7 (4P4B1L17'/
  7|-|@ 15 b3773R 53R/3D U71L1Z1||9 ( //|-|3|| 17 (0//35 70 L1||U>< bU7 1
  d19R355. 0|| 4||'/ ||073, 1Ph j00Z 9U'/5 (4|| p455 //3 7|-|3 50UR(3 (0D3 1
  (4|| d3D1(473 @ L3457 18+ |-|0UR5 (0//PL373L'/ 4||D 50L3L'/ 70//4RD5
  bU1LD1||9 0|| //|-|@ |-|3 |-|45. 1 //1LL b3 //0R3 7|-|4|| |-|4PP'/ 70
  5|-|4R3 4||'/7|-|1||9 7|-|@ 1 Ph1||D, d15(0/3R, 4||D/0R 3|||-|4||(3 //17|-|
  r394RD5 70 |-|15 3PhPh0R75. P34(3 0U7!
  
  Sent from sourceforge.net because you indicated interest in
  https://sourceforge.net/p/willowtree2/discussion/Dev/
  
  To unsubscribe from further messages, please visit
  https://sourceforge.net/auth/prefs/
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Racoon SilverMoon - 2012-10-19

http://svn.gib.me/public/borderlands2/trunk/

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

JakeBarrage - 2013-01-04

just use bruteforce save data http://aldostools.org/ps3tools.html hope that helps

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

David Sellers - 2014-01-06

Please see my post here.
Thank you.
https://sourceforge.net/p/willowtree2/discussion/General/thread/c9e23fb5/

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Save Game File Format (Hints)

Borderlands 2 save editor.

Forums

Help

Save Game File Format (Hints)

Save Game File Format (Hints)

Borderlands 2 save editor.

Forums

Help

Save Game File Format (Hints) document.SUBSCRIPTION_OPTIONS = { "thing": "topic", "subscribed": false, "url": "subscribe", "icon": { "css": "fa fa-envelope-o" } };

Save Game File Format (Hints)