Replay File Format

Any and all discussion relating to Rocket League. Post feedback and suggestions here!

Re: Replay File Format

Postby danielsamuels on Mon Mar 14, 2016 5:03 am

Really nice work, I've added your repo to the list of parsers. I've also added a 'status' column, which should be fairly accurate.
danielsamuels
Acrobatic
User avatar
 
Posts: 53
Joined: Thu Jul 30, 2015 10:41 pm
Location: Cambridge, UK
Steam ID: posix_
Region: EU

Re: Replay File Format

Postby calsmurf2904 on Mon Mar 14, 2016 5:30 am

Thanks! The status should be active though and not abandoned. :)
calsmurf2904
Supersonic
 
Posts: 3
Joined: Sun Mar 13, 2016 12:00 pm
Steam ID: calsmurf2904
Region: EU

Re: Replay File Format

Postby danielsamuels on Mon Mar 14, 2016 5:38 am

Whoops, thought I'd fixed that. Updated!
danielsamuels
Acrobatic
User avatar
 
Posts: 53
Joined: Thu Jul 30, 2015 10:41 pm
Location: Cambridge, UK
Steam ID: posix_
Region: EU

Re: Replay File Format

Postby fozworth on Sun Mar 27, 2016 5:33 pm

Has anyone figured out the CRC yet? I know there are two in each replay file. One is for the metadata and the other is for the rest. I have tried computing them with a standard CRC32 from zlib as well as the StrCrc_DEPRECATED and MemCrc_DEPRECATED functions from UE4. None of those worked. Since getting the CRC almost right is indistinguishable from getting it completely wrong, it's hard to figure out.
fozworth
Supersonic
 
Posts: 13
Joined: Wed Sep 30, 2015 7:44 pm
Steam ID: Gompers

Re: Replay File Format

Postby ZorMonkey on Sun Mar 27, 2016 5:45 pm

A few weeks ago I spent some time digging around in the exe trying to figure out the CRC, but I didn't get too far. I *did* find the CRC tables from the Unreal code in memory, the ones that start with 0x00000000, 0x77073096, 0xee0e612c if I remember correctly. So that suggests they're not using some custom crc code (although I suppose they could have 2 different crc routines in there). My bet is that different chunks of data need to be hashed in different orders and/or we need to seed the CRC with some unknown whatsit. I gave up for a while, maybe I'll revisit it someday...
ZorMonkey
Acrobatic
 
Posts: 36
Joined: Sun Oct 04, 2015 6:07 pm
Steam ID: ZorMonkey
Region: US

Re: Replay File Format

Postby fozworth on Sun Mar 27, 2016 5:56 pm

Yeah, there are a ton of variables with the CRC. Based on our experience with the net stream, I wouldn't be surprised if we had to flip each bit in the bytes and then also flip every group of four bytes :)
fozworth
Supersonic
 
Posts: 13
Joined: Wed Sep 30, 2015 7:44 pm
Steam ID: Gompers

Re: Replay File Format

Postby ZorMonkey on Sun Mar 27, 2016 6:00 pm

Yeah. :) Thats why I tried figuring out what the exe is doing, figured that might be the "easiest" way to determine exactly what it's doing. But I'm pretty horrible at that sort of stuff...
ZorMonkey
Acrobatic
 
Posts: 36
Joined: Sun Oct 04, 2015 6:07 pm
Steam ID: ZorMonkey
Region: US

Re: Replay File Format

Postby fozworth on Sun Mar 27, 2016 6:46 pm

For reference, the CRC table you mentioned can be found here: https://github.com/EpicGames/UnrealEngi ... c.cpp#L177
fozworth
Supersonic
 
Posts: 13
Joined: Wed Sep 30, 2015 7:44 pm
Steam ID: Gompers

Re: Replay File Format

Postby fozworth on Sun May 29, 2016 9:30 am

I'm happy to announce that my parser, Octane, can now parse every replay file I can throw at it. It even parses some things that PyRope doesn't! It's also about 10 times faster the PyRope. It completely parses most replays in less than 3 seconds. Binaries are available on GitHub. If you run into any replays that it can't parse, please open an issue!

Huge thanks to everyone in this thread, by the way. I copied liberally many of the existing parsers. I could not have done it without y'all!
fozworth
Supersonic
 
Posts: 13
Joined: Wed Sep 30, 2015 7:44 pm
Steam ID: Gompers

Re: Replay File Format

Postby danielsamuels on Sun May 29, 2016 9:44 am

Wow, nice work. I might have to use it for RLR. Processing time in Pyrope is my main problem at the moment.
danielsamuels
Acrobatic
User avatar
 
Posts: 53
Joined: Thu Jul 30, 2015 10:41 pm
Location: Cambridge, UK
Steam ID: posix_
Region: EU

Re: Replay File Format

Postby fozworth on Sun May 29, 2016 9:51 am

Thanks! And that would be awesome! It still has some rough edges that I'd like to work on before it's ready for production use. Also I'd have to make the JSON output match what PyRope spits out.
fozworth
Supersonic
 
Posts: 13
Joined: Wed Sep 30, 2015 7:44 pm
Steam ID: Gompers

Re: Replay File Format

Postby danielsamuels on Sun May 29, 2016 9:54 am

Pyrope doesn't output JSON directly, it's a library which I use to process the files and then generate the JSON in my code. To be honest, if you output the raw frame data as JSON then I can work with it.
danielsamuels
Acrobatic
User avatar
 
Posts: 53
Joined: Thu Jul 30, 2015 10:41 pm
Location: Cambridge, UK
Steam ID: posix_
Region: EU

Re: Replay File Format

Postby fozworth on Sun May 29, 2016 9:58 am

Yup, it currently outputs JSON. You should be able to download the binary and run it against a replay file to see what it gives you. It doesn't pretty-print the JSON, so I normally do something like:
Code: Select all
octane the.replay | python -m json.tool | less
.
fozworth
Supersonic
 
Posts: 13
Joined: Wed Sep 30, 2015 7:44 pm
Steam ID: Gompers

Re: Replay File Format

Postby Unclepeanut on Sat Jun 04, 2016 7:23 pm

Howdy o/

So I'm completely new to parsing, but I'm wondering whether it would be somewhat possible to merge replays together. Some of you may have seen a "1 - 20k project" for Trackmania, where they've had up to 20,000 cars racing on the one track, at one time. The chaos that ensues is almost breathtaking.

I wonder whether it would be possible to do it with Rocket League by changing a replay file somehow, but I'm a little confused as to how you use the parsing programs, like Octane or PyRope.

Cheers guys o/
Unclepeanut
Supersonic
 
Posts: 3
Joined: Sat Jun 04, 2016 6:28 pm
Steam ID: 2KF1 | Whiff Lord
Region: Australia

Re: Replay File Format

Postby rustixote on Sun Jun 05, 2016 12:05 pm

technically yes you could modify the netstream and add additional actors. but i imagine there are client-side checks that would prevent the replay from displaying a massive number of cars. also the cars have collision so im not sure how you would work around that
rustixote
Supersonic
 
Posts: 12
Joined: Mon Sep 28, 2015 4:52 pm
Steam ID: spike-
Region: us

Re: Replay File Format

Postby rustixote on Sun Jun 05, 2016 12:06 pm

oh and also we dont know how to generate the crc (basically a check to see if the file is valid/corrupted), so the game would probably just ignore the file altogether
rustixote
Supersonic
 
Posts: 12
Joined: Mon Sep 28, 2015 4:52 pm
Steam ID: spike-
Region: us

Re: Replay File Format

Postby danielsamuels on Sun Jun 05, 2016 12:11 pm

The biggest problem you'd have would be conflicting actor IDs.
danielsamuels
Acrobatic
User avatar
 
Posts: 53
Joined: Thu Jul 30, 2015 10:41 pm
Location: Cambridge, UK
Steam ID: posix_
Region: EU

Re: Replay File Format

Postby Unclepeanut on Sun Jun 05, 2016 2:35 pm

rustixote wrote:oh and also we dont know how to generate the crc (basically a check to see if the file is valid/corrupted), so the game would probably just ignore the file altogether


Is that currently being worked on?

rustixote wrote:technically yes you could modify the netstream and add additional actors. but i imagine there are client-side checks that would prevent the replay from displaying a massive number of cars. also the cars have collision so im not sure how you would work around that


Would these client-side checks be included in the crc? Car collision might be the biggest problem here.

danielsamuels wrote:The biggest problem you'd have would be conflicting actor IDs.


Would it be possible to just change someone's name in the replay? e.g. if I personally had two replays, I could call myself "asdf" and "asdf1". Is that what you mean by Actor IDs?
Unclepeanut
Supersonic
 
Posts: 3
Joined: Sat Jun 04, 2016 6:28 pm
Steam ID: 2KF1 | Whiff Lord
Region: Australia

Re: Replay File Format

Postby ZorMonkey on Sun Jun 05, 2016 9:26 pm

Some work was done with the crc in the past, but I'm not sure if anyone is still trying to crack it. I've had some ideas, but I'm not too motivated.

For the actor ids, internally every piddly thing has an arbitrary id associated with it. If you were to combine replays you'd have to assign new ids to everything. You'll also bump into the 1024 actor limit, but the replay viewer might be able to handle more actors if we told it the max was higher.

You'd also have to put some thought into how game state data is combined. For example, if I'm looking at the middle of a merged replay, what does the clock say? Is it paused because of a goal? Is it overtime?

Collisions might be a problem. For the most part there is no reason for the replay viewer to check collisions, so maybe it'd be fine. But certain physics elements aren't included in the replay, so it might freak out. For example, tire and suspension motion. Would it be able to determine if the tires are touching anything if they're somehow inside 40 other cars?

It'd probably be easier for someone's custom replay viewer to just handle multiple replays at once and the idiosyncrasies that come with that instead of tricking the real replay viewer into showing a merged relay. The downside is that it'd probably look like garbage. :) I havent worked on my janky replay viewer (it loads super slowly, sorry) (no Patreon required) for a couple months, but it might be an interesting challenge...
ZorMonkey
Acrobatic
 
Posts: 36
Joined: Sun Oct 04, 2015 6:07 pm
Steam ID: ZorMonkey
Region: US

Re: Replay File Format

Postby Unclepeanut on Mon Jun 06, 2016 1:06 am

I didn't realize external replay viewers existed. I can't find anything through Google.
Unclepeanut
Supersonic
 
Posts: 3
Joined: Sat Jun 04, 2016 6:28 pm
Steam ID: 2KF1 | Whiff Lord
Region: Australia

PreviousNext

Return to General Discussion

Who is online

Users browsing this forum: Bing [Bot] and 9 guests