The @Remnant_Game update on the Epic Games Store was 37 GB whereas the same update was only around 900 MB on Steam. People blamed Epic's platform for this, but I didn't quite think that was exactly right. A thread:

Thanks to my python-based @legendary_gl we have a parser for the Epic manifest format that we can use in ipython/jupyter notebooks to explore the manifest and see what's up. Only four files changed (exe + pdb and two paks) but those contain most of the game data.

Now let's see why the patch is so big. Normally the system should try to re-use chunks that exist in both versions and only download the changed ones. However, if we compare the two PAK files they share no chunks at all:

This would only make sense if e.g. the files were compressed differently or encrypted with a new key. But they aren't. They do share a lot of data, so what happened? Well let's run Epic's "BuildPatchTool" ourselves and see what it makes of it...

And indeed, when checking our own manifests with the BuildPatchTool's diff mode we see that we now have a patch size of ~923 MB, just like Steam (Legendary will say ~2.1 GiB here since its patcher isn't as good with reusing existing chunks yet).

So what went wrong?

Based on my limited knowledge of Epic's services I can make the following guesses:

- They didn't use the tool correctly so no old chunks were reused
- Epic's tool requires them locally, whereas Steam's tool can use already uploaded chunks
- The docs may not be clear about this

The reason no old chunks were reused may be that the Remnant developers cleared the output directory ("CloudDir") after the builds were uploaded. So the tool ran without old data available for more efficient patches.

So in conclusion: is this a problem with Epic's platform? Well, not really.

But it does highlight a potential issue with their tooling; that unlike Steam's tooling it requires you to keep a copy of the data you uploaded recently in order to build an efficient patch.

I do not have access to the Epic Store Services documentation unfortunately, so I have no idea what it says about building patches and uploading artifacts. Would be interesting to know if they specifically make note of this.

Another theory since I just noticed that the AppName (ArtifactId in BuildPatchTool) doesn't match between the latest and previous maniefsts of Remnant; maybe the latest build was created on a different machine than the previous ones?

Or maybe it was originally a staging build that was built from scratch and then moved to "Live" after uploading (not sure if that's something you can do) without running the tool against the old output to create optimized patches.

As another update to this: in order to get the patch size down to the 923 MB that Steam had the developers would also have to run the "ChunkDeltaOptimize" mode of the BuildPatchTool. So far I've not seen any game that actually had delta manifests available however.

So under "normal" circumstances the patch size would have been around 2.2 GB on Epic vs 923 MB on Steam, due to Steam running more optimizations by default. In my case I ran the delta optimizer but didn't realize that the diff mode would silently load the optimized manifest.

However the lessons learned have now made its way into @legendary_gl with support for delta manifests and their more aggressive chunk re-use :)