Serialization done, back to porting
Plenty of things done this week! Well, as much as my part time allowance ... allows.
Weak references
First of all, last week's problem is this week's solution. I've resolved the weak references issue by handling it in two ways. First, for all types except temporary effects, I've got some "entity data reference" datatype that store where/how can we find the strong owning reference. For example, we can provide the entity that owns the reference, an enum that can be mapped to code that goes through the entity and finds a container of strong references, and some id/key to pick the right one. Nice and simple.
Temporary effects are handled somewhat differently because when we create the weak references, we can't always know our exact location in the entity (or config database -- that's another potential "owner" of the strong reference). For these ones, after a lot of head scratching I decided: what the hell, I don't care if during deserialization it becomes a separate copy. What is imperative though is to ensure that any code that was finding a temporary effect by looking at object being the same (in C++ that would be an address check), now I need to create a unique stable key (persistent across runtimes) and compare keys. C# manages to be a PITA in that department, as the default GetHashCode() function is not stable and to check for equality (IEquatable) you're supposed to implement 3 functions that all look very similar and they all use all included member variables/properties. Tedious! So, I'm just building a string representation of the class and I'm using that. Maybe it's slightly slower, but it's going be very useful for debugging as well.
Serialization
Ok with weak references done, the road was clear for moving back to MemoryPack. The other hindrance was the requirement for base classes to be abstract, but that didn't become an issue during moving so that was nice. I started decorating everything as necessary, mostly manually. I counted them: 419 serializable types with 2086 attributes specified: if a class is serializable, if/what subclasses it contains, if some members should be explicitly included/ignored, and for each serializable member its order during serialization.
As I wrote last time, I'm going with the version tolerant option, and explicit member order. The good thing about it is that if I forgot to specify order in a serializable member, I get a compile error/reminder, rather than a sneaky bug. Great!
For increased robustness, I wrote a code analysis tool that inspects the codebase and reports if I forgot to make a new derived type serializable, or a number of other checks that wouldn't be caught otherwise.
After all was done, I ran a new benchmark. Well, long story short, the performance was the same, which means the "version tolerant" support had negligible effects, comparatively. Great!
JSON fixes for serializable class properties
Previously properties were never used to store data, but I've been refactoring the code to do that since MemoryPack supports that well. This conversion from fields to properties did allow for some cleaner code as well, with the easy specification of public getter and private setter. But to make things slightly more complex, any such properties that need to be serialized and are members of a base class, create problems for derived classes as MemoryPack demands better visibility, so the setters need to be protected rather than private. But I digress.
JSON.Net loading code expects serializable types with fields rather than properties, so any such logic needed to be amended. So, now it's amended.
Back to the game
With all the above sorted, I could now run the game again. Fixed quicksave/quickload, which are indeed pretty quick but not lightweight, as I'm storing a whopping 17MB file. Probably some unnecessary stuff, but it's pretty fast so I don't care for now.
Next up was the deepcopy operation, which kicked off the serialization rabbit hole. I replaced the code with using MemoryPack to deepcopy: convert to bytes and then deserialize to new object. Easy, fast, works.
Next up was the creation of an "adventure location", a dungeon. This was successful as well.
Next up was to be able to get into a new level, this is almost the penultimate bastion in the porting process (last one is effects!). Had to do some general bugfixing, and still working on it, currently porting shaders.
Reading from last render pass
Closing for today with another unexpected nice thing. My fog of war shader utilised background texture reading in the shader: the current pass had to read the texture generated from the last pass. Unity made that easy, Godot makes it easy too for their high level pipeline, and I was worried that my DIY render pipeline would make this quite tedious. Long story short, it was extremely simple to do, so with about 20 lines of code I have support for this in any shader. This allows things like heat haze effects, or other distortion effects such as the one at the top of the page and down here