Ok, enough with Unity. Starting a log for porting over to Godot, and refactoring heavily along the way. It's not going to take a weekend, or 14 hours, but it will get there. The bridges have been burnt and there's no way back.

Feasibility study

The first thing that I did was to check if I had to use C++ modules or GDNative extensions. Looking through the docs, I decided it looked way too tedious plus I'd have to port a lot of code. I can't see a definite reason for that choice, as the support for C# scripting looks reasonable enough, so I'll try to use it as-is. It should be a no-brainer for most people, but maybe I'm tempted by C++ a bit too much. GDScript is not an option at all, since it would be new, it's slow, and it's used in a way that I don't plan to use it.

So, decision #1 is to use C# for the project, and create a similar project/scene structure, but cleaner this time since I know what I'm doing better

  • The bulk of code is in C# scripts. The top-level class stores the persistent and runtime state, and it's saved in a node that persists across scenes.
  • Use of nodes will be very lightweight, mainly for GUI and rendering. There is no association between Godot nodes and any of the game's entities.

Given these, how does Godot score against my requirements from last week?

  • Easy save-game serialization As-is, with C# (can't wait to hit the lack of support for BinaryFormatter)
  • Interface with C++ plugin Tried that already, with [DllImport], seems to work, so code is mostly as-is
  • Debugging native code It's supported
  • Debugging script code Works, but I still need to find "wait for debugger" option
  • Easy writing/debugging shaders Looks fairly nice and GLSL-like, that I like
  • A solution for UI Looks reasonable, I'll deal with it when I get there

All aboard the Godot port

After a bit of feasibility checking, I think we're good to go for porting to Godot. Also Caves of Qud seems to be in the process of being ported, and Brian's thread is very entertaining and full of energy. Exactly what we need imo. My process is going to be slightly different, due to my messy code structure, so I'll refactor some things on the way, as I was looking for an excuse to do that for a while.

What needs to be ported? (or killed due to being deprecated)

  • Editor scripts (6.5k LOC, 49 files)
  • Game scripts (75k LOC, 443 files)
  • Shaders (12.5k LOC, 133 files)

What does not need to be ported?

  • C++ native plugin code: ~100k LOC
  • JSON configurations (instead of Unity ScriptableObjects, yay!): ~70k LOC

Below is an itemized log.

Day #1

  • Reading about C++ modules and GDNative extensions. There's a long page of requirements and setup, so it sounds tedious. A port is not realistic as there's still no nice serialisation like C#, so it's a red herring. Will use Godot/C# so that I can reuse all my code
  • C# tests and Rider seem fine. I tried to "emulate" my codebase by procgen a C# script with 10000 classes, and 1000 scripts with 1 class each, and opening the project. Timings looks fine, but I'm not sure if this test is really effective if you don't really use many of these classes
  • Looking into shaders, they look absolutely fine and the porting difficulty would not be great. It will be mostly syntactic, e.g. lots of float2 to vec2 etc.
  • The first thing I'm going to ensure it runs is the overworld generation, and I need render-to-texture (RTT) for that. So I figured how that works in Godot. For 2D we just create a Sprite2D that is going to display the RTT, and we also create a SubViewport with a ColorRect attached. Set ColorRect's layout to full rect, set the shader in the ColorRect, set the shader material to "local to scene" and set the subviewport as the Sprite's texture. Done! Also there's an option to trigger the RTT once, which is what I need for the overworld display. Ok, I'm confident about this.
  • While trying the RTT functionality, I realised that 3D objects need spatial shaders (not canvas_item) otherwise object looks black and there's no clear error.
  • Godot has globalize_path for converting "res://" folders to absolute paths, which is useful for porting things like Application.streamingAssetsPath
  • I successfully called the native plugin (well, just a subset of functions) and I reused most of the native plugin code. That's nice.
  • I read about having persistent state between scenes

~4h

Day #2

Spent quite a bit of time identifying what needs to be done.

How do I port? Do I copy over the existing code structure, do I fix before porting or do I fix throughout? The wise thing to do would probably be to fix it before porting, but I have a one track mind, and the track is currently "port now!". Porting throughout would require to move files carefully rather than en-masse, and I'd rather move things en-masse. So, I've spent some time on my "ideal project structure" and I'm going to port initially as-is and move things later. A special case are the editor scripts, that are very Unity dependent, and these will be moved at the end

Assets need to be ported too. I use binary Unity blobs for storing textures, but thankfully I create these textures from source assets in C#, so I'll just copy that code over and create a different blob which is loadable from Godot. A DDS would be nice if it's supported.

I need to add any nuget packages. I'm using Newtonsoft.Json and JonSkeet.MiscUtil only.

I need to find a math library that behaves like Unity.Mathematics that I'm now quite fond of (it's like GLM for C++). There's GlmSharp -- it looks a bit old, but it shouldn't matter as it's the simple API that I need.

~2h

Day #3

  • Added nuget packages. Looks super-easy through JetBrains Rider: go to Tools -> NuGet -> Manage NuGet Packages. But JonSkeet.MiscUtil latest package does not support .NET 6 so I'm not sure if I'll have issues with it. Same with GlmSharp.
  • Copied over all non-editor scripts to new project. 6,303 errors
  • Removing all "using Unity." imports. It turns out that automatically importing missing references results in all sorts of bizarre imports, as I got some localisation, advertising, codec and so on that were never used. Ok after this step ... 7,076 errors.
  • I'm now going to apply the Qud port approach of placeholder classes/functions: GameObject, MonoBehaviour, Debug.Assert, Debug.Log, GUILayout, SerializeField, PlayerPrefs, Profiler, math namespace functions/classes, ISerializationCallbackReceiver, Application, Color32 etc.

Ok, after lots of hours of doing the above, I'm down to 0 errors errors. It compiles! Of course these were just compile-time errors: the game is nowhere near ready to run. Now I have all the offenders contained in a single file (called UnityPlaceholders) so that I can refactor the code structure and make decisions about the different parts.

Nontrivial things to fix:

  • Unity jobs. These have to be moved to C++, they should not be complicated as the data types operated on are just PODs.
  • Unity things like Textures, Materials, Shaders, etc. I can fake them for now just to compile of course
  • Editor scripts, that relied on Unity a LOT. These are not as important and will be done later
  • Shaders! These are important though, and I need to start porting (at least partially to begin with) as soon as something compiles

I need to make a plan for the incremental placeholder removal now. The first and most obvious is the math library, as it's quite a big part of the placeholder code. After replacing the types with the glm types (ivec2, ivec3, vec2, vec3, etc), here's what's left from the shim layer

~10h

Day #4

Replaced unimplemented math code with glm and Mathf. So, what is left in there? Let's do some damage control (again, taken from here):

Low hanging fruit

  • PerlinNoise: I used a little bit Unity's built-in perlin noise, I'll replace this to whatever Godot offers, and if it doesn't, I'll just add some opensimplex library I guess.
  • Font: This is barely used (in a single spot) and is half-baked anyway. Very easy to use something more sensible.
  • SystemInfo: Just system capabilities, I used them in a few places for debugging, so I'll either remove them or find equivalents.
  • grayscale: that was a stray color extension function, so it's going to move to color utilities
  • Screen: simple utility struct to get width/height, easily portable
  • Rect: should be easy to find Godot equivalent. This was used in guilayout and a few other places
  • Debug: logging, assertions, etc. Should be easy to replace later, but I've already have the functionality in this shim layer
  • PlayerPrefs: For storage of some user settings, I barely used it so it's easy to port, or remove
  • JsonUtility: this will be refactored away to use Newtonsoft json, I used it in only a few spots in the beginning but never refactored it away
  • UnityEngine.Profiling.Profiler, ProfilerMarker: Need to find equivalent for Godot profiling
  • Color32: implement it to play along with Godot's color type (implicit conversions etc) and ensure it's 4 bytes, in the order specified, etc
  • Application, Time, LogType: These contain some globals e.g. paths, log handling and time. Need to find the equivalents
  • SerializeFieldAttribute, ISerializationCallbackReceiver: This is to make private variables visible in inspector and to process what's shown. Need to find the equivalent, if any, in Godot

Medium hanging fruit

  • GUIStyle, GUILayout: These are used for my developer gui. I need to find the equivalent Godot functions.
  • ReadOnly/WriteOnlyAttribute, Job, IJob, IJobParallelFor: These were used in the job system, so they'll just be removed after I move the jobs to C++
  • Singleton: This needs a bit of special handling, as apparently Singletons need to be in a global persistent scene that autoloads etc. The whole game will live in a singleton. I'm not even sure if I'm going to use different scenes, but I suspect I will due to the difference between root 2D and 3D nodes
  • Input, KeyCode, Event.keyCode: General input handling sprinkled through the codebase, shouldn't be hard to map even though Godot processes input events differently from what I read
  • AudioSource,AudioClip,AudioMixer: Again, basic use of these, so it shouldn't be too hard
  • MeshRenderer, MeshCollider, MeshFilter: These tend to be used as a means to an end, e.g. accessing materials etc.
  • EngineObject Destroy/DestroyImmediate: I made this wrapper to refer to all the explicit unity object destruction code. There should be something like that in Godot
  • NativeExtras.SubArray: that was a utility to efficiently slice a native array (for texture updates), I think in the new code it will transform to a Span<>, but it also depends on what is required to efficiently process/update texture data.
  • Physics.RayCast, Ray,RaycastHit: this was used in a couple of places only, to figure out what grid cell is under the mouse pointer. Easy to replace.

High-hanging fruit

  • GameObject, MonoBehaviour, Component, Transform: standard unity stuff, these should be replaced with Godot node handling code
  • Camera, LayerMask: Camera handling e.g. with mouse or through code. Also used for rendering to specific layers
  • Material, Shader: This is sprinkled in the codebase around areas where we setup rendering. It is going to be tricky
  • ComputeBuffer: That's a bit tricky. This was used for pushing buffer data to the GPU, e.g. sprite data or level tile data and so on. It shouldn't be too hard to find an equivalent though, as the basis is just a GPU buffer with typical APIs
  • Graphics.DrawMeshInstancedIndirect: This was my bread and butter for instanced quad rendering. Need to find the equivalent.
  • FilterMode,TextureWrapMode,TextureFormat,RenderTextureFormat,RenderTextureReadWrite,Texture,Texture2D,Texture2DArray,RenderTexture: all of the texture handling code

And that's all of what needs to be ported, in terms of non-editor-script, non-shader code! Phew. Doesn't look that bad. What's going to be harder is the code refactor, which means looking at ALL my code and splitting things around and organising them in a nice proper way, unlike the spaghetti organisation that I currently have

Ok, time to refactor these, lower hanging fruit first! Also, shudders, each of these needs to be tested.

PerlinNoise

Godot seems to have some FastNoiseLite, but it's a godot object with all the standard bloat? No thanks. I got Dotnetnoise from nuget, instantiate a static instance and that's it. I created an empty scene, added a node, attached a script that calls the code in order to gather some data aaaand ... problems! See rant below. After problem was solved, I did a quick check on the generated data, looks reasonable, ok now next bullet point, but first the ...

...Godot rant #1 It seems that I had a file in the project a few days ago which I removed, but when I build the project, godot is trying to compile that file and fails. For a while it showed the folder the file was located in the FileSystem tab, and when I clicked in that (non-existent) folder, Godot crashed. Oops. So I found what was wrong. The "play scene" did not play the scene that I have loaded, but some previous scene that included the deleted file. It is not clear which scene is being run. Bad Godot UI. I need to remember to always click on "Play current scene" rather than "run project".

Godot rant #2 I exported some histogram data in a CSV and I dared to put them in the project. What happened? I got about 500 auto-generated files, half of them with the suffix .translation. Apparently, CSV files are treated as translations by default. But in the docs, it doesn't say how to turn that off.

Font

That looks suspiciously easy. I just have to test and see that Godot.Font works as I expect it to. Created a new scene, added a label, and in the theme overrides I changed the font to one of the ones in assets, it was rather painless. All ok. That was not really porting.

SystemInfo

I looked in the docs for any functions that show support of particular texture formats etc depending on platform but I couldn't find any. Ok, no problem, this feature bites the dust, it was used just in a single place anyway and it was for information only.

grayscale

This used to be an extension of Color, I've now made it a utility function like other color conversions. Done.

Screen

There seem to be a few ways to get the screen dims, I used DisplayServer.WindowGetSize(). Made a new scene, with a label that prints the screen size, I resized the window and it kept changing, so looks good.

Rect

Godot has Rect2. But this is a suspect low hanging fruit. I need to ensure that the interface and the content matches, as it's not just "a struct with 4 numbers". After a quick look -- it does.

Debug

Easy! Redirecting to GD.Print, and putting it in a "godot" namespace so that I control which classes talk to Godot directly (for future refactoring)

Finally for today, sweet pre-sleep discovery: in C# 10, we can use something like "global using Rect2 = Godot.Rect2" so that we can create a project-wide alias, and this is something that was not supported before! This makes it FAR easier to write engine-agnostic code, e.g. the entire codebase can use Rect2 without importing Godot in all files. All godot references can be localised to a few files.

~4h