Don't Forget Developer Experience

[johnbanq]

Hi! I just ditched my bedrock server project (due to lack of time) and found this project. So I am sharing some of my thoughts here. Hope this is helpful and provides some good insights!

TL;DR

• We should make our server friendly to developers writing vanilla game logic & plugins
• And I mean more than good API & documentation, we need to:
  • Enable them to inspect the state of the game world and see how it changes during execution
  • Let them try out their changes very easily —— without restarting the server & re-login and revert & replay as they like
• A data-driven ECS design & check-detection based networking solution might be able to make both of them possible

Why should we care?

We all want our server to be used by other people, right?

So why would anyone want to use a third-party server, instead of BDS?

I think there are two main reasons:

• Performance: We might be able to pull off some clever optimizations(like async chunk storage / one world per thread) that makes us faster / has higher capacity than BDS.
• Plugin Ecosystem: Since we are open-sourcing our code, it's certainly easier (and potentially more reliable) to write plugins for our server than wrangling with debug symbols in BDS. So we can have a richer plugin ecosystem for server owners than BDS.

Other factors such as the number of vanilla features supported are also important, but I think we can make small compromises there(as long as we don't miss out on any major features?).

Let's leave performance for another day and focus on the plugin ecosystem for now.

So, we want to have a good plugin ecosystem, and how do we achieve it?

By making plugins easy and fun to write, of course!

Also, if we can make plugin developers happy, we can extend our tool to make developers of vanilla game logic in core server happy as well!

The two overlooked factors & Ways to improve them

This requires us to have good documentation and tutorials, APIs that are easy to learn, easy to use and hard to misuse, make simple things easy and hard things possible and all sorts of technobabble...

But there are two factors that can be easily overlooked:

• Observability of game logic
• Ease of trying out changes

Observability of game logic

Observability of game logic means one should be able to easily see the state of the world and how logic changes it. Since writing a plugin is essentially modifying existing game logic, that means one should know how the original logic works, how to modify it to meet their requirements and how to debug if things didn't work.

To achieve that, I think we should provide tools (at least the possibility of writing one) that acts like a debugger: one can step through parts of logic to visually see how they mutate the state of the world, how their plugin plays out with the rest of the system, etc......

Existing debuggers are more code-oriented and I think it would be great if we can provide something more high-level: see what changes would happen to block XYZ / entity XYZ if we move forward a tick, and allow developers to navigate it using a game window rather than looking at variables and members.

Ease of trying out changes

Ease of trying out changes means one should be able to change something and try it in-game quickly. Short waiting time leads to a happier, more focused and productive programmer. This is especially obvious if one has experience in modern web development: hot reload is a necessity there!

Compared to building websites, writing plugins are painful, after someone changes their code, they need to:

• recompile the code (this can be skipped if you are using interpreted languages)
• restart the server, they usually need to wait for a second or two at least
• rejoin the server, and test whether something works

This is particularly gruesome if you are ironing out some details or testing ways to do something.

To solve this problem, I believe naive code hot-reloading will not work, as the game state and client state synchronization can easily get messed up by the buggy code the programmer is still trying to get to work. We need to reload the code and revert the game state to a known snapshot to achieve stable hot-reloading.

Ideas on implementation

So far, I have proposed two features to improve developer experience:

• A debugger that can step through game logic and see how they change the game state
• A code hot-reloader that reloads code changes, revert game state to known snapshot(and deal with client state sync as well).

They are certainly hard to build, but I think it's not impossible. Since I am writing this to call attention to problems rather than hijacking project architecture, I am only going to give some general ideas.

We can consider taking a more data-driven approach, or even adopt ECS architecture —— if we can separate state and logic, then the state can be (hopefully) easily compared, saved and restored. The concept of systems also forms a good boundary —— developers can put "breakpoints" and see how each system changes the state of the world.

Pushing even further. If we can achieve f(world, actions(might be packets)) = world, then we can easily replay the evolution of the entire world, this might provide benefits in backup, replication and analysis systems such as anti-cheats.

The client state synchronization would be a challenge though, we might be able to get away by using a change-detection based networking solution that produces packets based on what state of the world has been changed?

Conclusion

So, that's all! I argued why we should make core game logic/plugin developers life better, proposed two features that can make life easier and gave ideas on how to implement them.

My goal in writing this is more about pointing out problems and directions than providing a concrete solution. I think this has to be considered while designing the architecture rather than added as an afterthought, and getting this right will give us an edge compared to existing server software.

Thank you for reading! It's not every day that some random dude on the Internet rush in and dump an entire thesis to try to hijack a project lol.

[JustSxm]

allow developers to navigate it using a game window rather than looking at variables and members

Why would you need a game window? Variables and members actually tell much more than can be visually seen.

[johnbanq]

Yes, but that might not be intuitive enough. I think taking a look through a game window is easier than looking at members of Chunk struct. Especially when the user (plugin developer) doesn't know the entire codebase well.

I am thinking using a game-like (can be a simplified, less detailed version) window to show what has changed and what is not and can be used to select what to inspect further.

[john-bv]

Developer Experience is one of the key factors to a software like this, however I think a lot of what you proposed is going to be extremely unrealistic to maintain. More specfically, a built-in replay system. Something this big and diverse should be done outside the core, as it's not really related to the software itself, not to mention the unnecessary performance hits this would cause on a live server. While I would love to implement something like this in the future, outside the core, as a external tool for developing, I definitely don't think it's a necessity.

I would like to mention my current plans for the initial design of the software:

CS Design

Everything about Netrex that can be applied to vanilla will be utilizing a data driven component system, mainly because this will make it easier to allow the Typescript/JS plugin api to modify and create custom components. However, I would like to note, there are plans to limit the ability of accessing network related things from within a plugin, as I believe if it's correctly implemented, no plugin will need to touch it. This doesn't necessarily mean I'm not going to allow things like custom blocks, but rather prevent things like protocol hacks and hacked login handling.

Getting back on track with your suggestion to ECS design; while I was already planning this, I'm going to be putting more time and thought into it as is essential for the future, and will definitely take note of this before I start implementing Entities.

Take aways on a "Replay" system

While I think a replay system in it's entirety is going to be extremely unrealistic to maintain, I do think it's foreseeable to have some sort of "state management" built in. The state system will extend on the design of a Entity Component System and quite simply, "provide a state" to everything: items, blocks, entities, etc. To explain further, while everything in the world will have a state, it's not going to be recorded and kept for long, if you want to do something like this, you should use a plugin.

Plugin Development Experience

I have already worked on a few things for the plugin development experience as I believe it's one of the most key factors into being able to consistently do something. The current things are complete and have yet to be integrated into the software:

• Plugin Archives, a "zip" like file format consisting of the plugin's code, that will be executed during runtime. (These files are validated with a signature) as well as checked in advance to see the kind of calls they make so we can prevent them as they occur

One of the things I was planning already in terms of plugin development that you mentioned was a "Module Reloader" (with deno). While my original thought of implementation for this wasn't as complex as your implementation however I think it will suffice for now as it will simply do all of the heavy lifting work for you, such as automatically reloading your plugin when something has changed. While I would love to implement a system capable of replaying entity states, I think it would be too much complexity to embed into the software and in the future I think i will probably make this as some sort of "internal plugin" that can opitionally be enabled, but it will not be designed into the api.

You mentioned the ability to observe game logic, this might be something I will be able to crunch into the core, but will probably only be accessible to the FFI and TS/JS plugin API, for obvious reasons: such as the in-ability to re-compile rust code efficiently, etc.

I appreciate the time you took to write this and I will definitely be referring to this issue for implementation in the future as I work on things. To crunch things into simpleton terms from what I said above:

• I already plan on making a "Hot module reloader" to help ease development experience, just not in the entireity of your suggestion.

• Extending the first response a little bit, I think that a entire replay / game state managment system embeded into the core is extremely un-realistic and hard to maintain and will probably be a plugin or internal plugin if I ever get around to commiting to a project like that.

• While a debugger was not originally planned I will now be thinking about ways to implement this, however it will only be accessible to the FFI and TS/JS plugin api as it will be easier to work with and provide the least amount of overhead.

• Performance has a lot to do with experience. If there's not good performance, it's going to be a headache to work around design flaws and built in implementations that cause massive overhead

[johnbanq]

I appreciate your detailed response as well!

I agree the replay system should not belong to the core, but I think adding some constraints to the core system design can be immensely helpful (regardless of doing replay or not):

• The game state (or components) should be designed to be easy to serialize & deserialize. They should probably not hold stuff like player connection object.
• The game logic should aim to be something like logic(received_packets, world_state) = (packets_to_send, new_world_state). Aka having logic only use received packets & past world state (that we can easily save and load) to derive packets to send and new world state.

Since we are planning to use something like ECS, I think they should be quite natural & obvious. Having these constraints can not only help replay but also help testing & decoupling as well.

We can be more careful about RNG and threading but I think these should be enough to make the core "replay-ready" (load a world snapshot and start from there) and have all the benefits without introducing much complexity.

But yeah, they are not free and will have some cost in terms of complexity and performance. So I am just suggesting ;)