The definitive Particle Effect rework

[BMagnu]

The definitive Particle Effect rework

Okay, frankly the particle spawning system has some issues. This is my attempt to clean it up once and for all, in a way that is at least more future proof than the current system.
Many thanks also to @Kestrellius, for helping me testing and debugging this PR.
For whoever might be reviewing this: I recommend looking at the following files as if they were entirely new instead of trying to dig through the not-very-helpful diff:

• code/particle/ParticleEffect.h
• code/particle/ParticleSource.cpp
• code/particle/ParticleSource.h

The current state and its problems

To understand both the need for this PR and the design goals, let's first discuss the issues I see with the current system, both from a maintainance and code extending standpoint, as well as from a user-facing standpoint.

Needless separation of Effect Types

Currently, particle effects tabled through -part.tbm files need to be one of 5 different types:

• Single
• Composite
• Cone
• Sphere
• Volume

This is suboptimal from both a code perspective, as well as from a user perspective. From a code perspective, this constitutes a lot of duplicated code, since large parts of the handling of these types is identical between them. After all, a Single-type particle is effectively identical to a Sphere-type particle with radius 0. Furthermore, Composite-type particles especially are a pain-point in the current system, as they behave sufficiently different from all other types to need special handling in many places. While most of these types are currently handled as polymorphic, Composite alone needs to be checked against in many places, defeating the purpose of using polymorphism for effect types.
Furthermore, adding new features to particles typically requires a similar if not the same change in all effect types, adding unnecessary maintainance overhead.

From a user perspective, the main drawback originating from the seperated effect types is missing flexibility. Right now, a lot of particle effect options are mutually exclusive. An example of this would be the velocity pattern of the Cone-type effect with the particle's themselves spawning in a spherical volume as would be the case in a Volume-type effect. Fundamentally, the only thing that is causing this mutual exclusivity is the type separation. There is no inherent cause why a particle effect could not have both of these properties. As such, the given type separation significantly reduces flexibility and possible applications for users.

Layers and layers of Particle Effect systems

Another highly significant issue for both maintainability and usability is the current layering of the particle system. To give a quick overview, currently, the following layers exist:

0. Basic particle handling on the backend, rendering, movement, etc.
1. Legacy particle emitters, typically used for particles tabled in the ship table directly.
2. Legacy particle emitters, wrapped within the "current" modern particle effect system.
3. Proper modern tabled particles.

As is evident, there is a significant stack of legacy particle handling in various states of modernization, all concurrently maintained and adding to the code bloat of the overall system.
Additionally, which layer any given particle is in has a significant effect on usability, both causing overall confusion to users, and reducing usability for the less capable types.
Layer 0 is mostly a backend layer which all other systems implicitly rely on, however there are a few things that directly create particles in layer 0, notably scripting and pspews.
Layer 1 mostly contains very old particles tabled directly in the ships or weapons table, such as ship split particles, damage spew, thruster particles, and so on. Notably, most of the layer 1 particles that have not been converted to layer 2 seem to be modified per-frame dependent on the current gameplay state. This is what I presume to be the reason for these particles not to be updated to layer 2 handling, as that requires a static particle effect.
Layer 2 is primarily two different wrappers around the layer 1 code (and thus also requires the level 1 code to be present and maintained). The main difference is that it adheres to the static effects of the modern particle effect system. This also allows this type of particle effect to be replaced by a modern layer 3 particle effect if tabled properly.
Layer 3 is the modern particle effects which are read from -part.tbm files, and are handled by the modern particle code which only relies on layer 0.
Not only are these different layers confusing, but the inability to use -part.tbm particle effects for layer 1 (and layer 0) particles strongly diminishes their usefulness.
In addition, the interdependence of the layers on one another makes maintainance fairly hard and painful.

Code redundancy

Perhaps due to the organic growth of the system, several bits of code (even outside the different particle effect types) are strongly redundant.
Especially for calculating the position and orientation of particle effect sources, global as well as local to potential parents, there is lot of duplicate code.
In addition, the submodel stack of a given model that spawns particles is potentially traversed multiple times for various different computational steps, when once would have been sufficient.
Furthermore, everything that could be queried from something that spawns particles is currently handled in its own function, with a large switch-case statement, repeating all possible particle source hosts every time instead of using polymorphism to properly abstract this.
Because of the complexity of this code, a few possible combinations of global/local coordinates in either in- and/or output are simply broken and will yield incorrect particles.

Insufficient separation of code responsibilities

Big parts of the particle code are stateful. That, in and of itself, is not a problem. However, these states tend to be modified as side effects of functions in very different parts of the code.
This makes understanding the code as well as maintaining the code very difficult. Things modifying the state of their callers in insufficiently documented side-effects is a common feature in the current particle code, which should be reduced for better code readability.

The approach for a redesign

Now that the issues with the existing system are discussed, we can get into the four core redesign aspects I have identified and implemented.

One Effect Type to rule them all

The core improvement over the existing system is the introduction of a new particle type. This "Omni" particle type is effectively a superset of all previously existing particle type.
This includes the tableable particle types, as well as legacy particles handled in layer 1 and 2. This has a few critical advantages:
First, all existing particle types can be removed. Since all particle types, such as the Sphere-type, only have a subset of the features of the new Omni-type, all existing effects can be specified as an Omni-type effect. In fact, this is what the parsing code for the -part.tbm does with this PR. Instead of parsing them as before, it will now use the parsed data to construct an Omni-type effect instead. This allows handling the new particle system as a replacement to layer 3, instead of as an additional fourth layer.
The same thing applies to layer 1 and layer 2 particles. The Omni-effect has all necessary configuartion options to represent all existing particle spawning methods used in layer 1 and 2. As such, the parsing code for these particles is similarly changed to construct Omni-type effects instead of the legacy effects. This allows us to entirely remove layers 1 to 3 of the current particle system, instead of just building on top of them. The problem of dynamic adaption of effects to the current gameplay situation is resolved by adding modular curves to particles, such that the adaption to gameplay can be handled as data instead of as hardcoded behaviour.
Overall, this new Omni-effect is thus able to completely eliminate the redundant code between effect types, and is able to significantly declutter the rest of the particle spawning system.
Furthermore, since one effect type is now able to handle everything, a user tabling this new effect is now able to combine all behaviours instead of having to decide between multiple mutually exclusive types.
The only prior effect type not directly handleable through this Omni-type effect is the composite effect. As the composite effect itself was handled very differently prior to this PR as well, it felt sensible to just properly differentiate the behaviour of "multiple effects per source" from just tabling different effects.
As a result, this PR now handles all effects as composites. Any effect ID which can be used to create a particle spawner now refers to a vector of particle effects instead of a singular effect.
As such, a non-composite effect is simply a composite containing one effect, and the code always expects that a given effect ID could imply any number of effects being spawned from a given particle source.

Abstracting Volumes

In order to facilitate the Omni-type effect, it is required to be able to specifiy volumes, both for the velocities of prior effect types like the Sphere or Cone-type, as well as for the particle positions of the Volume-type. As such, this PR now abstracts the concept of "Volume for particles" independent of how they are used. Currently, three types of volumes are known to the system:

1. Spheroids: Spheres which can be potentially streched, as well as with configurable density bias between the edge or center of the spheroid.
2. Cones: Cones can be configured to spawn particles in a given angular deviation from the given orientation, at a certain distance from the origin.
3. Legacy Axis-Aligned Cuboids: This type of volume cannot be parsed and used by end-users, and is instead just used to achieve the same volumes as used in certain old level 2 / 3 particle effects. It tends to result in either offset axis-aligned cuboids or highly irregular density distribution hollow spheres, depending on its usage.

Abstracting Effect Hosts

This PR also rips out the multitude of switch-case statements for various effect hosts (like object / existing particle / just a free floating vector) and replaces it with polymorphism.
This allows code maintainer to just create a new class instead of having to search for many switch-case statements to edit, some of which need to be closely matched to one another.
It also fixes some bugs where this exact matching wasn't done properly in some cases. Unlike the volumes, which need particle specific code to facilitate some of the modular curves, the effect hosts are sufficiently generic that they could be easily adapted to work for other things that are attached to objects or the like.

General decluttering

Additionally, this PR tidys up some of the more egregious issues as well as remove a lot of unnecessary boilerplate. Notably, this results in a clear hierarchy and process:

1. Some part of the code requests a particle source to be created with a specific effect, at a specific host
2. The particle source checks the tabled timing of all of its effects
3. If any of its effects is supposed to emit, it will give the respective effect a read-only reference to the current state and particle host data
4. The effect spawns a number of particles as is required by the tabled data
5. The particle source updates its state, and returns to 2. in the next frame

The notable change to before is that the only thing that has a modifiable state is the particle source.
And the only thing that modifies this state is the particle source itself. Neither does the particle spawning in 4. change the timing state (as it did before), nor does the result of 4. have any effect on whether the particle source will continue to spawn particles in the future. This is all managed by the particle source in one single function, greatly improving code readability, reducing complexity, and fixing some long-standing bugs.

Effective changes with this PR

The overall result from the previously discussed points is fourfold. Less code (around 1000 lines), even though a lot of functionality was added and the new parsing code and hardcoded definitions to adapt legacy particles takes up a lot of lines (even if simple ones). Less complexity and a lot better maintainability. More functionality, as will be discussed later. And finally, several bugfixes, as will also be discussed later.

Added User-Facing Functionality

To get this out of the way first, even with the new parsing options and significant changes, all existing tables work and will continue to work with no significant behavioural change.

• All existing options for particles can now be used at the same time, regardless of the effect type they belonged to in the prior system.
• All existing particles of layer 1 and 2 can now use tabled particles in -part.tbm. This notably includes thruster particles, damage spew, impact spew, and ship split particles, among others.
• All possible sources of particle velocity, from tabled volumes, source orientation, host velocity, and otherwise can be scaled by all types of random distributions.
• Several prior restrictions have been relaxed, such as allowing onetime effects with delay.
• The volume definitions have become slightly more flexible.
  • Closes Particle Effect FR: Ability to specify a 'minimum angle' for cone-type effects, similar to that recently added for weapon submunitions #2937, as curve-based random distributions can now be used for the angular deviation in the cone volume.
• Modular Curves can be used to make the otherwise statically tabled particles adapt to both the current gameplay situation (such as host hitpoints) as well as being more flexible overall (such as scaling with host size), see FR: Possible applications for curves #5365.

Fixed Bugs

• Fixes Delayed particle effects get banked, then released all at once #5653.
• Fixed Local To Parent type particle sources which had global target orientation overrides (which, given how new local to parent is, aren't used in many mods yet luckily).
• Fixed tabled particle offset for particle sources with certain global target orientation overrides.
• Fixed the auto-generated shield_impact_explosion_radius. Before, it was being set to a randomized value from the given distribution at parse, instead of the expected value for the random distribution.

Outstanding TODO's and future enhancements

In decreasing order of importance

• Some things, notably pspews, are directly creating layer 0 particles in the code. These should be overhauled to also be representable using the Omni-type effect. This should not be too hard, but given that layer 0 needs to continue existing anyways for the particle backend, it is punted to a future PR as this one is too large already anyways.
• Once the layer 0 particle spawners are integrated into this system, it is easy to give the particle an awareness of which particle effect spawned it. This allows the particle struct to become more leightweight (as it doesn't have to copy the data and can just reference the table entry), as well as allows the particle to become more flexible.
  • Given this PR, the particle itself could be granted access to its own modular curves struct (stored in the effect itself, however), greatly enhancing the capabilities of already-spawned particles. This will also facilitiate Particle Effects FR: Ability to set particle length based on current velocity #3052.
  • If a particle has easy and cheap access to the effect it was spawned from, Particle Effects FR: Spawn new effect on particle death #3054 becomes feasible without too much hassle.
  • This also facilitates cleaning up the particle struct itself, reducing the memory burden of the particles, as it can access the static data from its definition instead of requiring a copy.
• Integration of this system into scripting. See FR: API tools for tabled particle systems #5264.
• Adding more common volume definitions. Especially interesting would be a volume defined by a tabled curve rotated around the given orientation in order to facilitate maximally flexible and user-defined volumes.
• Converting particle-like systems to use actual particles, such as muzzle flashes (see Particle Effects FR: Merging muzzleflash system with particle/action system #3049).
• Extending the list of inputs for the modular curves, in order to allow more dynamic adaptation of the particles.
  • Will enable proper implementation of FR: Particle effects for thrusters #5727.
• Add a flag so that the cone volume to be truly round instead of squareish as is now (I don't want to hard fix this now, since it would change behaviour for all existing Cone-type effects)