Unique<T> for thread-safe usage of engine types

[TitanNano]

The Unique<T> type constrains the way non-thread-safe engine types can be used outside the main thread. With these constraints applied, we can safely send types that are wrapped by Unique<T> across threads.

Values that are wrapped by Unique can only be accessed via the Unique::apply or Unique::apply_gd functions. These functions accept a Send + Sync closure to prevent any non-thread-safe values from getting passed into the wrapped types.

Usage Example

std::thread::spawn(|| {
    let node: Unique<Gd<Node3D>> = Unique::new_alloc();
    let child: Unique<Gd<Node3D>> = Unique::new_alloc();
    
    node.apply_gd(move |node| {
         node.add_child(child);
    });

    node // A unique node can be returned by the thread back to the main thread.
});

Breaking Changes

This will very likely break code that relies on the experimental-threads feature. The extent of the breakage needs to be assessed, and ideally a migration path can be offered for all safe use cases.

• ToGodot has a new associated type Threads that has to be added to all manual implementations of the trait. Default values for associated types are still not stable.

To Dos:

• test the implementation with an actual project
  • It's currently possible to pass array and dictionary types to the engine by reference. This breaks the uniqueness constraints when storing such types in a thread-local variable.
• verify_unique_recursive needs more thorough testing.
• check older API version compatibility.

[GodotRust]

API docs are being generated and will be shortly available at: https://godot-rust.github.io/docs/gdext/pr-1524

[Bromeon]

Thanks a lot for the effort, very interesting approach!

Several of the symbols (UniqueType, Array::to_unique, ...) do not appear in generated docs, which makes the PR a bit hard to introspect. Please make sure the full public API is visible, including required bounds 🙂

Since engine methods currently take &AnyArray/&AnyDictionary rather than impl AsArg<...>, the implicit conversions from Unique to array/dict arguments won't work. I'm not sure about making everything generic, as it increases complexity and compile time -- for now there could be an escape hook (explicit conversion) maybe? Although that might open it up for abuse 🤔

Element: ThreadSafeArgContext can be quite a restriction and breaking change, no? Especially after we just opened up Element to custom types for v0.5. The trait ThreadSafeArgContext not having a blanket impl is a problem -- it means that we have yet another trait that has to be implemented manually (yes, #[derive] can do it, but I also anticipate moving to builders and proc-macro less APIs one day, and making them more boilerplaty isn't great 😉). There's some prior art: I made a blanket-impl with AsArg<Variant> for impl ToGodot<Pass=ByValue> -- it would mean removing many explicit impls though (and not sure if that works out for all).

I see that apply() takes exclusive refs &mut T, is there a distinction to allow shared-ref access &T, or should we treat both the same?

[TitanNano]

Several of the symbols (UniqueType, Array::to_unique, ...) do not appear in generated docs, which makes the PR a bit hard to introspect. Please make sure the full public API is visible, including required bounds 🙂

Yeah, docs weren't being rebuilt due to conflicts with master which prevented CI from running.

UniqueType in particular, though, is supposed to be private as it seals the T of Unique to a curated selection. This is not absolutely necessary, as Unique cannot be constructed for arbitrary types, so we could also remove it.

Since engine methods currently take &AnyArray/&AnyDictionary rather than impl AsArg<...>, the implicit conversions from Unique to array/dict arguments won't work. [...]

This is actually an unsolved problem. Explicit conversion creates a gap in the thread safety guarantees. It's possible to smuggle in non-thread-safe values via thread locals or singletons and pass them to an API of a Unique<T>, which breaks the thread-safety guarantees. E.g. this but with an Array or Dictionary.

Element: ThreadSafeArgContext can be quite a restriction and breaking change, no? Especially after we just opened up Element to custom types for v0.5. [...]

Breaking change, yes; restricting, I'm not sure. All user-defined types should be Send + Sync so there shouldn't be an issue with implementing ThreadSafeArg for them.

The blanked impl for ThreadSafeArgContext should be impl<T: Send + Sync> ThreadSafeArgContext for T but the compiler has no guarantee that godot-ffi will not make Opaque send and sync in the future. 😓

That's why I currently have the ThreadSafeArg marker trait, but it still requires manual implementation outside the derive macro.

I see that apply() takes exclusive refs &mut T, is there a distinction to allow shared-ref access &T, or should we treat both the same?

I don't see much use for the apply function besides mutating the inner type. You can't return anything from the closure, so immutable access appears to be pointless so far.

[Bromeon]

UniqueType in particular, though, is supposed to be private as it seals the T of Unique to a curated selection.

That's a reason why the trait should be sealed though, not private 🙂 It can have a private sealed::Sealed supertrait, we do this in a few places. But it would still be good to know its implementors, as it's used in a public bound.

---

This is actually an unsolved problem. Explicit conversion creates a gap in the thread safety guarantees.

AsArg<T> provides implicit conversions and currently no public methods of its own, so it's effectively only usable to forward something to an engine method accepting AsArg<T>. This lack of API seems to be something that the thread-safety relies on, but it's not really guaranteed as of now (it might be useful one day to allow users have their own impl AsArg parameters).

It's possible to enforce such absence, but it will limit AsArg in a way that it wasn't originally designed for, and it feels a bit like mixing unrelated concerns (implicit conversions vs. thread-safety). But I do see the appeal to it. Maybe we should still evaluate multiple options before committing to one, though.

---

The blanked impl for ThreadSafeArgContext should be impl<T: Send + Sync> ThreadSafeArgContext for T but the compiler has no guarantee that godot-ffi will not make Opaque send and sync in the future. 😓

That's why I currently have the ThreadSafeArg marker trait, but it still requires manual implementation outside the derive macro.

Is Opaque the only obstacle here, or are there other coherence issues?

What about other blanket impls, like the one I suggested? It's still semi-manual, but at least most users defining their own types can deal with one trait less that they might ultimately not care about.

---

I don't see much use for the apply function besides mutating the inner type. You can't return anything from the closure, so immutable access appears to be pointless so far.

You cannot return, but it's still possible to transport values outside through thread-safe means (e.g. Mutex, Arc, etc.). But I'm OK with only supporting &mut, if there's another need, we can always extend it.

[TitanNano]

[...] But it would still be good to know its implementors, as it's used in a public bound.

Fair point, I made it public and sealed now.

[...] But I do see the appeal to it. Maybe we should still evaluate multiple options before committing to one, though.

Piggybacking off AsArg currently solves two problems:

1. the Send + Sync closure of Unique::apply is not reliable and only serves as guardrails to nudge users into the correct way of using the type. To have true thread safety, we need to perform a runtime check because the valid usage of many engine types depends on which thread they are being used on. I think we could do this runtime check independently of AsArg, it's just convenient to do it during the conversion.
2. We need to support two sets of function signatures for all engine APIs. One unrestricted one for the main thread and one restricted one for other threads. Doing this via generic argument types or impl Trait seems like the most ergonomic and maintainable solution. If this gets decoupled from AsArg we still would require a generic argument trait to support both "normal" argument types and thread-safe argument types.

Is Opaque the only obstacle here, or are there other coherence issues?

What about other blanket impls, like the one I suggested? It's still semi-manual, but at least most users defining their own types can deal with one trait less that they might ultimately not care about.

It's not Opaque per se. We want to differentiate between Send and !Send types at runtime, and this is a real pain because user-defined !Send types in stable Rust are always implicit. The compiler does not trust that a type will remain !Send so we would need a negative trait bound to specify that the blanket implementation only applies to something like !ExplicitlyNonSend. Even when I move ThreadSafeArgContext into godot-ffi the error just moves up the chain, and the compiler complains about PhantomData<*const u8> possibly being Send in a future release.

Using T: AsArg<Variant> wouldn't be able to select only Send types. We want a blanket implementation for all thread-safe types and a specific one for the others.

We had the same issue when dealing with the signal arguments for the SignalFuture 😞. I keep thinking about this problem, but I haven't found a better solution so far.

[TitanNano]

I have now added an additional associated type to ToGodot which resolves our trait coherence issue. It is still a breaking change, as it requires this associated type to be added to any manual implementation of ToGodot.

Additionally, I have moved the runtime thread-safety validation into codegen so it is decoupled from AsArg. This is more involved than the previous approach but also covers much more of the API surface. Unfortunately it also breaks all non-main thread signal emission since all types are erased when calling Object::emit_signal. I think I will have to add an internal version of Object::emit_signal that skips the thread-safety validation so we still can correctly emit typed signals.

[Bromeon]

Thanks for keeping this up-to-date! There's probably still quite a few things to be looked at holistically, but I commented again on some parts to get a more detailed understanding on the approach 🙂

[Bromeon]

This is quite restrictive, is this method still useful in practice?

• Gd<Node> etc isn't necessarily owning, pointing to nodes in the scene tree is harmless
• No arrays/dictionaries excludes a ton of use cases

At the same time it's also not airtight:

• You don't list Variant which can contain anything else
• Any fields that aren't #[var] are invisible
• Gd strong refs can be trivially worked around with InstanceId weak refs -- although this can be OK if from_instance_id is sound
• The recursion hits a stack overflow when you have cyclic references

I'm not saying it needs to be perfect -- for threading we definitely have to make compromises, and best-effort is better than nothing. But we should probably state a clear goal here, and then depending on that decide how far we want to go.

It's also worth noting that such a recursive check can be extremely expensive at runtime.

[TitanNano]

This is quite restrictive, is this method still useful in practice?

This is something I'm currently evaluating inside my own project. My theoretical use case here is that this should work well for engine built-in resources like materials, textures, animations, and so on.

You don't list Variant which can contain anything else

Variant types are being inspected during conversion.

Any fields that aren't #[var] are invisible

Yes, and that's why only engine ref-counted classes without a script are allowed. It's even more restrictive than you thought. 😁 I see that this detail is not yet covered by the description.

Gd strong refs can be trivially worked around with InstanceId weak refs -- although this can be OK if from_instance_id is sound

Yes, but I don't think engine classes store InstanceIDs; at least I haven't come across one yet. So it should be covered by the previous point. try_from_ref_counted is now also restricted to the main thread as well.

The recursion hits a stack overflow when you have cyclic references

Good point. I should probably guard against that. So far I'm just trusting that engine classes don't have cycling references.

[TitanNano]

This now also tracks already visited InstanceIds to prevent cyclic references from causing stack overflows.

[Bromeon]

What about this?

It's also worth noting that such a recursive check can be extremely expensive at runtime.

We should probably bench it to see how much overhead this check introduces for a realistic scene/nested setup. And consider moving it behind safeguards-strict gate. It's best-effort either way...

[TitanNano]

We should probably bench it to see how much overhead this check introduces for a realistic scene/nested setup. And consider moving it behind safeguards-strict gate. It's best-effort either way...

We can benchmark it on some resources, but this check is completely voluntary. You have to manually call Unique::try_from_ref_counted(object) or the check never runs. You also can't run it on a node tree because they are manually managed, and we can't prove that we have unique access. It's really only meant for resources like textures, meshes, materials, and such.

[TitanNano]

We could also make Unique::new_unchecked public, as an alternative, but I think that has big potential for abuse.

[Bromeon]

I see. We currently have Gd::try_to_unique(), but it's just the object's refcount itself, not any nested parts. Do you think the 2 APIs should be aligned over time?

---

Also, maybe the docs should mention that the heuristic is deliberately conservative to err on the side of "not unique", even if that means many actually-unique objects won't pass it. Example:

Reference count of any property value is > 1.

An object with 2 object properties that are the same is unique, but won't pass this check. Of course, many things like scripts, Callables etc. are harmless as well but hard to prove statically.

---

We could also make Unique::new_unchecked public, as an alternative, but I think that has big potential for abuse.

Eventually we might need it, but for now I'm fine not exposing it.

[TitanNano]

I see. We currently have Gd::try_to_unique(), but it's just the object's refcount itself, not any nested parts. Do you think the 2 APIs should be aligned over time?

Yes, I think they should be aligned. With the new Unique::try_from_ref_counted the existing Gd::try_to_unique might be confusing.

[ValorZard]

This is a drive by comment, but can you downcast a node stored in a unique?

like, can you cast a Unique<Node> to Unique<MyCustomClass>

[TitanNano]

This is a drive by comment, but can you downcast a node stored in a unique?

like, can you cast a Unique<Node> to Unique<MyCustomClass>

At the moment that is not supported. Do you see a use case for downcasting here? Unique<Gd<NodeType>> can only be obtained by creating a new node. My assumption so far was that users would simply create the node they need, modify it, and then pass it to the main thread.

[TitanNano]

I think this is finally ready for review.

We probably should put this behind an opt-out feature and give people the ability to go back to the current experimental-threads. There is not really an incentive to enable this new behavior if we make it opt-in.