Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(wallet): allow PersistedWallet to be Send + Sync #1874

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

notmandatory
Copy link
Member

@notmandatory notmandatory commented Mar 6, 2025

Description

fixes #1873 based on solution proposed by @praveenperera .

Notes to the reviewers

This is not a breaking change since it's only changing the internal _marker field's type.

Changelog notice

  • Fix PersistedWallet to be Send + Sync, even when used with a !Sync persister type such as rusqlite::Connection.

Checklists

All Submissions:

  • I've signed all my commits
  • I followed the contribution guidelines
  • I ran cargo fmt and cargo clippy before committing

Bugfixes:

  • This pull request breaks the existing API
  • I've added tests to reproduce the issue which are now passing
  • I'm linking the issue being fixed by this PR

@notmandatory notmandatory self-assigned this Mar 6, 2025
@notmandatory notmandatory added the bug Something isn't working label Mar 6, 2025
@notmandatory notmandatory added this to the 1.2.0 milestone Mar 6, 2025
@notmandatory notmandatory marked this pull request as ready for review March 6, 2025 03:27
@notmandatory notmandatory force-pushed the fix/persistedwallet_send_sync branch from a9ef2af to 14152c0 Compare March 6, 2025 03:36
@YuriQueirozAndrade
Copy link

ACK, i run build, test, clippy and fmt. All looks goods

@notmandatory notmandatory force-pushed the fix/persistedwallet_send_sync branch from 14152c0 to ee4e2c8 Compare March 6, 2025 14:53
Copy link
Contributor

@oleonardolima oleonardolima left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm quite divided on this one, still taking a look in SQLite docs, but it doesn't feel right to override the Send + Sync implementation for any given persister.

I think there's no need for this change and it should inherit the Send + Sync implementation as it's patterns and use a Mutex instead, there's a few issues/discussions on rusqlite about this and it seems their decision is related on how sqlite flags are used, see rusqlite/rusqlite#188.

@praveenperera
Copy link
Contributor

praveenperera commented Mar 6, 2025

@oleonardolima when you call persist you have to pass in the Connection if this Connection is stored in the same struct as the Wallet you still need to wrap the Connection in a mutex.

Without this change both the Connection and the Wallet would both need to be wrapped in a mutex for no reason.

Example without this change

#[derive(Debug, uniffi::Object)]
pub struct Wallet {
    pub id: WalletId,
    pub network: Network,
    pub bdk: Mutex<bdk_wallet::PersistedWallet<Connection>>,
    pub metadata: WalletMetadata,
    db: Mutex<Connection>,
}

Example with this change:

#[derive(Debug, uniffi::Object)]
pub struct Wallet {
    pub id: WalletId,
    pub network: Network,
    pub bdk: bdk_wallet::PersistedWallet<Connection>,
    pub metadata: WalletMetadata,
    db: Mutex<Connection>,
}

There isn't a safety issue here because the wallet doesn't actually store the Connection which is what is not thread safe. The only reason the Connection is included in the type at all is to separate it from other persister types.

@@ -120,7 +120,7 @@ pub trait AsyncWalletPersister {
#[derive(Debug)]
pub struct PersistedWallet<P> {
inner: Wallet,
marker: PhantomData<P>,
_marker: PhantomData<fn(P) -> P>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
_marker: PhantomData<fn(P) -> P>,
_marker: PhantomData<*const P>,

According to @LLFourn

Copy link
Member Author

@notmandatory notmandatory Mar 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this approach and get this error running test_thread_safety:

error[E0277]: `*const Connection` cannot be sent between threads safely
    --> crates/wallet/tests/wallet.rs:4215:19
     |
4215 |     thread_safe::<PersistedWallet<bdk_chain::rusqlite::Connection>>();
     |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `*const Connection` cannot be sent between threads safely
     |
     = help: within `PersistedWallet<Connection>`, the trait `Send` is not implemented for `*const Connection`
     = help: the trait `Send` is implemented for `Connection`
note: required because it appears within the type `PhantomData<*const Connection>`
    --> /Users/steve/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/marker.rs:757:12
     |
757  | pub struct PhantomData<T: ?Sized>;
     |            ^^^^^^^^^^^
note: required because it appears within the type `PersistedWallet<Connection>`
    --> /Users/steve/git/notmandatory/bdk/crates/wallet/src/wallet/persisted.rs:121:12
     |
121  | pub struct PersistedWallet<P> {
     |            ^^^^^^^^^^^^^^^
note: required by a bound in `thread_safe`
    --> crates/wallet/tests/wallet.rs:4212:23
     |
4212 |     fn thread_safe<T: Send + Sync>() {}
     |                       ^^^^ required by this bound in `thread_safe`

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@notmandatory Sorry yeah my suggestion does the complete opposite of what we need.

@notmandatory
Copy link
Member Author

notmandatory commented Mar 7, 2025

According to Llama 3.3 70B I should use PhantomData<fn(&mut T)>:

"In this case, you want to ensure that your struct remains Send and Sync, even if the functions implemented on it have a &mut T parameter and T is not Sync.

To achieve this, you can use PhantomData<fn(&mut T)> instead of PhantomData<fn(T)>. This tells Rust that your struct is a "consumer" of &mut T, but it does not own T and therefore should not be tied to its lifetime."

@notmandatory notmandatory force-pushed the fix/persistedwallet_send_sync branch from ee4e2c8 to 09e3178 Compare March 7, 2025 00:38
@notmandatory
Copy link
Member Author

I updated the code per comment above and also included some of the explanation in the commit message.

@notmandatory notmandatory force-pushed the fix/persistedwallet_send_sync branch from 09e3178 to b525a92 Compare March 7, 2025 00:42
…!Sync

The goal of this change is to ensure that PersistWallet<P> remains Send and Sync, even if the functions implemented on it
have a &mut P parameter and P is not Sync.

The reason to change PersistWallet<P>'s _marker field type from PhatonData<P> to PhantomData<fn(&mut P)> is to tell the
Rust compiler that this struct is a "consumer" of &mut P, but it does not own P and therefore should not be tied to its
lifetime.
@notmandatory notmandatory force-pushed the fix/persistedwallet_send_sync branch from b525a92 to 3c2fc86 Compare March 7, 2025 00:43
The latest ring version 0.17.13 changed their MSRV from 1.63 to 1.66.
The bdk_esplora client depends on ring.

ring v0.17.12
├── rustls v0.21.12
│   ├── minreq v2.13.2
│   │   ├── esplora-client v0.11.0
│   │   │   ├── bdk_esplora v0.20.1
@praveenperera
Copy link
Contributor

@notmandatory Perplexity seems to agree too

Screenshot 2025-03-06 at 7 15 55 PM

@notmandatory
Copy link
Member Author

I pushed another commit to fix the CI issue. Ring just (4hrs ago) bumped their MSRV from 1.63 to 1.66 with a patch release 😞 (see briansmith/ring#2449).

@praveenperera
Copy link
Contributor

praveenperera commented Mar 7, 2025

@notmandatory I think in our case since we aren't dealing with lifetimes of any unsafe things, I think all three are practically equal, all of this compiles. I think the AIs are wrong.

PhantomData<fn(P) -> P>
PhantomData<fn(&mut P)>
PhantomData<fn(P)>
use std::marker::PhantomData;
use rusqlite::Connection;

pub struct PersistedWallet<P> {
    _marker: PhantomData<fn(&mut P)>,
}

impl<P> PersistedWallet<P> {
    pub fn new() -> Self {
        Self {
            _marker: PhantomData,
        }
    }
    pub fn persist(&self, _wallet: &mut P) {}
    pub fn persist_2(&self, _wallet: P) {}
}

pub struct PersistedWallet2<P> {
    _marker: PhantomData<fn(P)>,
}

impl<P> PersistedWallet2<P> {
    pub fn new() -> Self {
        Self {
            _marker: PhantomData,
        }
    }
    pub fn persist(&self, _wallet: &mut P) {}
    pub fn persist_2(&self, _wallet: P) {}
}

pub struct PersistedWallet3<P> {
    _marker: PhantomData<fn(P) -> P>,
}

impl<P> PersistedWallet3<P> {
    pub fn new() -> Self {
        Self {
            _marker: PhantomData,
        }
    }
    pub fn persist(&self, _wallet: &mut P) {}
    pub fn persist_2(&self, _wallet: P) {}
}

pub type Wallet = PersistedWallet<Connection>;
pub type Wallet2 = PersistedWallet2<Connection>;
pub type Wallet3 = PersistedWallet3<Connection>;

fn thead_safe<T>(_wallet: T)
where
    T: Sync + Sync,
{
}

fn main() {
    thead_safe(Wallet::new());
    thead_safe(Wallet2::new());
    thead_safe(Wallet3::new());
}

@notmandatory
Copy link
Member Author

notmandatory commented Mar 7, 2025

If all are safe and enable the same functionality then I think the one suggested by the AIs is still the best since it communicates that PersistWallet "consumes" the &mut P type in it's functions without owning P. But happy to change it to one of the other options if there's some compelling reason to.

@praveenperera
Copy link
Contributor

That makes sense to me

Copy link
Member

@evanlinjin evanlinjin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK cdba39d

TIL something about PhantomData.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Needs Review
Development

Successfully merging this pull request may close these issues.

Make Sqlite PersistedWallet<Connection> Send+Sync
5 participants