Support for the solid gems?

[leoncruz]

Hey Mike! Thanks for the incredible work!

I would like to know if there are plans to support the solid gems too. All three have support to use multiple database, but the tenant creation is not supported yet.

I'm thinking something like:

# database.yml
production:
  primary:
    <<: *default
    database: storage/production/%{tenant}/main.sqlite3
    tenanted: true
  cache:
    <<: *default
    database: storage/production/%{tenant}/cache.sqlite3
    migrations_paths: db/cache_migrate
    tenanted: true
  queue:
    <<: *default
    database: storage/production/%{tenant}/queue.sqlite3
    migrations_paths: db/queue_migrate
    tenanted: true
  cable:
    <<: *default
    database: storage/production/%{tenant}/cable.sqlite3
    migrations_paths: db/cable_migrate
    tenanted: true

And then ApplicationRecord.create_tenant("mike") create the databases and migrate all then.

[flavorjones]

Hi @leoncruz. Thanks for asking about this.

I'd like to know more about your use case, if you wouldn't mind explaining. One important point I'd like to understand is whether the tenant for each of these databases the same name, or are they different names?

If they are the same names: have you considered putting all of the solid databases into the same sqlite file/database? If that wouldn't work for you, can you help me understand why not?

If they are not the same names: I've already started on support for this, but it will require an upstream change to Rails that's awaiting review: rails/rails#55927

[flavorjones]

However, each of those subsystems will also need some changes in order to work as tenanted. For example, making solid queue "tenanted" would mean making the supervisor, dispatcher, and worker processes understand tenants. It's not as simple as simply making the database tenanted. Does that make sense?

[leoncruz]

Hi @flavorjones!

One important point I'd like to understand is whether the tenant for each of these databases the same name, or are they different names?

The tenant does not have same name. Each tenant has his own folder. Moving the others databases to this folders will not hava name conflict, so to tenants can have the same database name. Something like:

storage/
  production/
    mike/
      main.sqlite3
      queue.sqlite3
      ...
    leon/
      main.sqlite3
      queue.sqlite3
      ...

If they are the same names: have you considered putting all of the solid databases into the same sqlite file/database?

I hadn't thought about that! It's actually a good point. I will try later. Since I'm working on a new product, I don't have specific usage metrics to share.

For example, making solid queue "tenanted" would mean making the supervisor, dispatcher, and worker processes understand tenants

Yes, I think solid_queue will be more difficult to implement this. I'm not sure either about solid_cable. I'm searching through the source code of theses to understand better.

[flavorjones]

In the app I'm working on, we're leaving solid queue, cable, and cache in an untenanted database, and that's working fine for us performance-wise.

Side note, I am moving the application's solid queue database into a tenanted database but it's related to database replication and perhaps not relevant to your question.

If it's interesting, I also have a branch where I have added callbacks to #with_tenant and #current_tenant= which might make it possible to set multiple tenants at once, for example:

class ApplicationRecord < ActiveRecord::Base
  class << self
    set_callback :with_tenant,        :around, :with_solid_queue_zone
    set_callback :set_current_tenant, :after,  :set_solid_queue_zone

    def with_solid_queue_zone(&block)
      SolidQueue::Record.with_tenant(solid_queue_zone, &block)
    end

    def set_solid_queue_zone
      SolidQueue::Record.current_tenant = solid_queue_zone
    end
  end
end

which will set the SolidQueue::Record tenant whenever the ApplicationRecord tenant is set. LMK if you'd like to try that branch out? I still need to write tests but otherwise it's workable.

[leoncruz]

I think this is great! Could be applied to SolidCable and SolidCache too?

[flavorjones]

@leoncruz I haven't looked into those gems, but I imagine something like I describe above would be a good start.

Also, I've merged the callback support into main in #247

[davidhan527]

I have a related question. Most applications have some sort of cron job/periodic job to perform a task across tenants. Is that supported with activerecord tenanted and solid queue? Thank you for your work!

[flavorjones]

Definitely supported, but I think we could make it more idiomatic and easier to do. Please take a look at #195 where some patterns for iterating over tenants are discussed.