Rake tasks and background jobs: patterns for iterating over tenants

[flavorjones]

After chatting with @andrewmarkle this morning, I wanted to open a thread to see what patterns people have noticed in their apps around running tasks or jobs for multiple tenants.

Maybe if I give a specific example it will be easier to have a conversation about it.

In the app I'm currently helping build, we have a job that deletes all "due" cards by calling:

Card.auto_close_all_due

But we want this to run for all tenants! So the actual job looks like:

class Card::AutoCloseAllDueJob < ApplicationJob
  def perform
    ApplicationRecord.with_each_tenant do |tenant|
      Card.auto_close_all_due
    end
  end
end

That is, the job loops over all the tenants when it runs. Because it's idempotent, I don't care if the job re-runs.

I can easily imagine another pattern, though, which is to kick off a job for each tenant, something like:

class Card::AutoCloseAllDueJob < ApplicationJob
  def perform
    Card.auto_close_all_due
  end
end

# config/recurring.yml
production:
  auto_close_all_due:
    command: "ApplicationRecord.with_each_tenant { Card::AutoCloseAllDueJob }

For database tasks, there is a similar set of problems: how to indicate whether an application's rake task should be run over all tenants? One idea is to add a rake task argument:

bin/rails close_due[all_tenants]

or add a scoped subtask:

bin/rails close_due:tenants

but neither feels great to me.

Does anybody have thoughts on what an API might look like for Active Job and what task declaration style might work for the database tasks, to indicate "add magic here for all tenants"?

[raphaeljlps]

This is great, looking at my codebase we always set the tenant before running the job (we don't have jobs that apply to all tenants)
With regards to the:
That is, the job loops over all the tenants when it runs. Because it's idempotent, I don't care if the job re-runs.

Could we perhaps leverage the Active Job Continuations here to track tenants if we don't have idempotent job runs?

Regarding the question about api for rails tasks, I'd prefer to be explicit here to show the intent that is to apply to all tenants

bin/rails close_due:all_tenants

bin/rails close_due:single_tenant

[flavorjones]

Could we perhaps leverage the Active Job Continuations here

@raphaeljlps You should be able to use Continuations just fine -- I was just saying, in my case, I'm not using them because the job is idempotent.

I'd prefer to be explicit here to show the intent that is to apply to all tenants

Thank you for your thoughts! I'm still thinking hard about how to adapt existing rake tasks.

[raphaeljlps]

good to know that's possible! Thank you for the work @flavorjones

[andrewmarkle]

I've been thinking about this a bunch and I think I'm leaning towards just setting up an ENV variable. Similar to how you can use the RAILS_ENV in a rake task. That seems pretty natural to most Rails developers and it theory it's pretty simple to implement (we don't need to add an arg to all the rake tasks). Haven't tested if this works but maybe ARTENANT already does this?:

ARTENANT=foo bin/rails my:task

That would solve most of the problems. Maybe we could improve the error message so it's clear to devs what to do. For our app though we're setting up regions as tenants. So I might make an alias to make it really clear what you're doing. Maybe something like:

REGION=us bin/rails my:task

[asmorris]

I quite like the idea of having an env var be the way you can differentiate between tenants, as that also allows for various forms of pattern matching within the env var for different scenarios. i.e

1. If you want to do multiple tenants at once can do comma separated like so:
   TENANT='foo,bar,baz' bin/rails my:task
   and split in your task with

tenants = ENV["TENANTS"].split(",").map(&:strip)

2. if you want to match based on a regexp, you can still do so
   TENANTS='/^prod_/' rails my:task
   and something like this:

if ENV["TENANTS"].start_with?("/")
  pattern = ENV["TENANTS"].gsub(/^\/|\/$/, "")
  tenants = Regexp.new(pattern)

in your task. With that ability, you should be able to account for subdomain patterns like so: region1.production.domain, region2.production.domain or what have you, as well as one fell swoop for all staging or testing or similar subdomains if you need that.

[flavorjones]

@andrewmarkle Yes, ARTENANT was created to serve this purpose. If that's sufficient for everyone's needs, then it's probably fine to keep it (and document it!).

But keep in mind that not every database task has been reimplemented (yet) to support that env var, which is why I'm asking if folks are OK with this!

[flavorjones]

@asmorris I like the idea of expanding the use of an enviroment variable (like ARTENANT) to match against several tenants. If anyone else would use such a feature, I'd love to hear from you!

[lairtonmendes]

@flavorjones Should this already be working today?

production:
  auto_close_all_due:
    command: "ApplicationRecord.with_each_tenant { Card::AutoCloseAllDueJob }

I'm trying something similar, but I'm running into the error below.

[lairtonmendes]

Forget it, I found the issue, I was forcing the execution of the recurring jobs through Mission Control, and the request was going through the middleware (TenantResolver) hahaha.
The good news is that the gem is actually working, once the tenant is set, you can't swap it anymore 😄 .