Release v2.12.0

CHANGELOG.md

@@ -1,284 +1,122 @@
-# Changelog
+# Changelog for Oban v2.12
 
 _🌟 Looking for changes to Web or Pro? Check the [Oban.Pro Changelog][opc] or
 the [Oban.Web Changelog][owc]. 🌟_
 
-Oban v2.11 requires a v11 migration, Elixir v1.11+ and Postgres v10.0+
+[Oban v2.12 Upgrade Guide](v2-12.html)
 
-[Oban v2.11 Upgrade Guide][upg]
+Oban v2.12 was dedicated to enriching the testing experience and expanding
+config, plugin, and queue validation across all environments.
 
-Oban v2.11 focused on reducing database load, bolstering telemetry-powered
-introspection, and improving the production experience for all users. To that
-end, we've extracted functionality from Oban Pro and switched to a new global
-coordination model.
+## Testing Modes
 
-## Leadership
+Testing modes bring a new, vastly improved, way to configure Oban for testing.
+The new `testing` option makes it explicit that Oban should operate in a
+restricted mode for the given environment.
 
-Coordination between nodes running Oban is crucial to how many plugins operate.
-Staging jobs once a second from multiple nodes is wasteful, as is pruning,
-rescuing, or scheduling cron jobs. Prior Oban versions used transactional
-advisory locks to prevent plugins from running concurrently, but there were some
-issues:
+Behind the scenes, the new testing modes rely on layers of validation within
+Oban's `Config` module. Now production configuration is validated automatically
+during test runs. Even though queues and plugins aren't _started_ in the test
+environment, their configuration is still validated.
 
-* Plugins don't know if they'll take the advisory lock, so they still need to
-  run a query periodically.
-
-* Nodes don't usually start simultaneously, and time drifts between machines.
-  There's no guarantee that the top of the minute for one node is the same as
-  another's—chances are, they don't match.
-
-Oban 2.11 introduces a table-based leadership mechanism that guarantees only one
-node in a cluster, where "cluster" means a bunch of nodes connected to the same
-Postgres database, will run plugins. Leadership is transparent and designed for
-resiliency with minimum chatter between nodes.
-
-See the [Upgrade Guide][upg] for instructions on how to create the peers table
-and get started with leadership. If you're curious about the implementation
-details or want to use leadership in your application, take a look at docs for
-`Oban.Peer`.
-
-## Alternative PG (Process Groups) Notifier
-
-Oban relies heavily on PubSub, and until now it only provided a Postgres
-adapter. Postres is amazing, and has a highly performant PubSub option, but it
-doesn't work in every environment (we're looking at you, PG Bouncer).
-
-Fortunately, many Elixir applications run in a cluster connected by distributed
-Erlang. That means Process Groups, aka PG, is available for many applications.
-
-So, we pulled Oban Pro's PG notifier into Oban to make it available for
-everyone! If your app runs in a proper cluster, you can switch over to the PG
-notifier:
+To switch, stop overriding `plugins` and `queues` and enable a testing mode
+in your `test.exs` config:
 
 ```elixir
-config :my_app, Oban,
-  notifier: Oban.Notifiers.PG,
-  ...
+config :my_app, Oban, testing: :manual
 ```
 
-Now there are two notifiers to choose from, each with their own strengths and
-weaknesses:
-
-* `Oban.Notifiers.Postgres` — Pros: Doesn't require distributed erlang,
-  publishes `insert` events to trigger queues; Cons: Doesn't work with PGBouncer
-  intransaction mode, Doesn't work in tests because of the sandbox.
-
-* `Oban.Notifiers.PG` — Pros: Works PG Bouncer in transaction mode, Works in
-  tests; Cons: Requires distributed Erlang, Doesn't publish `insert` events.
-
-## Basic Lifeline Plugin
-
-When a queue's producer crashes or a node shuts down before a job finishes
-executing, the job may be left in an `executing` state. The worst part is that
-these jobs—which we call "orphans"—are completely invisible until you go
-searching through the jobs table.
+Testing in `:manual` mode is identical to testing in older versions of Oban:
+jobs won't run automatically so you can use helpers like `assert_enqueued` and
+execute them manually with `Oban.drain_queue/2`.
 
-Oban Pro has awlays had a "Lifeline" plugin for just this ocassion—and now we've
-brought a basic `Lifeline` plugin to Oban.
-
-To automatically rescue orphaned jobs that are still `executing`, include the
-`Oban.Plugins.Lifeline` in your configuration:
+An alternate `:inline` allows Oban to bypass all database interaction and run
+jobs _immediately in the process that enqueued them_.
 
 ```elixir
-config :my_app, Oban,
-  plugins: [Oban.Plugins.Lifeline],
-  ...
+config :my_app, Oban, testing: :inline
 ```
 
-Now the plugin will search and rescue orphans after they've lingered for 60
-minutes.
+Finally, new [testing guides][tst] cover test setup, unit [testing
+workers][tsw], integration [testing queues][tsq], and testing [dynamic
+configuration][tsc].
 
-_🌟 Note: The `Lifeline` plugin may transition jobs that are genuinely
-`executing` and cause duplicate execution. For more accurate rescuing or to
-rescue jobs that have exhausted retry attempts see the `DynamicLifeline` plugin
-in [Oban Pro][pro]._
+[tst]: testing.html
+[tsw]: testing_workers.html
+[tsq]: testing_queues.html
+[tsc]: testing_config.html
 
-[pro]: https://getoban.pro
+## Global Peer Module
 
-## Reindexer Plugin
+Oban v2.11 introduced centralized leadership via Postgres tables. However,
+Postgres based leadership isn't always a good fit. For example, an ephemeral
+leadership mechanism is preferred for integration testing.
 
-Over time various Oban indexes (heck, any indexes) may grow without `VACUUM`
-cleaning them up properly. When this happens, rebuilding the indexes will
-release bloat and free up space in your Postgres instance.
-
-The new `Reindexer` plugin makes index maintenance painless and automatic by
-periodically rebuilding all of your Oban indexes concurrently, without any
-locks.
-
-By default, reindexing happens once a day at midnight UTC, but it's configurable
-with a standard cron expression (and timezone).
+In that case, you can make use of the new `:global` powered peer module for
+leadership:
 
 ```elixir
 config :my_app, Oban,
-  plugins: [Oban.Plugins.Reindexer],
+  peer: Oban.Peers.Global,
   ...
 ```
 
-See `Oban.Plugins.Reindexer` for complete options and implementation details.
-
-## Improved Telemetry and Logging
-
-The default telemetry backed logger includes more job fields and metadata about
-execution. Most notably, the execution state and formatted error reports when
-jobs fail.
-
-Here's an example of the default output for a successful job:
-
-```json
-{
-  "args":{"action":"OK","ref":1},
-  "attempt":1,
-  "duration":4327295,
-  "event":"job:stop",
-  "id":123,
-  "max_attempts":20,
-  "meta":{},
-  "queue":"alpha",
-  "queue_time":3127905,
-  "source":"oban",
-  "state":"success",
-  "tags":[],
-  "worker":"Oban.Integration.Worker"
-}
-```
-
-Now, here's an sample where the job has encountered an error:
-
-```json
-{
-  "attempt": 1,
-  "duration": 5432,
-  "error": "** (Oban.PerformError) Oban.Integration.Worker failed with {:error, \"ERROR\"}",
-  "event": "job:exception",
-  "state": "failure",
-  "worker": "Oban.Integration.Worker"
-}
-```
-
-## 2.11.2 — 2022-02-25
-
-### Bug Fixes
-
-- [Peer] Retain election schedule timing after a peer shuts down.
-
-  A bug in the Peer module's "down" handler overwrote the election scheduling
-  interval to `0`. As soon as the leader crashed all other peers in the cluster
-  would start trying to acquire leadership as fast as possible. That caused
-  excessive database load and churn.
-
-  In addition to the interval fix, this expands the scheduling interval to 30s,
-  and warns on any unknown messages to aid debugging in the future.
-
-- [Notifier.Postgres] Prevent crashing after reconnect.
-
-  The `handle_result` callback no longer sends an errorneous reply after a
-  disconnection.
-
-- [Job] Guard against typos and unknown options passed to `new/2`. Passing an
-  unrecognized option, such as `:scheduled_in` instead of `:schedule_in`, will
-  make a job invalid with a helpful base error.
-
-  Previously, passing an unknown option was silently ignored without any warning.
-
-## 2.11.1 — 2022-02-24
+## 2.12.0 — 2022-04-19
 
 ### Enhancements
 
-- [Oban] Validate the configured `repo` by checking for `config/0`, rather than
-  the more obscure `__adapter__/0` callback. This change improves integration
-  with Repo wrappers such as `fly_postgres`.
+- [Oban] Replace queue, plugin, and peer test configuration with a single
+  `:testing` option. Now configuring Oban for testing only requires one change,
+  setting the test mode to either `:inline` or `:manual`.
 
-- [Cron] Expose `parse/1` to facilitate testing that cron expressions are valid
-  and usable in a crontab.
+  - `:inline`—jobs execute immediately within the calling process and without
+    touching the database. This mode is simple and may not be suitable for apps
+    with complex jobs.
+  - `:manual`—jobs are inserted into the database where they can be verified and
+    executed when desired. This mode is more advanced and trades simplicity for
+    flexibility.
 
-### Bug Fixes
-
-- [Notifier.Postgres] Overwrite configured repo name when configuring the
-  long-lived Postgres connection.
-
-- [Lifeline] Fix rescuing when using a custom prefix. The previous
-  implementation assumed that there was an `oban_jobs_state` enum in the public
-  prefix.
+- [Testing] Add `with_testing_mode/2` to temporarily change testing modes
+  within the context of a function.
 
-- [Lifeline] Set `discarded_at` when discarding exhausted jobs.
-
-## 2.11.0 — 2022-02-13
-
-### Enhancements
+  Once the application starts in a particular testing mode it can't be changed.
+  That's inconvenient if you're running in `:inline` mode and don't want a
+  particular job to execute inline.
 
-- [Migration] Change the order of fields in the base index used for the primary
-  Oban queries.
+- [Config] Add `validate/1` to aid in testing dynamic Oban configuration.
 
-  The new order is much faster for frequent queries such as scheduled job
-  staging. Check the v2.11 upgrade guide for instructions on swapping the
-  index in existing applications.
+- [Config] Validate full plugin and queue options on init, without the need
+  to start plugins or queues.
 
-- [Worker] Avoid spawning a separate task for workers that use timeouts.
+- [Peers.Global] Add an alternate `:global` powered peer module.
 
-- [Engine] Add `insert_job`, `insert_all_jobs`, `retry_job`, and
-  `retry_all_jobs` as required callbacks for all engines.
+- [Plugin] A new `Oban.Plugin` behaviour formalizes starting and validating
+  plugins. The behaviour is implemented by all plugins, and is the foundation of
+  enhanced config validation.
 
-- [Oban] Raise more informative error messages for missing or malformed plugins.
-
-  Now missing plugins have a different error from invalid plugins or invalid
-  options.
-
-- [Telemetry] Normalize telemetry metadata for all engine operations:
-
-  - Include `changeset` for `insert`
-  - Include `changesets` for `insert_all`
-  - Include `job` for `complete_job`, `discard_job`, etc
-
-- [Repo] Include `[oban_conf: conf]` in `telemetry_options` for all Repo
-  operations.
-
-  With this change it's possible to differentiate between database calls made by
-  Oban versus the rest of your application.
+- [Plugin] Emit `[:oban, :plugin, :init]` event on init from every plugin.
 
 ### Bug Fixes
 
-- [Telemetry] Emit `discard` rather than `error` events when a job exhausts all retries.
-
-  Previously `discard_job` was only called for manual discards, i.e., when a job
-  returned `:discard` or `{:discard, reason}`. Discarding for exhausted attempts
-  was done within `error_job` in error cases.
-
-- [Cron] Respect the current timezone for `@reboot` jobs. Previously, `@reboot`
-  expressions were evaluated on boot without the timezone applied. In that case
-  the expression may not match the calculated time and jobs wouldn't trigger.
-
-- [Cron] Delay CRON evaluation until the next minute after initialization. Now
-  all cron scheduling ocurrs reliably at the top of the minute.
-
-- [Drainer] Introduce `discard` accumulator for draining results. Now exhausted
-  jobs along with manual discards count as a `discard` rather than a `failure`
-  or `success`.
-
-- [Oban] Expand changeset wrapper within multi function.
-
-  Previously, `Oban.insert_all` could handle a list of changesets, a wrapper map
-  with a `:changesets` key, or a function. However, the function had to return a
-  list of changesets rather than a changeset wrapper. This was unexpected and
-  made some multi's awkward.
-
-- [Testing] Preserve `attempted_at/scheduled_at` in `perform_job/3` rather than
-  overwriting them with the current time.
+- [Executor ] Skip timeout check with an unknown worker
 
-- [Oban] Include `false` as a viable `queue` or `plugin` option in typespecs
+  When the worker can't be resolved we don't need to check the timeout. Doing so
+  prevents returning a helpful "unknown worker" message, and instead causes a
+  function error for `nil.timeout/1`.
 
-### Deprecations
+- [Testing] Include `log` and `prefix` in generated conf for `perform_job`.
 
-- [Telemetry] Hard deprecate `Telemetry.span/3`, previously it was
-  soft-deprecated.
+  The opts, and subsequent conf, built for `perform_job` didn't include the
+  `prefix` or `log` options. That prevented functions that depend on a job's
+  `conf` within `perform/1` from running with the correct options.
 
-### Removals
+- [Drainer] Retain the currently configured engine while draining a queue.
 
-- [Telemetry] Remove circuit breaker event documentation because `:circuit`
-  events aren't emitted anymore.
+- [Watchman] Skip pausing queues when shutdown is immediate. This prevents
+  queue's from interacting with the database during short test runs.
 
-For changes prior to v2.11 see the [v2.10][prv] docs.
+For changes prior to v2.12 see the [v2.11][prv] docs.
 
 [opc]: https://getoban.pro/docs/pro/changelog.html
 [owc]: https://getoban.pro/docs/web/changelog.html
-[prv]: https://hexdocs.pm/oban/2.10.1/changelog.html
-[upg]: https://hexdocs.pm/oban/v2-11.html
+[prv]: https://hexdocs.pm/oban/2.11.2/changelog.html

README.md

@@ -154,7 +154,7 @@ dependencies in `mix.exs`:
 ```elixir
 def deps do
   [
-    {:oban, "~> 2.11"}
+    {:oban, "~> 2.12"}
   ]
 end
 ```

guides/installation.md

@@ -6,7 +6,7 @@ dependencies in `mix.exs`:
 ```elixir
 def deps do
   [
-    {:oban, "~> 2.11"}
+    {:oban, "~> 2.12"}
   ]
 end
 ```