Skip to content

feat(elasticsearch-plugin): index and search per-currency variant prices#30

Open
tbouliere-datasolution wants to merge 9 commits into
vendurehq:mainfrom
tbouliere-datasolution:main
Open

feat(elasticsearch-plugin): index and search per-currency variant prices#30
tbouliere-datasolution wants to merge 9 commits into
vendurehq:mainfrom
tbouliere-datasolution:main

Conversation

@tbouliere-datasolution
Copy link
Copy Markdown

Context

Until now the Elasticsearch index used ${channelId}_${entityId}_${languageCode} as document ID and only stored the price for the channel's default currency. As a consequence, a channel with availableCurrencyCodes: [GBP, EUR] would expose
GBP-priced variants only — querying it with currencyCode = EUR returned the GBP price.

Changes

Indexing

  • New CurrencyAwareMutableRequestContext (extends MutableRequestContext) that exposes setCurrencyCode() and overrides the currencyCode getter so the indexing context can switch currency without mutating the active Channel.
  • ElasticsearchIndexerController.getId(...) now takes a currencyCode and produces ${channelId}_${entityId}_${languageCode}_${currencyCode}. One document per (channel × language × currency) instead of one per (channel × language).
  • The main indexing loop in updateProductsInternal iterates over channel.availableCurrencyCodes (fallback to [channel.defaultCurrencyCode]). For each currency it calls ctx.setCurrencyCode(currencyCode) then applyChannelPriceAndTax(variant, ctx). The DefaultProductVariantPriceSelectionStrategy already filters by ctx.currencyCode, so the matching ProductVariantPrice is selected automatically — no patch of the price applicator was needed.
  • deleteProductOperations and deleteVariantsInternalOperations now iterate over each channel's currencies as well, to keep deletions symmetrical with indexing. The channels query was extended to select defaultCurrencyCode and
    availableCurrencyCodes. The internal helper signature changed from channelIds: ID[] to channels: Channel[] (single internal caller updated).

Reading

  • build-elastic-body.ts adds a term: { currencyCode: ctx.currencyCode } filter when ctx.currencyCode is defined. This mirrors the read-side behavior of DefaultProductVariantPriceSelectionStrategy and avoids returning N duplicates per
    variant when several currencies are indexed. In production ctx.currencyCode is always populated (RequestContext falls back to channel.defaultCurrencyCode at construction), so the filter is always applied; the if guard exists only for tests
    that build a context without a channel default.

Out of scope / known caveats

  • Custom productVariantPriceSelectionStrategy implementations that ignore ctx.currencyCode will still produce identical prices across currencies during indexing. Users who override that strategy must keep ctx.currencyCode as a discriminator
    if they want this feature to work.
  • The ES currencyCode field was already mapped as keyword in indexing-utils.ts, so no mapping change is required. A full reindex is required after merge because document IDs change shape.

Tests

  • e2e/elasticsearch-plugin.e2e-spec.tsmultiple currency handling describe:
  • beforeAll fetches variants of T_3 and T_4, switches to the multi-currency channel and sets EUR prices via updateProductVariants (prices: [{ currencyCode: EUR, price: ... }]).
  • New test return EUR when requested queries the shop API with the currencyCode: EUR header and asserts the returned price.min/max matches the EUR amounts that were set.
  • Existing return GBP by default test still passes — the channel default currency is GBP.
  • build-elastic-body.spec.ts keeps passing as-is: the test RequestContext is built without a currency, so ctx.currencyCode is undefined and the new filter is not applied.

Migration

After deploy, trigger a full reindex mutation on each channel. Old documents (without currency in their _id) will be left dangling until they are explicitly cleaned up — alternatively, drop and recreate the index.

@michaelbromley
Copy link
Copy Markdown
Member

Hi! I just merged another PR that causes conflicts with this one - can you take a look at the merge conflicts?

Copy link
Copy Markdown

@grolmus grolmus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together — the diagnosis is right (single-currency-per-doc is broken for multi-currency channels) and the indexing/reading symmetry is exactly the shape of the fix. I pulled the branch, ran unit + e2e (all 99 green) and then probed a few scenarios the new tests don't cover. Found a handful of blockers I'd like to discuss before this lands.

I'm happy to take some/all of these on rather than ping-pong; let me know which you'd prefer.

Blockers

1. Zero-priced phantom documents

When a variant is assigned to a channel with availableCurrencyCodes = [GBP, EUR] but only has a ProductVariantPrice row in GBP, the EUR document is indexed with price = 0, priceWithTax = 0.

Reproduced locally with a probe e2e: T_1 assigned to a [GBP, EUR] channel, no EUR price added, reindex. Direct ES lookup of _doc/2_1_en_EUR returns price: 0, priceWithTax: 0, currencyCode: \"EUR\". The shop API then surfaces this in searchProductsShopDocument with currencyCode: EUR as a normal hit.

Root cause is in core, not this PR — but this PR exposes it:

  • DefaultProductVariantPriceSelectionStrategy.selectPrice (core: packages/core/src/config/catalog/default-product-variant-price-selection-strategy.ts:18-22) returns undefined when no matching currency exists.
  • ProductPriceApplicator.applyChannelPriceAndTax (core: packages/core/src/service/helpers/product-price-applicator/product-price-applicator.ts:66-106) defaults throwIfNoPriceFound = false, falls through with inputPrice: channelPrice?.price ?? 0, and writes variant.listPrice = 0, variant.currencyCode = ctx.currencyCode.
  • The indexer then unconditionally builds and ships a doc.

Sort impact: sort: { price: ASC } floats every phantom doc to the top of the result page. priceRange: { min: 1 } works around it but consumers shouldn't need to know that.

Two options, either acceptable:

  • Skip: after applyChannelPriceAndTax, if variant.listPrice === 0 and no ProductVariantPrice exists in variant.productVariantPrices for ctx.currencyCode + ctx.channelId, do not push operations for that (variant, currency) tuple. Log a debug message.
  • Gate: make per-currency indexing opt-in via a new option (see #2). When the option is off, behaviour is unchanged from main. When on, document that variants must have explicit prices in every available currency or rely on a custom ProductVariantPriceSelectionStrategy fallback.

2. No opt-in option (parity with core's DefaultSearchPlugin)

Vendure core's DefaultSearchPlugin indexer gates per-currency indexing behind this.options.indexCurrencyCode (packages/core/src/plugin/default-search-plugin/indexer/indexer.controller.ts:438-442). This PR has no equivalent, so:

  • Every existing deployment — including single-currency channels — gets a forced behaviour change and a forced reindex on upgrade (the doc _id schema changes).
  • Single-currency users gain nothing from the change.

Suggest mirroring core: add indexCurrencyCode?: boolean to ElasticsearchPluginOptions (default false). When false, iterate [channel.defaultCurrencyCode] and keep the _id shape backward-compatible (or write to both schemas for a migration window).

3. Deletion path leaves orphaned per-currency docs

The delete operations in deleteVariantsInternalOperations (packages/elasticsearch-plugin/src/indexing/indexer.controller.ts:804-852) and deleteProductOperations (:715-802) iterate the current channel.availableCurrencyCodes. If the channel's available currencies change between indexing and deletion, docs for the removed currencies are never deleted.

Reproduced locally:

  1. Channel [GBP, EUR], variant T_1 indexed → ES contains _ids: ['2_1_en_GBP', '2_1_en_EUR'].
  2. Update channel to availableCurrencyCodes: [GBP].
  3. Delete variant T_1 → ES now contains _ids: ['2_1_en_EUR'] — the EUR doc is orphaned.

Suggest replacing per-currency delete ops with a single delete_by_query keyed on { channelId, productVariantId } (and for synthetic products { channelId, productId, productVariantId: -product.id }). Same network shape, idempotent, immune to channel-config drift.

4. createVariantIndexItem cross-currency contamination is one refactor away

applyChannelPriceAndTax mutates variant.listPrice, variant.listPriceIncludesTax, variant.currencyCode, variant.taxRateApplied in place on the same object reference across currency iterations (indexer.controller.ts:566-612). Today it works because the doc build is awaited inline and captures eagerly. Any future refactor that batches/defers createVariantIndexItem calls (e.g. parallel Promise.all across variants, deferred bulk-op composition) will silently produce cross-currency contamination — variant A's GBP doc shipped with variant B's EUR price.

Minimum: a comment near :569-570 pinning the invariant ("do not move this await outside the inner loop; the variant entity is mutated in place"). Better: snapshot the relevant price fields into local variables before pushing the bulk op, so the captured doc is immune to later mutation.

5. Mutated ctx state needs try/finally

setCurrencyCode and setChannel mutate ctx, and the same ctx is reused across products in updateProductsOperations (:666-672). An exception mid-channel-loop leaves mutatedCurrencyCode and ctx.channel in a wrong state for the next product. Wrap the channel loop body in try/finally and reset in finally. Pair the setCurrencyCode(undefined) (:652) and setChannel(originalChannel) (:654) into the same cleanup.

6. CurrencyAwareMutableRequestContext.deserialize via Object.setPrototypeOf

currency-aware-request-context.ts:22-25 retrofits the prototype after the parent factory returns. Works today, but:

  • If core ever migrates MutableRequestContext to private # fields, the swap can't reach them.
  • V8 deopts on prototype mutation of existing objects (not catastrophic, but worth avoiding in hot indexing paths).

The lighter currency-only swap is otherwise a nice improvement over core's new Channel({ ...channel, defaultCurrencyCode }) pattern — it sidesteps the side effects of mutating the channel reference (cache keys, tax-zone resolution memoised by channelId, etc.). Suggest keeping the approach but rewriting deserialize to construct an instance directly from ctxObject without prototype manipulation. Failing that, a comment noting the upstream-coupling risk.

7. CHANGELOG and README

The PR introduces:

  • A forced full reindex on upgrade (doc _id schema change).
  • A new requirement that variants have explicit ProductVariantPrice rows in every channel-available currency (or a custom ProductVariantPriceSelectionStrategy with fallback semantics).

Neither is documented yet. README's plugin section + a top-level MIGRATION.md note would catch most users; CHANGELOG will be generated by lerna at release time.

Non-blocking

  • The channel.availableCurrencyCodes?.length ? channel.availableCurrencyCodes : [channel.defaultCurrencyCode] fallback is duplicated 3x (:562, :754, :823). Worth extracting to a getChannelIndexCurrencies(channel) helper.
  • build-elastic-body.ts:42's if (ctx.currencyCode) guard exists only to keep build-elastic-body.spec.ts's test contexts (which build a RequestContext from new Channel({ id }) with no defaultCurrencyCode) passing. The conditional reads like an optional contract; fixing the tests to construct a proper ctx and dropping the guard would be cleaner.
  • The PR description acknowledges that custom ProductVariantPriceSelectionStrategy implementations that ignore ctx.currencyCode will produce duplicate prices. A startup-time warning log when configService.catalogOptions.productVariantPriceSelectionStrategy is not the default and multi-currency indexing is on would catch misconfigurations early.
  • Synthetic product docs (createSyntheticProductIndexItem) are now channels × languages × currencies per variantless product. Fine functionally, slight index bloat for multi-currency stores with many empty products.
  • A regression e2e covering the missing-currency-price case (the scenario above) would prevent reintroduction.

Rolling-deploy note

UpdateProductMessageData / VariantChannelMessageData shapes are unchanged, so old workers won't crash on new messages. But during a rolling deploy, old workers will write docs with the 3-part _id schema while new workers write the 4-part schema, producing index inconsistency until everyone catches up + a reindex runs. Worth a one-line note in the migration doc.


Solid groundwork — the indexing/reading symmetry and the CurrencyAwareMutableRequestContext design are both better than core's pattern. Mainly the missing-price handling, the opt-in, and the deletion path need to be addressed before this is mergeable. Happy to pick up any subset of these on a follow-up PR if that's easier — let me know.

@tbouliere-datasolution
Copy link
Copy Markdown
Author

Hello @grolmus.
I'm going to do a first pass. I'll let you know what I might not include.

@tbouliere-datasolution
Copy link
Copy Markdown
Author

«build-elastic-body.ts:42's if (ctx.currencyCode) guard exists only to keep build-elastic-body.spec.ts's test contexts (which build a RequestContext from new Channel({ id }) with no defaultCurrencyCode) passing. The conditional reads like an optional contract; fixing the tests to construct a proper ctx and dropping the guard would be cleaner.»

This guard was implemented to avoid having to refactor the existing end-to-end test that simulates the request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants