Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add federated search #393

Merged
merged 5 commits into from
Mar 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions .rubocop_todo.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# This configuration was generated by
# `rubocop --auto-gen-config`
# on 2025-03-14 19:22:00 UTC using RuboCop version 1.27.0.
# on 2025-03-24 19:18:19 UTC using RuboCop version 1.27.0.
# The point is for the user to remove these configuration records
# one by one as the offenses are removed from the code base.
# Note that changes in the inspected code, or installation of new
Expand Down Expand Up @@ -66,7 +66,7 @@ Lint/UnusedMethodArgument:
Exclude:
- 'lib/meilisearch-rails.rb'

# Offense count: 13
# Offense count: 15
# Configuration parameters: IgnoredMethods, CountRepeatedAttributes.
Metrics/AbcSize:
Max: 105
Expand All @@ -82,12 +82,12 @@ Metrics/BlockLength:
Metrics/ClassLength:
Max: 171

# Offense count: 9
# Offense count: 11
# Configuration parameters: IgnoredMethods.
Metrics/CyclomaticComplexity:
Max: 28

# Offense count: 20
# Offense count: 23
# Configuration parameters: CountComments, CountAsOne, ExcludedMethods, IgnoredMethods.
Metrics/MethodLength:
Max: 102
Expand All @@ -97,7 +97,7 @@ Metrics/MethodLength:
Metrics/ModuleLength:
Max: 437

# Offense count: 8
# Offense count: 9
# Configuration parameters: IgnoredMethods.
Metrics/PerceivedComplexity:
Max: 35
Expand Down Expand Up @@ -139,7 +139,7 @@ RSpec/ContextWording:
- 'spec/options_spec.rb'
- 'spec/system/tech_shop_spec.rb'

# Offense count: 61
# Offense count: 72
# Configuration parameters: CountAsOne.
RSpec/ExampleLength:
Max: 16
Expand Down Expand Up @@ -177,11 +177,12 @@ RSpec/VerifiedDoubles:
Exclude:
- 'spec/configuration_spec.rb'

# Offense count: 3
# Offense count: 6
# Configuration parameters: ForbiddenMethods, AllowedMethods.
# ForbiddenMethods: decrement!, decrement_counter, increment!, increment_counter, insert, insert!, insert_all, insert_all!, toggle!, touch, touch_all, update_all, update_attribute, update_column, update_columns, update_counters, upsert, upsert_all
Rails/SkipsModelValidations:
Exclude:
- 'spec/federated_search_spec.rb'
- 'spec/multi_search_spec.rb'

# Offense count: 2
Expand Down Expand Up @@ -240,7 +241,7 @@ Style/StringLiterals:
Exclude:
- 'spec/ms_clean_up_job_spec.rb'

# Offense count: 16
# Offense count: 20
# This cop supports safe auto-correction (--auto-correct).
# Configuration parameters: AllowHeredoc, AllowURI, URISchemes, IgnoreCopDirectives, IgnoredPatterns.
# URISchemes: http, https
Expand Down
167 changes: 167 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
- [⚙️ Settings](#️-settings)
- [🔍 Custom search](#-custom-search)
- [🔍🔍 Multi search](#-multi-search)
- [🔍🔍 Federated search](#-federated-search)
- [🪛 Options](#-options)
- [Meilisearch configuration & environment](#meilisearch-configuration--environment)
- [Pagination with `kaminari` or `will_paginate`](#backend-pagination-with-kaminari-or-will_paginate-)
Expand Down Expand Up @@ -323,6 +324,172 @@ But this has been deprecated in favor of **federated search**.

See the [official multi search documentation](https://www.meilisearch.com/docs/reference/api/multi_search).

## 🔍🔍 Federated search

Federated search is similar to multi search, except that results are not grouped but sorted by ranking rules.

```ruby
results = Meilisearch::Rails.federated_search(
queries: [
{ q: 'Harry', scope: Book.all },
{ q: 'Attack on Titan', scope: Manga.all }
]
)
```

An enumerable `FederatedSearchResult` is returned, which can be iterated through with `#each`:

```erb
<ul>
<% results.each do |record| %>
<li><%= record.title %></li>
<% end %>
</ul>


<ul>
<!-- Attack on Titan appears first even though it was specified second,
it's ranked higher because it's a closer match -->
<li>Attack on Titan</li>
<li>Harry Potter and the Philosopher's Stone</li>
<li>Harry Potter and the Chamber of Secrets</li>
</ul>
```

The `queries` parameter may be a multi-search style hash with keys that are either classes, index names, or neither:

```ruby
results = Meilisearch::Rails.federated_search(
queries: {
Book => { q: 'Harry' },
Manga => { q: 'Attack on Titan' }
}
)
```

```ruby
results = Meilisearch::Rails.federated_search(
queries: {
'books_production' => { q: 'Harry', scope: Book.all },
'mangas_production' => { q: 'Attack on Titan', scope: Manga.all }
}
)
```

```ruby
results = Meilisearch::Rails.federated_search(
queries: {
'potter' => { q: 'Harry', scope: Book.all, index_uid: 'books_production' },
'titan' => { q: 'Attack on Titan', scope: Manga.all, index_uid: 'mangas_production' }
}
)
```

### Loading records <!-- omit in toc -->

Records are loaded when the `:scope` option is passed (may be a model or a relation),
or when a hash query is used with models as keys:

```ruby
results = Meilisearch::Rails.federated_search(
queries: [
{ q: 'Harry', scope: Book },
{ q: 'Attack on Titan', scope: Manga },
]
)
```

```ruby
results = Meilisearch::Rails.federated_search(
queries: {
Book => { q: 'Harry' },
Manga => { q: 'Attack on Titan' }
}
)
```

If the model is not provided, hashes are returned!

### Scoping records <!-- omit in toc -->

Any relation passed as `:scope` is used as the starting point when loading records:

```ruby
results = Meilisearch::Rails.federated_search(
queries: [
{ q: 'Harry', scope: Book.where('year <= 2006') },
{ q: 'Attack on Titan', scope: Manga.where(author: Author.find_by(name: 'Iseyama')) },
]
)
```

### Specifying the search index <!-- omit in toc -->

In order of precedence, to figure out which index to search, Meilisearch Rails will check:

1. `index_uid` options
```ruby
results = Meilisearch::Rails.federated_search(
queries: [
# Searching the 'fantasy_books' index
{ q: 'Harry', scope: Book, index_uid: 'fantasy_books' },
]
)
```
2. The index associated with the model
```ruby
results = Meilisearch::Rails.federated_search(
queries: [
# Searching the index associated with the Book model
# i. e. Book.index.uid
{ q: 'Harry', scope: Book },
]
)
```
3. The key when using hash queries
```ruby
results = Meilisearch::Rails.federated_search(
queries: {
# Searching index 'books_production'
books_production: { q: 'Harry', scope: Book },
}
)
```

### Pagination and other options <!-- omit in toc -->

In addition to queries, federated search also accepts `:federation` parameters which allow for finer control of the search:

```ruby
results = Meilisearch::Rails.federated_search(
queries: [
{ q: 'Harry', scope: Book },
{ q: 'Attack on Titan', scope: Manga },
],
federation: { offset: 10, limit: 5 }
)
```
See a full list of accepted options in [the meilisearch documentation](https://www.meilisearch.com/docs/reference/api/multi_search#federation).

#### Metadata <!-- omit in toc -->

The returned result from a federated search includes a `.metadata` attribute you can use to access everything other than the search hits:

```ruby
result.metadata
# {
# "processingTimeMs" => 0,
# "limit" => 20,
# "offset" => 0,
# "estimatedTotalHits" => 2,
# "semanticHitCount": 0
# }
```

The metadata contains facet stats and pagination stats, among others. See the full response in [the documentation](https://www.meilisearch.com/docs/reference/api/multi_search#federated-multi-search-requests).

More details on federated search (such as available `federation:` options) can be found on [the official multi search documentation](https://www.meilisearch.com/docs/reference/api/multi_search).

## 🪛 Options

### Meilisearch configuration & environment
Expand Down
62 changes: 57 additions & 5 deletions lib/meilisearch/rails/multi_search.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
require_relative 'multi_search/result'
require_relative 'multi_search/multi_search_result'
require_relative 'multi_search/federated_search_result'

module Meilisearch
module Rails
Expand All @@ -12,23 +13,60 @@ def multi_search(searches)
normalize(options, index_target)
end

MultiSearchResult.new(searches, client.multi_search(search_parameters))
MultiSearchResult.new(searches, client.multi_search(queries: search_parameters))
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is necessary due to this change in ms-ruby: meilisearch/meilisearch-ruby@055f7d0

end

def federated_search(queries:, federation: {})
if federation.nil?
Meilisearch::Rails.logger.warn(
'[meilisearch-rails] In federated_search, `nil` is an invalid `:federation` option. To explicitly use defaults, pass `{}`.'
)

federation = {}
end

queries.map! { |item| [nil, item] } if queries.is_a?(Array)

cleaned_queries = queries.filter_map do |(index_target, options)|
model_class = options[:scope].respond_to?(:model) ? options[:scope].model : options[:scope]
index_target = options.delete(:index_uid) || index_target || model_class

strip_pagination_options(options)
normalize(options, index_target)
end

raw_results = client.multi_search(queries: cleaned_queries, federation: federation)

FederatedSearchResult.new(queries, raw_results)
end

private

def normalize(options, index_target)
index_target = index_uid_from_target(index_target)

return nil if index_target.nil?

options
.except(:class_name, :scope)
.merge!(index_uid: index_uid_from_target(index_target))
.merge!(index_uid: index_target)
end

def index_uid_from_target(index_target)
case index_target
when String, Symbol
index_target
else
index_target.index.uid
when Class
if index_target.respond_to?(:index)
index_target.index.uid
else
Meilisearch::Rails.logger.warn <<~MODEL_NOT_INDEXED
[meilisearch-rails] This class was passed to a multi/federated search but it does not have an #index: #{index_target}
[meilisearch-rails] Are you sure it has a `meilisearch` block?
MODEL_NOT_INDEXED

nil
end
end
end

Expand All @@ -44,6 +82,20 @@ def paginate(options)
options[:page] ||= 1
end

def strip_pagination_options(options)
pagination_options = %w[page hitsPerPage hits_per_page limit offset].select do |key|
options.delete(key) || options.delete(key.to_sym)
end

return if pagination_options.empty?

Meilisearch::Rails.logger.warn <<~WRONG_PAGINATION
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These warnings will be very helpful!

[meilisearch-rails] Pagination options in federated search must apply to whole federation.
[meilisearch-rails] These options have been removed: #{pagination_options.join(', ')}.
[meilisearch-rails] Please pass them after queries, in the `federation:` option.
WRONG_PAGINATION
end

def pagination_enabled?
Meilisearch::Rails.configuration[:pagination_backend]
end
Expand Down
Loading