Skip to content

_search drops results from successful backends when another backend raises first #427

Description

@ddett

Before you open an issue:

  • I have the latest version. (Check: ddgs version. Update: pip install -U ddgs)
  • I tried reinstalling the library. (pip install -I ddgs)
  • I tried using a proxy

Describe the bug

When text() is called with multiple backends (e.g. backend="brave,google,duckduckgo"), a single failing backend can cause all results — including those from successful backends — to be silently dropped, raising DDGSException: No results found.

The issue is in _search() (ddgs.py L191–216):

done, not_done = wait(futures.values(), return_when=FIRST_EXCEPTION)

If a backend raises before the others complete, the successful futures land in not_done and are never collected. Only done is iterated — which contains the failed future — so err is set, and the method raises instead of returning the results that other backends would have produced.

Steps to reproduce the problem:

from ddgs import DDGS

# google backend reliably fails (see #412); if it raises before
# brave/duckduckgo finish, their results are silently discarded
results = DDGS().text("python programming", backend="brave,google,duckduckgo", max_results=10)
# -> DDGSException: No results found.

# Same query without the failing backend works fine:
results = DDGS().text("python programming", backend="brave,duckduckgo", max_results=10)
# -> returns 10 results

The bug is non-deterministic: because backends race, sometimes a healthy backend finishes before the failing one raises, and results appear. Repeat the first snippet a few times and it will fail intermittently.

Expected behavior: If any backend returns results, those should be included. Only raise if every backend fails.

Suggested fix: Replace FIRST_EXCEPTION with ALL_COMPLETED (or as_completed) and collect results from all futures that succeeded:

done, _ = wait(futures.values(), return_when=ALL_COMPLETED)
results = []
for future in done:
    try:
        results.extend(future.result())
    except Exception:
        pass  # skip failed backends
if not results:
    raise DDGSException("No results found.")

Specify this information

  • OS: macOS 15 / Ubuntu 22.04
  • Environment: Python 3.12, pip venv
  • ddgs version: 9.11.3

Related issues:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions