Feature/literature mcp

[SoberPizza]

improve boolean parsing and logging in literature search functions

Summary by Sourcery

添加一个多源文献检索 MCP 服务器以及配套的 OpenAlex 客户端和工具。

新功能：

• 引入用于文献检索的 OpenAlex API 客户端，支持限流、重试，并将结果转换为统一的数据模型。
• 添加一个 WorkDistributor，用于协调多个文献数据源的搜索，并对结果进行聚合、去重和排序。
• 暴露一个 search_literature MCP 工具，对分发器进行封装，并返回带有 JSON 结果的结构化 Markdown 报告。
• 提供共享的文献数据模型以及 DOI 规范化/去重工具，以在不同数据源之间实现一致的数据处理。

测试：

• 为 OpenAlex 客户端、工作分发器（WorkDistributor）、DOI 工具以及共享文献模型添加全面的单元测试套件，以验证行为和边界情况。

Original summary in English

Summary by Sourcery

Add a multi-source literature search MCP server and supporting OpenAlex client and utilities.

New Features:

• Introduce an OpenAlex API client for literature search with rate limiting, retries, and result transformation into a common model.
• Add a WorkDistributor to orchestrate searches across multiple literature data sources and aggregate, deduplicate, and sort results.
• Expose a search_literature MCP tool that wraps the distributor and returns a structured markdown report with JSON results.
• Provide shared literature data models and DOI normalization/deduplication utilities for consistent handling across sources.

Tests:

• Add comprehensive unit test suites for the OpenAlex client, work distributor, DOI utilities, and shared literature models to validate behavior and edge cases.

[sourcery-ai]

Reviewer's Guide

引入一个多数据源的文献检索子系统（OpenAlex 客户端、DOI 工具、工作分发器、MCP 服务器），提供健壮的布尔值/年份解析、日志记录和完整测试，用于支持统一的规范化模型、去重逻辑，并在不同工具之间实现一致的检索行为。

新文献检索 MCP 流程的时序图

sequenceDiagram
    actor User
    participant MCP as MCP_literature_server
    participant Tool as search_literature_tool
    participant Distributor as WorkDistributor
    participant OAClient as OpenAlexClient
    participant OpenAlexAPI as OpenAlex_API

    User->>MCP: request search_literature(query, filters)
    MCP->>Tool: call search_literature(...)

    Tool->>Tool: parse years, booleans, max_results
    Tool->>Tool: create SearchRequest
    Tool->>Distributor: new WorkDistributor(openalex_email)

    Tool->>Distributor: search(request)
    activate Distributor

    Distributor->>Distributor: clamp request.max_results
    Distributor->>Distributor: determine data_sources

    loop for each source
        Distributor->>OAClient: search(request)
        activate OAClient
        OAClient->>OAClient: resolve author/institution/source ids
        OAClient->>OpenAlexAPI: GET /authors, /institutions, /sources
        OpenAlexAPI-->>OAClient: JSON ids

        OAClient->>OpenAlexAPI: GET /works?page=1...N with filters
        OpenAlexAPI-->>OAClient: paginated results
        OAClient->>OAClient: transform to LiteratureWork list
        OAClient-->>Distributor: works, warnings
        deactivate OAClient
    end

    Distributor->>Distributor: deduplicate_by_doi(all_works)
    Distributor->>Distributor: sort and slice by max_results
    Distributor-->>Tool: aggregated_result
    deactivate Distributor

    Tool->>Tool: merge year and boolean warnings
    Tool->>Tool: format markdown + JSON via _format_search_result
    Tool-->>MCP: markdown_report
    MCP-->>User: markdown_report

Loading

新文献检索子系统的类图

classDiagram
    direction LR

    class SearchRequest {
      +str query
      +str author
      +str institution
      +str source
      +int year_from
      +int year_to
      +bool is_oa
      +str work_type
      +str language
      +bool is_retracted
      +bool has_abstract
      +bool has_fulltext
      +str sort_by
      +int max_results
      +list~str~ data_sources
    }

    class LiteratureWork {
      +str id
      +str doi
      +list~dict~ authors
      +int publication_year
      +int cited_by_count
      +str abstract
      +str journal
      +bool is_oa
      +str oa_url
      +str source
      +dict raw_data
    }

    class BaseLiteratureClient {
      <<abstract>>
      +search(request SearchRequest) async tuple~list~LiteratureWork~~ list~str~~
    }

    class OpenAlexClient {
      +str email
      +int rate_limit
      +_RateLimiter rate_limiter
      +httpx_AsyncClient client
      +search(request SearchRequest) async tuple~list~LiteratureWork~~ list~str~~
      +close() async None
      +pool_type str
      -_build_query_params(request SearchRequest, author_id str, institution_id str, source_id str) dict~str,str~
      -_resolve_author_id(author_name str) async tuple~str,bool,str~
      -_resolve_institution_id(institution_name str) async tuple~str,bool,str~
      -_resolve_source_id(source_name str) async tuple~str,bool,str~
      -_fetch_all_pages(params dict~str,str~, max_results int) async list~dict~
      -_request_with_retry(url str, params dict~str,str~) async dict~str,Any~
      -_transform_work(work dict~str,Any~) LiteratureWork
      -_reconstruct_abstract(inverted_index dict~str,list~int~~) str
    }

    class _RateLimiter {
      -float _min_interval
      -asyncio_Lock _lock
      -float _last_request
      -asyncio_Semaphore _semaphore
      +__aenter__() async None
      +__aexit__(exc_type type, exc BaseException, tb Any) async None
      -_throttle() async None
    }

    class WorkDistributor {
      +dict~str,Any~ clients
      +str openalex_email
      +search(request SearchRequest) async dict~str,Any~
      +close() async None
      -_register_clients() None
      -_sort_works(works list~LiteratureWork~, sort_by str) list~LiteratureWork~
    }

    class WorkWithDOI {
      <<protocol>>
      +str doi
      +int cited_by_count
      +int publication_year
    }

    class doi_cleaner_module {
      +normalize_doi(doi str) str
      +deduplicate_by_doi(works list~WorkWithDOI~) list~WorkWithDOI~
    }

    class mcp_literature_module {
      +search_literature(query str, mailto str, author str, institution str, source str, year_from str, year_to str, is_oa str, work_type str, language str, is_retracted str, has_abstract str, has_fulltext str, sort_by str, max_results int, data_sources list~str~, include_abstract bool) async str
      -_format_search_result(request SearchRequest, result dict~str,Any~, include_abstract bool) str
    }

    BaseLiteratureClient <|-- OpenAlexClient
    OpenAlexClient o-- _RateLimiter
    WorkDistributor o-- BaseLiteratureClient
    WorkDistributor --> SearchRequest
    WorkDistributor --> LiteratureWork
    doi_cleaner_module ..> WorkWithDOI
    WorkDistributor ..> doi_cleaner_module
    mcp_literature_module ..> WorkDistributor
    mcp_literature_module ..> SearchRequest
    OpenAlexClient ..> SearchRequest
    OpenAlexClient ..> LiteratureWork

Loading

文件级变更

Change	Details	Files
添加一个带有速率限制、ID 解析和重试逻辑的 OpenAlex 文献客户端，将 OpenAlex 的 works 映射到通用的 LiteratureWork 模型。	实现 OpenAlexClient，支持礼貌/默认的池选择、全局速率限制器以及异步 httpx 客户端初始化。 提供 search() 方法，用于将作者/机构/来源名称解析为 ID，构建 OpenAlex 查询参数，遍历 /works 分页结果，并将结果转换为 LiteratureWork 实例。 实现辅助方法，用于解析实体 ID、从倒排索引重建摘要、转换原始 OpenAlex work 负载，以及在超时、403 和 5xx 响应时执行带指数回退的 HTTP 请求重试。	service/app/utils/literature/openalex_client.py service/tests/unit/test_literature/test_openalex_client.py
引入共享的文献模型、基础客户端抽象、DOI 规范化/去重工具以及用于跨数据源聚合结果的工作分发器。	定义 SearchRequest 和 LiteratureWork 数据类，作为所有文献客户端的统一规范化接口。 创建 BaseLiteratureClient 抽象类，要求实现一个异步 search 方法，返回 works 和 warnings。 添加 normalize_doi 和 deduplicate_by_doi 辅助方法，用于标准化 DOI，并基于引用次数和年份为每个 DOI 保留最佳记录。 实现 WorkDistributor，用于注册客户端（当前为 OpenAlex）、并行分发检索请求、聚合和去重 works、限制 max_results、按相关性/引用数/日期排序，并通过异步上下文管理器管理客户端生命周期。	service/app/utils/literature/models.py service/app/utils/literature/base_client.py service/app/utils/literature/doi_cleaner.py service/app/utils/literature/work_distributor.py service/app/utils/literature/__init__.py service/tests/unit/test_literature/test_base_client.py service/tests/unit/test_literature/test_doi_cleaner.py service/tests/unit/test_literature/test_work_distributor.py
通过 MCP 工具对外暴露文献检索能力，并对布尔和数值输入进行谨慎解析，同时生成信息丰富的 Markdown 报告。	添加基于 FastMCP 的文献 MCP 服务器，提供 search_literature 工具，用于构建 SearchRequest 并交由 WorkDistributor 处理。 解析 is_oa、is_retracted、has_abstract、has_fulltext 和 include_abstract 等过滤项的字符串/布尔输入，当值无法识别时输出警告而不是直接失败。 对年份和 max_results 进行限制和校验，在起止年份颠倒时交换范围，并添加面向用户的警告说明所做的调整。 将检索结果格式化为 Markdown 报告，其中包括检索条件、统计信息、警告、JSON 格式的 work 列表（可选包含摘要），以及用于帮助用户优化后续检索的指导信息。	service/app/mcp/literature.py
为新的文献组件和 MCP 行为添加全面的单元测试覆盖，同时保留了一些需要解决的测试文件合并冲突标记。	覆盖 OpenAlexClient 初始化、带多种过滤条件的检索流程、ID 解析警告、分页参数、摘要重建、work 转换、重试逻辑以及上下文管理器行为。 测试 BaseLiteratureClient 协议行为、SearchRequest 和 LiteratureWork 数据类、DOI 规范化边界情况和去重规则，以及 WorkDistributor 的聚合/去重/排序和错误处理。 验证 WorkDistributor 上下文管理器能够安全地关闭异步和同步客户端。 注意：若干新的测试文件中仍包含未解决的 git 合并冲突标记（<<<<<<<, =======, >>>>>>>），在合并前必须清理。	service/tests/unit/test_literature/test_openalex_client.py service/tests/unit/test_literature/test_base_client.py service/tests/unit/test_literature/test_doi_cleaner.py service/tests/unit/test_literature/test_work_distributor.py

---

Tips and commands

Interacting with Sourcery

• 触发新的代码审查： 在 Pull Request 中评论 @sourcery-ai review。
• 继续讨论： 直接回复 Sourcery 的审查评论即可继续对话。
• 从审查评论生成 GitHub issue： 在审查评论下回复，请求 Sourcery 从该评论创建一个 issue。你也可以直接回复 @sourcery-ai issue 来从该评论生成 issue。
• 生成 Pull Request 标题： 在 Pull Request 标题任意位置写上 @sourcery-ai 即可随时生成标题。你也可以在 Pull Request 中评论 @sourcery-ai title 来（重新）生成标题。
• 生成 Pull Request 摘要： 在 Pull Request 描述正文的任意位置写上 @sourcery-ai summary，即可在对应位置生成 PR 摘要。也可以在 Pull Request 中评论 @sourcery-ai summary 来随时（重新）生成摘要。
• 生成 Reviewer's Guide： 在 Pull Request 中评论 @sourcery-ai guide，即可随时（重新）生成 Reviewer's Guide。
• 一次性解决所有 Sourcery 评论： 在 Pull Request 中评论 @sourcery-ai resolve，可将所有 Sourcery 评论标记为已解决。如果你已经处理完所有评论且不想再看到它们，这会很有用。
• 一次性关闭所有 Sourcery 审查： 在 Pull Request 中评论 @sourcery-ai dismiss，以关闭所有现有的 Sourcery 审查。若你希望重新开始一次全新的审查，特别有用——别忘了再评论 @sourcery-ai review 来触发新审查！

Customizing Your Experience

访问你的 dashboard 以：

• 启用或停用诸如 Sourcery 自动生成 PR 摘要、Reviewer's Guide 等审查功能。
• 更改审查语言。
• 添加、移除或编辑自定义审查指令。
• 调整其他审查设置。

Getting Help

• 如有问题或反馈，请联系支持团队。
• 访问我们的文档以获取详细指南和信息。
• 通过 X/Twitter、LinkedIn 或 GitHub 关注 Sourcery 团队，获取最新动态。

Original review guide in English

Reviewer's Guide

Introduce a multi-source literature search subsystem (OpenAlex client, DOI utilities, work distributor, MCP server) with robust boolean/year parsing, logging, and comprehensive tests, enabling normalized models, deduplication, and unified search behavior across tools.

Sequence diagram for the new literature search MCP flow

sequenceDiagram
    actor User
    participant MCP as MCP_literature_server
    participant Tool as search_literature_tool
    participant Distributor as WorkDistributor
    participant OAClient as OpenAlexClient
    participant OpenAlexAPI as OpenAlex_API

    User->>MCP: request search_literature(query, filters)
    MCP->>Tool: call search_literature(...)

    Tool->>Tool: parse years, booleans, max_results
    Tool->>Tool: create SearchRequest
    Tool->>Distributor: new WorkDistributor(openalex_email)

    Tool->>Distributor: search(request)
    activate Distributor

    Distributor->>Distributor: clamp request.max_results
    Distributor->>Distributor: determine data_sources

    loop for each source
        Distributor->>OAClient: search(request)
        activate OAClient
        OAClient->>OAClient: resolve author/institution/source ids
        OAClient->>OpenAlexAPI: GET /authors, /institutions, /sources
        OpenAlexAPI-->>OAClient: JSON ids

        OAClient->>OpenAlexAPI: GET /works?page=1...N with filters
        OpenAlexAPI-->>OAClient: paginated results
        OAClient->>OAClient: transform to LiteratureWork list
        OAClient-->>Distributor: works, warnings
        deactivate OAClient
    end

    Distributor->>Distributor: deduplicate_by_doi(all_works)
    Distributor->>Distributor: sort and slice by max_results
    Distributor-->>Tool: aggregated_result
    deactivate Distributor

    Tool->>Tool: merge year and boolean warnings
    Tool->>Tool: format markdown + JSON via _format_search_result
    Tool-->>MCP: markdown_report
    MCP-->>User: markdown_report

Loading

Class diagram for the new literature search subsystem

classDiagram
    direction LR

    class SearchRequest {
      +str query
      +str author
      +str institution
      +str source
      +int year_from
      +int year_to
      +bool is_oa
      +str work_type
      +str language
      +bool is_retracted
      +bool has_abstract
      +bool has_fulltext
      +str sort_by
      +int max_results
      +list~str~ data_sources
    }

    class LiteratureWork {
      +str id
      +str doi
      +list~dict~ authors
      +int publication_year
      +int cited_by_count
      +str abstract
      +str journal
      +bool is_oa
      +str oa_url
      +str source
      +dict raw_data
    }

    class BaseLiteratureClient {
      <<abstract>>
      +search(request SearchRequest) async tuple~list~LiteratureWork~~ list~str~~
    }

    class OpenAlexClient {
      +str email
      +int rate_limit
      +_RateLimiter rate_limiter
      +httpx_AsyncClient client
      +search(request SearchRequest) async tuple~list~LiteratureWork~~ list~str~~
      +close() async None
      +pool_type str
      -_build_query_params(request SearchRequest, author_id str, institution_id str, source_id str) dict~str,str~
      -_resolve_author_id(author_name str) async tuple~str,bool,str~
      -_resolve_institution_id(institution_name str) async tuple~str,bool,str~
      -_resolve_source_id(source_name str) async tuple~str,bool,str~
      -_fetch_all_pages(params dict~str,str~, max_results int) async list~dict~
      -_request_with_retry(url str, params dict~str,str~) async dict~str,Any~
      -_transform_work(work dict~str,Any~) LiteratureWork
      -_reconstruct_abstract(inverted_index dict~str,list~int~~) str
    }

    class _RateLimiter {
      -float _min_interval
      -asyncio_Lock _lock
      -float _last_request
      -asyncio_Semaphore _semaphore
      +__aenter__() async None
      +__aexit__(exc_type type, exc BaseException, tb Any) async None
      -_throttle() async None
    }

    class WorkDistributor {
      +dict~str,Any~ clients
      +str openalex_email
      +search(request SearchRequest) async dict~str,Any~
      +close() async None
      -_register_clients() None
      -_sort_works(works list~LiteratureWork~, sort_by str) list~LiteratureWork~
    }

    class WorkWithDOI {
      <<protocol>>
      +str doi
      +int cited_by_count
      +int publication_year
    }

    class doi_cleaner_module {
      +normalize_doi(doi str) str
      +deduplicate_by_doi(works list~WorkWithDOI~) list~WorkWithDOI~
    }

    class mcp_literature_module {
      +search_literature(query str, mailto str, author str, institution str, source str, year_from str, year_to str, is_oa str, work_type str, language str, is_retracted str, has_abstract str, has_fulltext str, sort_by str, max_results int, data_sources list~str~, include_abstract bool) async str
      -_format_search_result(request SearchRequest, result dict~str,Any~, include_abstract bool) str
    }

    BaseLiteratureClient <|-- OpenAlexClient
    OpenAlexClient o-- _RateLimiter
    WorkDistributor o-- BaseLiteratureClient
    WorkDistributor --> SearchRequest
    WorkDistributor --> LiteratureWork
    doi_cleaner_module ..> WorkWithDOI
    WorkDistributor ..> doi_cleaner_module
    mcp_literature_module ..> WorkDistributor
    mcp_literature_module ..> SearchRequest
    OpenAlexClient ..> SearchRequest
    OpenAlexClient ..> LiteratureWork

Loading

File-Level Changes

Change	Details	Files
Add an OpenAlex literature client with rate limiting, ID resolution and retry logic that maps OpenAlex works into a common LiteratureWork model.	Implement OpenAlexClient with polite/default pool selection, global rate limiter and async httpx client initialization. Provide search() that resolves author/institution/source names to IDs, builds OpenAlex query params, paginates through /works, and transforms results to LiteratureWork instances. Implement helper methods for resolving entity IDs, reconstructing abstracts from inverted index, transforming raw OpenAlex work payloads, and performing HTTP requests with exponential backoff on timeouts, 403, and 5xx responses.	service/app/utils/literature/openalex_client.py service/tests/unit/test_literature/test_openalex_client.py
Introduce shared literature models, base client abstraction, DOI normalization/deduplication utilities, and a work distributor that aggregates results across data sources.	Define SearchRequest and LiteratureWork dataclasses as the normalized interface for all literature clients. Create BaseLiteratureClient abstract class that requires an async search method returning works plus warnings. Add normalize_doi and deduplicate_by_doi helpers to standardize DOIs and keep the best work per DOI based on citations and year. Implement WorkDistributor to register clients (currently OpenAlex), fan out searches, aggregate and deduplicate works, clamp max_results, sort by relevance/citations/date, and manage client lifecycle via async context manager.	service/app/utils/literature/models.py service/app/utils/literature/base_client.py service/app/utils/literature/doi_cleaner.py service/app/utils/literature/work_distributor.py service/app/utils/literature/__init__.py service/tests/unit/test_literature/test_base_client.py service/tests/unit/test_literature/test_doi_cleaner.py service/tests/unit/test_literature/test_work_distributor.py
Expose the literature search capability as an MCP tool with careful parsing of boolean and numeric inputs plus rich markdown reporting.	Add FastMCP-based literature MCP server with a search_literature tool that builds a SearchRequest and delegates to WorkDistributor. Parse string/boolean inputs for filters like is_oa, is_retracted, has_abstract, has_fulltext and include_abstract, emitting warnings when values are unrecognized instead of failing. Clamp and validate years and max_results, swapping inverted year ranges and adding user-facing warnings when adjustments are made. Format search results into a markdown report that includes conditions, statistics, warnings, a JSON work list (optionally with abstracts), and guidance for refining subsequent searches.	service/app/mcp/literature.py
Add comprehensive unit test coverage for the new literature components and MCP-facing behavior, while leaving some merge conflict markers in tests to be resolved.	Cover OpenAlexClient initialization, search flows with various filters, ID resolution warnings, pagination parameters, abstract reconstruction, work transformation, retry logic, and context manager behavior. Test BaseLiteratureClient protocol behavior, SearchRequest and LiteratureWork dataclasses, DOI normalization edge cases and deduplication rules, and WorkDistributor aggregation/dedup/sorting and error handling. Verify WorkDistributor context manager closes both async and sync clients safely. Note: several new test files still contain unresolved git merge conflict markers (<<<<<<<, =======, >>>>>>>) which must be cleaned up before merging.	service/tests/unit/test_literature/test_openalex_client.py service/tests/unit/test_literature/test_base_client.py service/tests/unit/test_literature/test_doi_cleaner.py service/tests/unit/test_literature/test_work_distributor.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Sorry @SoberPizza, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery