Skip to content

Commit

Permalink
add explanation table for settings, limit search to twice having the …
Browse files Browse the repository at this point in the history
…same query, add agentic default to high_quality
  • Loading branch information
mskarlin committed Sep 11, 2024
1 parent e673c43 commit 618005c
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 4 deletions.
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,21 @@ or I can use the normal ask
pqa -i nanomaterials ask 'Are there nm scale features in thermoelectric materials?'
```

Both the CLI and module have pre-configured settings based on prior performance and our publications, they can be invoked as follows:

```
pqa --settings <setting name> ask 'Are there nm scale features in thermoelectric materials?'
```

Here is a table of some useful settings:

| Setting Name | Description |
| ------------ | ---------------------------------------------------------------------------------------------------------------------------- |
| high_quality | Highly performant, relatively expensive (due to having `evidence_k` = 15) query using a `ToolSelector` agent. |
| fast | Setting to get answers cheaply and quickly. |
| wikicrow | Setting to emulate the Wikipedia article writing used in our WikiCrow publication. |
| contracrow | Setting to find contradictions in papers, your query should be a claim that needs to be flagged as a contradiction (or not). |

### Module Usage

PaperQA's full workflow can be accessed via Python directly:
Expand Down
7 changes: 3 additions & 4 deletions paperqa/agents/tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ async def paper_search(
"""
Search for papers to increase the paper count.
Repeat previous calls with the same query and years to continue a search.
Repeat previous calls with the same query and years to continue a search. Only repeat a maximum of twice.
This tool can be called concurrently.
This tool introduces novel papers, so invoke this tool when just beginning or when unsatisfied with the current evidence.
Expand Down Expand Up @@ -210,9 +210,7 @@ class GenerateAnswer(NamedTool):
def did_not_fail_to_answer(cls, message: str) -> bool:
return not message.startswith(cls.FAILED_TO_ANSWER)

async def gen_answer(
self, question: str, state: EnvironmentState # noqa: ARG002
) -> str:
async def gen_answer(self, question: str, state: EnvironmentState) -> str:
"""
Ask a model to propose an answer using current evidence.
Expand All @@ -224,6 +222,7 @@ async def gen_answer(
question: Question to be answered.
state: Current state.
"""
logger.info(f"Generating answer for '{question}'.")
# TODO: Should we allow the agent to change the question?
# self.answer.question = query
state.answer = await state.docs.aquery(
Expand Down
3 changes: 3 additions & 0 deletions paperqa/configs/high_quality.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,8 @@
},
"prompts": {
"use_json": true
},
"agent": {
"agent_type": "ToolSelector"
}
}

0 comments on commit 618005c

Please sign in to comment.