Skip to content

ForecastBot Refactor and MC/Numeric Questions #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
CodexVeritas merged 32 commits into from
Dec 18, 2024
Merged

ForecastBot Refactor and MC/Numeric Questions #10

CodexVeritas merged 32 commits into from
Dec 18, 2024

Conversation

@CodexVeritas
Copy link
Owner

@CodexVeritas CodexVeritas commented Dec 18, 2024 •
edited
Loading

This pull request includes several updates and improvements:

  • Refactor from ForecastTeam/ForecastManager to ForecastBot which can be inherited from and act independently
  • Documentation
  • Added Multiple choice and continuous forecasting
  • Added scheduling
  • Other refactors
  • Experimented with Screenshot tool
  • Improved deviation scoring
  • Added tournament simulation notebook

@CodexVeritas
Reorgnanized file structure and added UrlScraper screenshot capability
8ce4621
@CodexVeritas
Added screenshoting to the smartsearcher
196b73a
@CodexVeritas
Incorporated screenshot search into background link parsing
57d732f
@CodexVeritas
Updated key factors deduplication, and renovated the MetaculusAPI
0cf2264
@CodexVeritas
Added question id field to question object
4247052
@CodexVeritas
Added benchmark for screenshot feature, and a few touch ups
1b67ed5
@CodexVeritas
Changed the smart searcher back to pre-screenshot config
ed2053f
@CodexVeritas
Added screenshot-with-smart-searcher reverted benchamrk
1775850
@CodexVeritas
Added jupyter notebook support
9b81110
@CodexVeritas
Added dspy to poetry
1da3210
@CodexVeritas
Created 'simulate-a-tournament' notebook
ff9357c
@CodexVeritas
Tested more scoring method with the tournament simulation notebook
24a73f7
@CodexVeritas
Added benchmark files for gpt4o and o1-preview
5508518
@CodexVeritas
Added benchmark files for o1-preview that I missed
9201130
@CodexVeritas
Removed URL Scraper and summary that was added to research_coordinator
89d7780
@CodexVeritas
Replaced spaces with %20 in smart searcher
9a2de50
@CodexVeritas
Removed the 'exclude text' from exa searcher
1a8040a
@CodexVeritas
Renamed unit test folder
763301b
@CodexVeritas
Changed deviation score caculation to use expected log score
d9e8a00
@CodexVeritas
Renamed tournament forecast script
6d2da30
@CodexVeritas
Got initial interface for forecast bot finished and team_manager repl…
70bfd38 
…aced
@CodexVeritas
Got first draft of default bot, and numeric distribution code written
525c63d
@CodexVeritas
Removed forecastTeam, FinalDecisionAgent, and TeamManager. Imports an…
ff95342 
…d tests reworked
@CodexVeritas
All tests passing except for runing and publishing the new numeric an…
ee41126 
…d mc forecasts
@CodexVeritas
Got mc/binary/numeric prediction and posting working for template bot
faf0387
@CodexVeritas
Got the publish to tournament script working
2bae01c
@CodexVeritas
Refactored create_unified_explanation
2d1ec91
@CodexVeritas
Did a number of refactors, added github action, started documentation
d0a2d8a
@CodexVeritas
Finished adding new tests for reports and bots
f83a93a
@CodexVeritas
Finished adding new tests for reports and bots
33a097c
@CodexVeritas CodexVeritas merged commit 5d3bdba into main Dec 18, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@CodexVeritas