feat: Weighted majority voting for classification domain ensemble

[Ryuketsukami]

Problem

ensemble.py currently selects the single best-scoring agent from the archive and returns its prediction for each question. This is argmax selection, not ensembling — it cannot benefit from diversity across agents.

For classification domains (search_arena: Accept/Reject, paper_review: Accept/Reject, imo_grading: 4-class), a globally best agent may be wrong on specific question types where a lower-scoring agent would answer correctly. Majority voting is a well-established baseline that exploits archive diversity.

The ensemble score feeds into parent selection (type="max" in get_saved_score), so a better ensemble would also improve the evolutionary search dynamics.

Solution

For classification domains (search_arena, paper_review, imo_grading), implements weighted majority voting:

1. Gets top-3 agents from archive by score
2. For each question, collects predictions from all 3 agents
3. Accumulates weighted votes (weight = agent's eval score)
4. Returns the prediction with highest total weight

Falls back to single-best when:

• Fewer than 3 agents exist in the archive
• Domain is not classification-type
• can_domain_ensembled() returns False

The meta-agent can still evolve ensemble.py — this improves the initialization, not the ceiling.

Tests

Added tests/test_ensemble.py (12 tests):

• Majority vote logic (simple majority, weight override, ties, None handling)
• Domain gating (classification vs non-classification)
• Fallback to single-best with <3 agents
• can_ensemble flag blocks voting

Question for maintainers

Was the argmax ensemble chosen as a deliberately minimal baseline for the self-improvement to iterate on? If so, this change still preserves that property — the meta-agent can still modify ensemble.py freely. The improvement is just a stronger starting point. If there's a reason to keep the baseline minimal (e.g., to measure the delta of self-improvement from a weaker start), let us know and we can revert.

[meta-cla]

Hi @Ryuketsukami!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!