Auto report puzzles with multiple solutions #16293
Conversation
|
I like the idea |
Should NOT use `show: modal`, otherwise the dialog promise only resolve once closed
…one in lichess-puzzler v49
Relevant extracts from the code:
```py3
pair_limit = chess.engine.Limit(depth = 50, time = 30, nodes = 25_000_000)
// ...
# is pair.best the only continuation?
def is_valid_attack(self, pair: NextMovePair) -> bool:
return (
pair.second is None or
self.is_valid_mate_in_one(pair) or
win_chances(pair.best.score) > win_chances(pair.second.score) + 0.7
)
```
|
This is now feature-complete, and would gladly take some reviews. The code (especially ts dialog) is harrowing, hoping for some help there. How exactly to store the reports is also left undecided, I've went for separate collection for now so it's much easier to clean the db in case it does not make it, but there're several other options. |
a reported puzzle should be disabled in less than a week. we'll reassess this if we see too many duplicated reports in zulip.
|
It's a lot of code added to You could be using a simple |
curious how it passed typechecking
|
Moved everything to how it looks |
| @@ -37,6 +37,8 @@ final class PuzzleApi( | |||
| def setIssue(id: PuzzleId, issue: String): Fu[Boolean] = | |||
| colls.puzzle(_.updateField($id(id), Puzzle.BSONFields.issue, issue).map(_.n > 0)) | |||
|
|
|||
| val reportDedup = scalalib.cache.OnceEvery[PuzzleId](7.days) | |||
There was a problem hiding this comment.
Since the cache is in heap in practice it'll more likely mean at most once per restart, but should still be good enough
There was a problem hiding this comment.
It also does not check if the puzzle id really exist anymore, but the potential for abuse is low
|
It's a great idea to fuzz our puzzles with user browsers. I would not even hate a completely automated solution where javascript just auto reports them to the server in the background. But I know you're quite sensitive to tracking concerns and this dialog is 1000x better than the nothing we have now. |
When users ceval detect multiple good moves after a certain depth, offer to sent it to zulip like that
[content-site:puzzle reports] [Kraktus](https://lichess.org/@/Kraktus?mod¬es) reported [FJtVd](https://lichess.org/training/FJtVd) because after move 24. Kh2, depth 23, there're multiple solutions, pvs f8f5: -592, a8b8: -428Currently the users need to reach us via the feedback forum/discord and we need to check back, which is time consuming for them and us. Hopefully making it integrated will allow to flag the few faulty puzzles much earlier and with better coverage. cc @TBestLittleHelper for what you'd think.
(The dialog need working)
The
Hide this popup for a weekis very important part, that way in case it is released with too many false positives, people can disable it easily, but once those tweaks are fixed, they will be able to contribute back after a while.The code is pretty much WIP for the time being, but I want to make sure this would be accepted before going forward.
Testing data is a faulty puzzle recently reported on discord:
faulty-puzzle-check.json{"_id":"FJtVd","gameId":"2ceTqBfV","fen":"r4q1k/pQp4p/3p2p1/8/2P2n2/1P3P1P/P4P2/R4RK1 w - - 1 24","themes":["crushing","defensiveMove","sacrifice","long","endgame"],"glicko":{"r":2229.5228893165554,"d":76.8391894980614,"v":0.09000724648992095},"plays":783,"vote":0.9432234168052673,"vu":1061,"vd":31,"line":"g1h2 f8f5 b7a8 h8g7 a8c8 f5c8","cp":629,"users":["bobinsquares","azel1117"]}mongoimport -d lichess -c puzzle2_puzzle faulty-puzzle-check.jsongame-associated-faulty-puzzle-check.jsonmongoimport -d lichess -c game5 game-associated-faulty-puzzle-check.jsonthen
localhost:9663/training/FJtVd