Optionally allow BestSolutionEvents for new, equal score solutions too.

[zhetao1116]

We have a closed issue. Post it here to be discussed with you and shared with a bigger audience who might face the same issue.
#352
I checked the doc of the recommendFit() method. But from my understanding, it sounds not a perfect case if we just want to record all the solutions with equal score. recommendFit() is aimed to solve the problem that after optimizing, if a new planning entity is added, how could we make faster optimizations to assign planning variables to the new planning entity. But in my case, I don't have new planning entity added. I just need the history of the problem solving process.
The proposals from @ge0ffrey all look good, but it sounds hard for me to hack it. Do you have some other suggestions on my issue? How could I make a hack on BestSolutionRecaller(I don't want to fork from your master branch to hack it)? Also BestSolutionRecaller is a final field which cannot be manually overridden outside timfold. Or could you give more insights on how to apply recommendFit() to fit into our use case?
Thanks!

[Christopher-Chianelli]

Well, if you just need the history, you can use DEBUG or TRACE logging, which will show what moves were applied/tried and their corresponding scores. See the docs on logging for details.

If you need the actual solution, you would need to access internal API that is subject to breakage between versions, and probably will considerably slow down the solver as it would most likely significantly increase the number of times the working solution is cloned, and increase memory pressure.

From what I read in the issue, it seem you want simulations; that is, testing how different soft constraint weights change the solution and deciding which set of weights leads to the "best" solution. For instance, assume two soft constraints for VRP:

• Minimize fuel usage
• Minimize the number of left turns

These two constraints go against one other, since in order to avoid left turns, sometimes you need to take a longer route and thus spend more fuel.

So what constraint configuration does is allow users to change the constraint weights depending on business needs. For instance:

• Is a route with 5 left turns and uses 200L of fuel better than a route with 3 left turns and 400L of fuel?
• How about a route with 500 left turns and uses 1500L of fuel better than a route with 450 left turns and 1600L of fuel?

Different businesses will have different ideas on which of the two solutions are "better". How the solver decide which solution is better is by using the constraint weights for "Minimize fuel usage" and "Minimize the number of left turns".

So what you actually want to do is quickly run the solver using different constraint configurations, and show the solution generated by each constraint configuration to the user, and have the user decide based on the solutions produced what constraint configuration to use.

{ "Minimize fuel usage": "-1soft", "Minimize the number of left turns": "-100soft"} would produce the solution "3 left turns and 400L of fuel", whereas { "Minimize fuel usage": "-1soft", "Minimize the number of left turns": "-20soft"} would produce the solution "5 left turns and uses 200L", and these two solutions (and their configurations) would be presented to the user to decide.

[zhetao1116]

Thanks @Christopher-Chianelli ! I understand your point of using constraint configuration. It makes sense if we have complicated problems which always have soft constraints violated. However, in our case, there could be a lot of cases that none of these constraints are violated and hence we could have several solutions that are feasible and acceptable. So in that case, I want to store these actual solutions and return all of them to the user.
You mentioned that I would need to access internal API that is subject to breakage between versions. I'd like to take it even though it might increase the solving time(It's fine in our case as timefold solves our issue very fast now). Could you explain how I could achieve it without fork a branch of timefold?

[triceo]

I really want to advise you against that.

If two solutions have the exact same score, and yet they are meaningfully different to the user, then that's called a score trap; arguably, you should introduce a more fine-grained constraint, so that this situation does not happen. As an additional bonus, the solver will be able to find better solutions, because it will have more information on which to make decisions.

[zhetao1116]

Thanks @triceo for the reply.

[ge0ffrey]

BestSolutionRecaller is a dead end for multi-objective optimization. It won't make your users happy.
Why?

• No UX can help a user pick from 10 (let alone 10 million) solutions if a solution visualizes larger than a single page.
• All solutions will still be found through the same score comparison, the same set of objectives.
• If you do this on only one dataset, you're overfitting.

That's just the tip of the tip of the iceberg. And yet, it is solvable. It will take a while, but we'll tackle it.

[zhetao1116]

Thanks @ge0ffrey for reply. Yeah I understand that BestSolutionRecaller needs to remember hundreds of thousands of solutions with the same score during solving the problem which sounds to be bad. And if the dataset is large, we cannot present it well to the user.
BTW, you mentioned that it is solvable, and you will tackle it in the future. How would you tackle it and when could it be available?

[ge0ffrey]

Early next year at the soonest. We call it "goal alignment" and a part of our designs have just started. It's not a solver feature.

I believe the BestSolutionRecaller approach will do what you want it to do, what your users asked for, but will be 100% useless to them. This is from experience. Goal Alignment is what they actually need, I believe.

[zhetao1116]

Got it! Thanks for information!