There’s a feature of optimize.differential_evolution that uses optimize.minimize to polish the solution produced by DE. This is controlled by the polish keyword. If polish is True then a suitable minimize method is chosen (‘L-BFGS-B’ or ‘trust-constr’ depending on whether the problem has constraints). There have been several issues in the past (e.g. optimize.differential_evolution: possibility to pass parameters to the polish function · Issue #13561 · scipy/scipy · GitHub) that request better control over the final polishing step.
Another suggested approach is to overload the polish keyword to allow it to accept a callable that uses the minimize interface, i.e. polish_func(fun, x0), and returns an OptimizeResult. In a simple approach one could use functools.partial and minimize. However, it would rely on the user to make sure that the polishing callable still took care of things like bounds and constraints.
Do people have any thoughts on which approach might be preferable?
It looks like this is the type of thing that constrains what the callable solution would look like; a simple functools.partial approach won’t cut it. So it looks like it needs to be prototyped before it’s possible (for me at least) to have an informed opinion here.
@mdhaber in #13561 you suggested using a callable (polish_func(func, x0) ) obeying the minimize interface to overload the polish keyword. Do you still think that’s a good alternative?
I still think it’s better than a minimizer_kwargs argument. But I don’t think my opinion has changed much - neither a minimizer_kwargs argument or polish accepting a callable offer much convenience compared to the user passing the output of the global optimizer to a local optimizer of their choice. IIRC, the idea was offered as a compromise, not because I think it is the ideal design.
Note that if we do go the polish callable route, the original dea was for the callable to satisfy the minimize interface, including acceptingbounds and constraints as arguments. So I thought that the global optimizer would pass it the objective function, the initial guess, the bounds, and any other constraints; the user would only “partial out” minimizer-specific options. This seems different than what is currently implemented in gh-23538.
Polishing was originally added because the route to adding further global optimizers was showing that they could tackle a whole bunch of global optimizer problems at least as well as those we already had. The difficulty with checking you’ve found the global minimum is that the optimizer may be good at finding the location of the global minimum in a reasonable time, but it may not be good at being at the minimum within a high precision. Failing because it’s not within a 1e-7 x-tolerance, but the global minimum was correctly identified is not a true fail. Hence the polishing step.
Note that if we do go the polish callable route, the original dea was for the callable to satisfy the minimize interface, including acceptingbounds and constraints as arguments.
There are a few difficulties with that. It’s entirely possible that the user uses minimize with a method that doesn’t use bounds and constraints, e.g. BFGS. If such a polishing function is sent bounds and constraints there will be a subsequent RuntimeWarning: Method BFGS cannot handle bounds. Moreover, the actual solution may not satisfy those bounds and constraints. So what’s the point in giving them as arguments if the user just ignores them? The demand for flexibility has to come with responsibility as well. Giving them total responsibility for the polish step might be a better approach, and is akin to diffev not having integrated polishing in the first place.
In addition, if bounds and constraints are automatically supplied to the polisher along the lines of:
and the user has already done polish = partial(minimize, bounds=bounds) then there will be an Exception because the same keyword argument has been supplied twice.
EDIT: no exception is raised in this case - wasn’t expecting that.
At the end of the day I would prefer not to make any changes as I believe existing behaviour is mostly sufficient, but we still get issues and PRs being raised. IMO the suggested PR is the best compromise.