Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modified newtons method #1068

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

strMikhailPotapenko
Copy link

@strMikhailPotapenko strMikhailPotapenko commented Mar 1, 2024

My proposed solution which resolves #1067.

I added an option to Newton Minimizer which forces the Hessian to be positive definite by reversing negative eigenvalues. This method is described in

Philip E. Gill, Walter Murray, and Margaret H. Wright, Practical Optimization, 1981, 107–8.

I also added a unit test, FindMinimum_SixHumpCamel_IndefiniteHessian(), to demonstrate the benefit of this modification. If the optional argument is changed to HessianModifiers.None then the test fails.

Please see the issue for further illustration of the use case.

I made the optional argument an enum because this method of forcing positive definiteness can be improved upon in terms of efficiency and this gives some future developer the hooks to be able to do so.

Regards,
Mikhail

… guess so that the following conditions are all met:

1. Hessian is indefinite
2. Directional Derivative is negative
3. The initial guess is between a local maximum and a global minimum

This causes Newton's method with and without line search to converge to the local maximum
…te by reversing negative eigenvalues. Added optional enum parameter to NewtonMinimizer class to allow for future implementations of other methods of Hessian modification.
@strMikhailPotapenko
Copy link
Author

@cdrnet, Would it be possible to get this pull request reviewed?

@strMikhailPotapenko
Copy link
Author

@jvangael, @jkalias, @JohanLarsson, I think you are also listed as member of the Math.Net team. Would any of you be willing to consider this pull request?

@jkalias
Copy link
Member

jkalias commented Mar 25, 2024

@strMikhailPotapenko I would be more than happy to. The problem is that I can only merge PRs for the MathNet.Spatial project, for the Numerics only @cdrnet has the relevant rights to.

@strMikhailPotapenko
Copy link
Author

@jkalias Ah I didn't know that, sorry about that and thank you for the response.

Just looking at open issues and pull requests it does seem like there isn't a lot of activity with this repo, which is too bad because I have been using this library quite often lately and have been pretty satisfied with it. Do you happen to have any information on the plans of @cdrnet as far as actively maintaining this repo or any way to request that information from him?

@jkalias
Copy link
Member

jkalias commented Mar 28, 2024

@strMikhailPotapenko nothing to be sorry about :)

I have no information on whether/when @cdrnet will continue to actively maintain this repo. On the other hand, it's an open source project, so there is no obligation to do so, and rightfully so.

@strMikhailPotapenko
Copy link
Author

@jkalias Very true!

Copy link
Member

@jkalias jkalias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find the documentation and the links to the original paper excellent!

src/Numerics/Optimization/HessianModifiers.cs Outdated Show resolved Hide resolved
src/Numerics/Optimization/NewtonMinimizer.cs Outdated Show resolved Hide resolved
src/Numerics/Optimization/NewtonMinimizer.cs Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Modify NewtonMinimizer so that it forces Hessian to be positive definite
2 participants