Investigate other gradient-free optimisation algorithms

[ben18785]

Seeing as there is unlikely to be one sampling mode which does best, there is also unlikely to be one (gradient free or otherwise) maximisation mode that fits best. As such, it may be interesting to try and implement some of the following, especially seeing as there will be much that can be borrowed from the MCMC stuff. They also tend to be simpler than sampling algorithms (I think/hope),

• Nelder-Mead (as far as I can tell has no sampling equivalent),
• Differential evolution (differential evolution, DREAM and emcee hammer on the sampling side),
• Simulated annealing (SMC and population MCMC on the sampling side),
• Random search.

I also wonder if we could try and make our own optimisation variants based on multinest/nested sampling for Bayesian problems seeing as this algorithm appears to be particularly good at finding modes.

[MichaelClerx]

Nelder-mead is a good thing to have for local optimisation. Scipy has a version we could wrap, but would be nicer to have something we can use ask-and-tell on. Will investigate!

[MichaelClerx]

See also #55 and #54

[MichaelClerx]

Closed as duplicate of #684