1*bf2c3715SXin Li// This file is part of Eigen, a lightweight C++ template library 2*bf2c3715SXin Li// for linear algebra. 3*bf2c3715SXin Li// 4*bf2c3715SXin Li// Copyright (C) 2009 Thomas Capricelli <[email protected]> 5*bf2c3715SXin Li// 6*bf2c3715SXin Li// This Source Code Form is subject to the terms of the Mozilla 7*bf2c3715SXin Li// Public License v. 2.0. If a copy of the MPL was not distributed 8*bf2c3715SXin Li// with this file, You can obtain one at http://mozilla.org/MPL/2.0/. 9*bf2c3715SXin Li 10*bf2c3715SXin Li#ifndef EIGEN_NONLINEAROPTIMIZATION_MODULE 11*bf2c3715SXin Li#define EIGEN_NONLINEAROPTIMIZATION_MODULE 12*bf2c3715SXin Li 13*bf2c3715SXin Li#include <vector> 14*bf2c3715SXin Li 15*bf2c3715SXin Li#include "../../Eigen/Core" 16*bf2c3715SXin Li#include "../../Eigen/Jacobi" 17*bf2c3715SXin Li#include "../../Eigen/QR" 18*bf2c3715SXin Li#include "NumericalDiff" 19*bf2c3715SXin Li 20*bf2c3715SXin Li/** 21*bf2c3715SXin Li * \defgroup NonLinearOptimization_Module Non linear optimization module 22*bf2c3715SXin Li * 23*bf2c3715SXin Li * \code 24*bf2c3715SXin Li * #include <unsupported/Eigen/NonLinearOptimization> 25*bf2c3715SXin Li * \endcode 26*bf2c3715SXin Li * 27*bf2c3715SXin Li * This module provides implementation of two important algorithms in non linear 28*bf2c3715SXin Li * optimization. In both cases, we consider a system of non linear functions. Of 29*bf2c3715SXin Li * course, this should work, and even work very well if those functions are 30*bf2c3715SXin Li * actually linear. But if this is so, you should probably better use other 31*bf2c3715SXin Li * methods more fitted to this special case. 32*bf2c3715SXin Li * 33*bf2c3715SXin Li * One algorithm allows to find a least-squares solution of such a system 34*bf2c3715SXin Li * (Levenberg-Marquardt algorithm) and the second one is used to find 35*bf2c3715SXin Li * a zero for the system (Powell hybrid "dogleg" method). 36*bf2c3715SXin Li * 37*bf2c3715SXin Li * This code is a port of minpack (http://en.wikipedia.org/wiki/MINPACK). 38*bf2c3715SXin Li * Minpack is a very famous, old, robust and well renowned package, written in 39*bf2c3715SXin Li * fortran. Those implementations have been carefully tuned, tested, and used 40*bf2c3715SXin Li * for several decades. 41*bf2c3715SXin Li * 42*bf2c3715SXin Li * The original fortran code was automatically translated using f2c (http://en.wikipedia.org/wiki/F2c) in C, 43*bf2c3715SXin Li * then c++, and then cleaned by several different authors. 44*bf2c3715SXin Li * The last one of those cleanings being our starting point : 45*bf2c3715SXin Li * http://devernay.free.fr/hacks/cminpack.html 46*bf2c3715SXin Li * 47*bf2c3715SXin Li * Finally, we ported this code to Eigen, creating classes and API 48*bf2c3715SXin Li * coherent with Eigen. When possible, we switched to Eigen 49*bf2c3715SXin Li * implementation, such as most linear algebra (vectors, matrices, stable norms). 50*bf2c3715SXin Li * 51*bf2c3715SXin Li * Doing so, we were very careful to check the tests we setup at the very 52*bf2c3715SXin Li * beginning, which ensure that the same results are found. 53*bf2c3715SXin Li * 54*bf2c3715SXin Li * \section Tests Tests 55*bf2c3715SXin Li * 56*bf2c3715SXin Li * The tests are placed in the file unsupported/test/NonLinear.cpp. 57*bf2c3715SXin Li * 58*bf2c3715SXin Li * There are two kinds of tests : those that come from examples bundled with cminpack. 59*bf2c3715SXin Li * They guaranty we get the same results as the original algorithms (value for 'x', 60*bf2c3715SXin Li * for the number of evaluations of the function, and for the number of evaluations 61*bf2c3715SXin Li * of the Jacobian if ever). 62*bf2c3715SXin Li * 63*bf2c3715SXin Li * Other tests were added by myself at the very beginning of the 64*bf2c3715SXin Li * process and check the results for Levenberg-Marquardt using the reference data 65*bf2c3715SXin Li * on http://www.itl.nist.gov/div898/strd/nls/nls_main.shtml. Since then i've 66*bf2c3715SXin Li * carefully checked that the same results were obtained when modifying the 67*bf2c3715SXin Li * code. Please note that we do not always get the exact same decimals as they do, 68*bf2c3715SXin Li * but this is ok : they use 128bits float, and we do the tests using the C type 'double', 69*bf2c3715SXin Li * which is 64 bits on most platforms (x86 and amd64, at least). 70*bf2c3715SXin Li * I've performed those tests on several other implementations of Levenberg-Marquardt, and 71*bf2c3715SXin Li * (c)minpack performs VERY well compared to those, both in accuracy and speed. 72*bf2c3715SXin Li * 73*bf2c3715SXin Li * The documentation for running the tests is on the wiki 74*bf2c3715SXin Li * http://eigen.tuxfamily.org/index.php?title=Tests 75*bf2c3715SXin Li * 76*bf2c3715SXin Li * \section API API: overview of methods 77*bf2c3715SXin Li * 78*bf2c3715SXin Li * Both algorithms needs a functor computing the Jacobian. It can be computed by 79*bf2c3715SXin Li * hand, using auto-differentiation (see \ref AutoDiff_Module), or using numerical 80*bf2c3715SXin Li * differences (see \ref NumericalDiff_Module). For instance: 81*bf2c3715SXin Li *\code 82*bf2c3715SXin Li * MyFunc func; 83*bf2c3715SXin Li * NumericalDiff<MyFunc> func_with_num_diff(func); 84*bf2c3715SXin Li * LevenbergMarquardt<NumericalDiff<MyFunc> > lm(func_with_num_diff); 85*bf2c3715SXin Li * \endcode 86*bf2c3715SXin Li * For HybridNonLinearSolver, the method solveNumericalDiff() does the above wrapping for 87*bf2c3715SXin Li * you. 88*bf2c3715SXin Li * 89*bf2c3715SXin Li * The methods LevenbergMarquardt.lmder1()/lmdif1()/lmstr1() and 90*bf2c3715SXin Li * HybridNonLinearSolver.hybrj1()/hybrd1() are specific methods from the original 91*bf2c3715SXin Li * minpack package that you probably should NOT use until you are porting a code that 92*bf2c3715SXin Li * was previously using minpack. They just define a 'simple' API with default values 93*bf2c3715SXin Li * for some parameters. 94*bf2c3715SXin Li * 95*bf2c3715SXin Li * All algorithms are provided using two APIs : 96*bf2c3715SXin Li * - one where the user inits the algorithm, and uses '*OneStep()' as much as he wants : 97*bf2c3715SXin Li * this way the caller have control over the steps 98*bf2c3715SXin Li * - one where the user just calls a method (optimize() or solve()) which will 99*bf2c3715SXin Li * handle the loop: init + loop until a stop condition is met. Those are provided for 100*bf2c3715SXin Li * convenience. 101*bf2c3715SXin Li * 102*bf2c3715SXin Li * As an example, the method LevenbergMarquardt::minimize() is 103*bf2c3715SXin Li * implemented as follow: 104*bf2c3715SXin Li * \code 105*bf2c3715SXin Li * Status LevenbergMarquardt<FunctorType,Scalar>::minimize(FVectorType &x, const int mode) 106*bf2c3715SXin Li * { 107*bf2c3715SXin Li * Status status = minimizeInit(x, mode); 108*bf2c3715SXin Li * do { 109*bf2c3715SXin Li * status = minimizeOneStep(x, mode); 110*bf2c3715SXin Li * } while (status==Running); 111*bf2c3715SXin Li * return status; 112*bf2c3715SXin Li * } 113*bf2c3715SXin Li * \endcode 114*bf2c3715SXin Li * 115*bf2c3715SXin Li * \section examples Examples 116*bf2c3715SXin Li * 117*bf2c3715SXin Li * The easiest way to understand how to use this module is by looking at the many examples in the file 118*bf2c3715SXin Li * unsupported/test/NonLinearOptimization.cpp. 119*bf2c3715SXin Li */ 120*bf2c3715SXin Li 121*bf2c3715SXin Li#ifndef EIGEN_PARSED_BY_DOXYGEN 122*bf2c3715SXin Li 123*bf2c3715SXin Li#include "src/NonLinearOptimization/qrsolv.h" 124*bf2c3715SXin Li#include "src/NonLinearOptimization/r1updt.h" 125*bf2c3715SXin Li#include "src/NonLinearOptimization/r1mpyq.h" 126*bf2c3715SXin Li#include "src/NonLinearOptimization/rwupdt.h" 127*bf2c3715SXin Li#include "src/NonLinearOptimization/fdjac1.h" 128*bf2c3715SXin Li#include "src/NonLinearOptimization/lmpar.h" 129*bf2c3715SXin Li#include "src/NonLinearOptimization/dogleg.h" 130*bf2c3715SXin Li#include "src/NonLinearOptimization/covar.h" 131*bf2c3715SXin Li 132*bf2c3715SXin Li#include "src/NonLinearOptimization/chkder.h" 133*bf2c3715SXin Li 134*bf2c3715SXin Li#endif 135*bf2c3715SXin Li 136*bf2c3715SXin Li#include "src/NonLinearOptimization/HybridNonLinearSolver.h" 137*bf2c3715SXin Li#include "src/NonLinearOptimization/LevenbergMarquardt.h" 138*bf2c3715SXin Li 139*bf2c3715SXin Li 140*bf2c3715SXin Li#endif // EIGEN_NONLINEAROPTIMIZATION_MODULE 141