NUMERICAL OPTIMIZATION METHODS
Academic year and teacher
If you can't find the course description that you're looking for in the above list,
please see the following instructions >>
- Versione italiana
- Academic year
- 2020/2021
- Teacher
- VALERIA RUGGIERO
- Credits
- 6
- Didactic period
- Secondo Semestre
- SSD
- MAT/08
Training objectives
- The course deals with some of the modern methods for nonlinear optimization of regular functions. Theoretical basis are provided for unconstrained as well as constrained optimization and some methods are introduced, which are widely used in applied mathematics. The methods are analyzed with respect to their convergence and convergence speed. Some laboratory sessions will be also carried out.
The main knowledges provided by the course are:
- the basic principles of both unconstrained and constrained numerical optimization;
- the meaning and the relevance of descent directions, Karush-Kuhn-Tucker system, constraints qualification, convergence speed;
- the main theoretical results concerning the search for critical points of real functions of one or several real variables;
- some of the main numerical methods for solving unconstrained as well as constrained mathematical programming problems for regular functions;
- numerical solution of some real-world applications.
The main skills (which are the abilitIes to apply the acquired knowledge) will be:
- ability to analyze the type of optimization problem to be solved and its main properties;
- ability to mathematically describe the problem;
- ability to identify the class of solution methods among those presented in the lectures;
- ability to check the necessary conditions for the application of the methods;
- ability to estimate the convergence properties of the methods;
- ability to apply the studied methods to solve simple problems of unconstrained and constrained mathematical programming.
The knowledge and skills acquired can be used in all further classes on applied mathematics. Prerequisites
- Linear algebra, calculus (sequences, limits, derivatives, integrals, study of functions), multidimensional differential calculus, basic knowledge of the Matlab language and environment.
Course programme
- The course includes 42 hours. About 13 hours are dedicated to laboratory activity.
Introduction to unconstrained optimization (6 hours)
Short recall of multidimensional differential calculus. Unconstrained mathematical programming: problem formulation, examples. Formulation of necessary and sufficient conditions of the first and second order, geometric interpretation.
Methods for the unconstrained minimization (10 hours)
Descent methods, line-search method, trust-region technique, convergence and convergence speed of descent methods. The steepest descent method, quasi-Newton methods (BFGS, DFP, SR1). Geometric interpretations and convergence results of numerical methods.
Introduction to constrained optimization (6 hours)
Constrained mathematical programming: problem formulation, examples. Global and local solutions, classification. Constraints qualification conditions. Formulation of necessary and sufficient conditions of the first and second order, Karush-Khun-Tucker (KKT) theorem and geometric interpretation. Duality theory.
Methods for constrained minimization (7 hours)
Lagrange multipliers. The null space method for NLP problems with only equality constraints.
Inequality constraints cases. Elements of penalty methods and sequential quadratic programming (SQP).
Introduction to stochastic optimization methods.
Laboratory activity (13 hours)
Implementation in Matlab of the main methods; numerical solution of some application problems. Didactic methods
- Frontal lectures; laboratory sessions with Matlab.
Learning assessment procedures
- The oral test consists in providing the student with an optimization problem; the student's ability to arrive at the solution with the most appropriate method will be evaluated; ; it is required that the student discusses convergence and efficiency of the method up to the numerical implementation part.
Reference texts
- - J. Nocedal, S. Wright, "Numerical Optimization", 2nd ed., 2006.
- R. Fletcher, "Practical Methods of Optimization", 2nd ed., 2000.
- J. E. Dennis, Jr. and R. B. Schnabel, "Numerical Methods for Unconstrained Optimization and Nonlinear Equations", Classics in Applied Mathematics, SIAM, 1996
- D. P. Bertsekas, "Nonlinear Programming", Second Edition, Athena Scientific, Belmont, Massachusetts, 1999.
