Dynamic programming and optimal control 3rd pdf download

Dynamic Programmingand Optimal ControlVolume ITHIRD EDITIONP. BertsekasMassachusetts Institute of TechnologyWWW site for book information andwww.cronistalascolonias.com.ar Athena Scientific, Belmont,Athena ScientificPost Office Box NH www.cronistalascolonias.com.ar: info@www.cronistalascolonias.com.ar: www.cronistalascolonias.com.ar:mCover Design: Ann Gallager, www.cronistalascolonias.com.ar , , Dimitri P. BertsekasAll rights reserved. No part of this book may be reproduced in any formby ~ 1 I l ~ electronic or mechanical means (including photocopying, recording,or mlormation storage and retrieval) without permission in writing fromthe www.cronistalascolonias.com.arher's Cataloging-in-Publication DataBertsekas, Dimitri www.cronistalascolonias.com.arc Programming and Optimal ControlIncludes Bibliography and Index1. Mathematical Optimization. 2. Dynamic Programming. L www.cronistalascolonias.com.ar ISBN ABOUT THE AUTHORDimitri Bertsekas studied Mechanical and Electrical Engineering atthe National Technical University of Athens, Greece, and obtained hisPh.D. in system science from the Massachusetts Institute of Technology. Hehas held faculty positions with the Engineering-Economic Systems Dept.,Stanford University, and the Electrical Engineering Dept. of the Univer-sity of Illinois, Urbana. Since he has been teaching at the ElectricalEngineering and Computer Science Department of the Massachusetts In-stitute of Technology (www.cronistalascolonias.com.ar), where he is currently McAfee Professor www.cronistalascolonias.com.ar research spans several fields, including optimization, control, la,rge-scale computation, and data communication networks, and is closely tiedto his teaching and book authoring activities. He has written llUInerousresearch papers, and thirteen books, several of which are used as textbooksin MIT classes. He consults regularly with private industry and has heldeditorial positions in several www.cronistalascolonias.com.arsor Bertsekas was awarded the INFORMS Prize for H,e-search Excellence in the Interface Between Operations Research and Com-puter Science for his book "Neuro-Dynamic Programming" (co-authoredwith John Tsitsiklis), the Greek National Award for Operations Re-search, and the ACC John R. Ragazzini Education Award. In ,he was elected to the United States National Academy of www.cronistalascolonias.com.ar SCIENTIFICOPTIMIZATION AND COl\1PUTATION SERIES1. Convex Analysis and Optimization, by Dimitri P. Bertsekas, withAngelia Nedic and Asuman E. Ozdaglar, , ISBN , pages2. Introduction to Probability, by Dimitri P. Bertsekas and John www.cronistalascolonias.com.arklis, , ISBN X, pages3. Dynamic Programming and Optimal Control, Two-Volume Set,by Dimitri P. Bertsekas, , ISBN , pages4. Nonlinear Programming, 2nd Edition, by Dimitri P. Bertsekas,, ISBN , pages5. Network Optimization: Continuous and Discrete Models, by Dim-itri P. Bertsekas, , ISBN , pages6. Network Flows and Monotropic Optimization, by R. Tyrrell Rock-areUar, , ISBN X, pages7. Introduction to Linear Optimization, by Dimitris Bertsimas andJohn N. Tsitsiklis, , ISBN , pages8. Parallel and Distributed Computation: Numerical Methods, byDimitri P. Bertsekas and John N. Tsitsiklis, , ISBN , pages9. Neuro-Dynamic Programming, by Dimitri P. Bertsekas and JohnN. Tsitsiklis, , ISBN , pages Constra,ined Optimization and Lagrange Multiplier Methods, byDimitri P. Bertsekas, , ISBN f, pages Stochastic Optirnal Control: The Discrete-Time Case, by DimitriP. Bertsekas and Steven E. Shreve, , ISBN , pagesContents1. The Dynamic Programming Algorithm Introduction . . . . . . . . The Basic Problem. . . . . . . . . The Dynamic Programming Algorithm State Augmentation and Other Reformulations Some Mathematical Issues . . . . . . . Dynamic Prograrnming and Minimax Control Notes, Sources, and Exercises . . . . . . Deterministic Systems and the Shortest Path Probleln Finite-State Systems and Shortest Paths Some Shortest Path Applications Critical Path Analysis Hidden Markov Models and the Viterbi Algorithm Shortest Path Algorithms . . . . . . . . . . Label Correcting Methods. . . . . . . Label Correcting Variations - A* Algorithm Branch-and-Bound . . . . . . . . . Constrained and Multiobjective Problems Notes, Sources, and Exercises Deterministic Continuous-Time Continuous-Time Optimal Control The Hamilton-Jacobi-Bellman Equation The Pontryagin Minimum Principle An Informal Derivation Using the HJB Equation A Derivation Based on Variational Ideas Minimum Principle for Discrete-Time Problems Extensions of the Minimum Principle Fixed Terminal State Free Initial Statep. 2p. 12p. 18ppp. 46pp. 64p. fl8p. 68ppp. 78p. 87ppp. 97ppppp. pp. pp Control Certainty Equivalent and Adaptive Control p. G.l.l. Caution, Probing, and Dual Control p. Two-Phase Control and Identifiability p. Certainty Equivalent Control and Identifiability p. Self-Tuning Regulators p. G Open-Loop Feedback Control . . . . . . . . . " p. Limited Lookahead Policies . . . . . . . . . . .. p. Performance Bounds for Limited Lookahead Policies p. Computational Issues in Limited Lookahead . . . p. G Problem Approximation - Enforced Decomposition p. Aggregation . . . . . . . . . . . . p. Parametric Cost-to-Go Approximation p. Rollout Algorithms. . . . . . . . . . p. Discrete Deterministic Problems . p. Q-Factors Evaluated by Simulation p Q-Factor Approximation p. vi Free Terminal Time Time-Varying System and Cost Singular Problems . . Notes, Sources, and Exercises . . . Problellls with Perfect State Information Linear Systems and Quadratic Cost Inventory ControlL Dynamic Portfolio Analysis . . . Optimal Stopping Problems . . . Scheduling and the Interchange Argument Set-Membership Description of Uncertainty Set-Membership Estimation . . . Control with Unknown-but-Bounded Disturbances Notes, Sources, and Exercises . . . . . . . . . . . Problen'ls with Imperfect State Information Reduction to the Perfect Information Case Linear Systems and Quadratic Cost Minimum Variance Control of Linear Systems SufIicient Statistics and Finite-State Markov Chains The Conditional State Distribution Finite-State Systems Notes, Sources, and ExercisesContentspppppp. ppp. pppppppppppContents Model Predictive Control and Related Methods Rolling Horizon Approximations . . . Stability Issues in Model Predictive Control Restricted Structure Policies . Additional Topics in Approximate DP Discretization . . . . . . . Other Approximation Approaches Notes, Sources, and Exercises . . . . Introduction to Infinite Horizon Problems An Overview . . . . . . . . Stochastic Shortest Path Problems Discounted Problems . . . . . Average Cost per Stage Problems Semi-Markov Problems . . Notes, Sources, and Exercises . .Appendix A: Mathematical ReviewA Sets .A Euclidean Space.A Matrices . . . .A Analysis . . . .A Convex Sets and FunctionsAppendix B: On Optimization TheoryB Optimal Solutions . . . . . . .B Optimality Conditions . . . . .B Minimization of Quadratic J:iormsAppendix C: On Probability TheoryC.l. Probability Spaces. . .C Random VariablesC Conditional ProbabilityAppendix D: On Finite-State Markov ChainsD.l. Stationary Markov ChainsD Classification of StatesD Limiting ProbabilitiesD First Passage Times .viip. ppp. p. p. p. 38L1p. pppppp. pppp. pppppp. p. ppppG: Forrnulating Problems of Decision Under Uncer-viiiP'C'A.a'www.cronistalascolonias.com.ar. E: Kalman FilteringE.l. Least-Squares Estimation .E Linear Least-Squares EstimationE . ~ 1 . State Estimation Kalman FilterE Stability Aspects . . . . . . .E Gauss-Markov EstimatorsE Deterministic Least-Squares EstimationAppendix F: lVIodeling of Stochastic Linear SystemsF Linear Systems with Stochastic In