Introduction to Stochastic Search and Optimization

Preface.

Stochastic Search and Optimization: Motivation and Supporting Results.

Direct Methods for Stochastic Search.

Recursive Estimation for Linear Models.

Stochastic Approximation for Nonlinear Root-Finding.

Stochastic Gradient Form of Stochastic Approximation.

Stochastic Approximation and the Finite-Difference Method.

Simultaneous Perturbation Stochastic Approximation.

Annealing-Type Algorithms.

Evolutionary Computation I: Genetic Algorithms.

Evolutionary Computation II: General Methods and Theory.

Reinforcement Learning via Temporal Differences.

Statistical Methods for Optimization in Discrete Problems.

Model Selection and Statistical Information.

Simulation-Based Optimization I: Regeneration, Common Random Numbers, and Selection Methods.

Simulation-Based Optimization II: Stochastic Gradient and Sample Path Methods.

Markov Chain Monte Carlo.

Optimal Design for Experimental Inputs.

Appendix A. Selected Results from Multivariate Analysis.

Appendix B. Some Basic Tests in Statistics.

Appendix C. Probability Theory and Convergence.

Appendix D. Random Number Generation.

Appendix E. Markov Processes.

Answers to Selected Exercises.

References.

Frequently Used Notation.

Index.

JAMES C. SPALL is a member of the Principal Professional Staff at the Johns Hopkins University, Applied Physics Laboratory, and is the Chair of the Applied and Computational Mathematics Program within the Johns Hopkins School of Engineering. Dr. Spall has published extensively in the areas of control and statistics and holds two U.S. patents. Among other appointments, he is Associate Editor at Large for the IEEE Transactions on Automatic Control and Contributing Editor for the Current Index to Statistics. Dr. Spall has received numerous research and publications awards and is an elected Fellow of the Institute of Electrical and Electronics Engineers (IEEE).

J. C. Spall, The Johns Hopkins Univ., Laurel, Maryland