随机过程是对随时间和空间变化的随机现象进行建模和分析的学科。许多年前，我们不能在现实问题求解中应用随机过程，但随着数值方法和计算工具的快速发展，这种状况已经发生了变化。本书很好地将计算机的使用和随机过程教学结合起来，采用MATLAB的计算机解题方法，使本书充满现代感，又具备实用的特点。本书采用面向应用和计算的方式，强调通过各种示例和习题来开发学生在随机建模和分析中的实战能力，同时将计算的任务交给计算机去完成。
　　本书是为那些有兴趣学习随机过程的概念、模型和计算方法的学生编写的，是随机过程课程的入门教材，适合管理、金融、工程、统计、计算机科学和应用数学等专业的高年级本科生或低年级研究生阅读。
　

Edward P．C．Kao于斯坦福大学获得博士学位：现为休斯敦大学工商管理学院决策与信息科学系教授。

This is an introductory book on stochastic processes--a subject about mod-eling and analysis of random phenomena occurring over time or space.
　　Many years ago, we could not do stochastic processes in a serious way in the con-text of real-world problem solving. The rapid advancements in numerical meth-ods and computing facilities have profoundly changed the landscape. This text responds to the challenges of incorporating computer use in the teaching and learning of stochastic process.
　　This book is written for students who are interested in learning concepts, mod-els, and computational approaches in stochastic processes. The intended audience includes upper-level undergraduates and first-year graduate students in operations research, management science, finance, engineering, statistics, computer science,and .applied mathematics. The prerequisites for the text are intermediate-level cal-culus, elementary linear algebra, and an introductory course in probability with an emphasis in operational skills on conditioning.
　　This book takes an application and computation oriented approach instead of the standard formal and mathematically rigorous approach. The emphasis is on the development of operational skills in stochastic modeling and analysis through a variety of examples drawn from diverse areas while relegating the burden of computation to its rightful master-the computer. Following our approach, we are able to present many topics of practical importance in detail at a very early
stage. One such example is the study of a time-dependent service system covered in Chapter 2. There we see that once the model is constructed the time-dependent solutions of the system of differential equations with time-varying parameters can be obtained rather convenienfiy on a computer.
　　Organization and CoverageThe book covers standard topics in a first course in stochastic processes. It also includes some additional materials reflecting recent development in computational probability. The first chapter reviews some preliminary materials. They include a brief introduction, transform methods, and some basic concepts in mathematical analysis. Chapters 2 to 6 are organized in a logical sequence. We start with a Poisson process and its variants in Chapter 2, and move to a more general counting process called the renewal process in Chapter 3. To model dependency in random phenomena, we study discrete-time and continuous-time Markov chains in Chapters 4 and 5, respectively. The Markov renewal process presented in Chapter 6 can be considered as the generalization of most models studied earlier. The capstone of all these is the semi-regenerative process in Sec-tion 6.4. The last chapter is about Brownian motion, diffusion processes, and Ito's lemmas. The chapter also contains applications of diffusion process in finance.
　　The book provides a great deal of flexibility for instructors. For students in business and management, Chapters 1-5 should provide a good introduction to stochastic models in management science. For students majoring in finance, the first few sections of Chapters 2-5 along with the last chapter will give them the preliminary background in stochastic processes for further study in continuous-time finance. For students in computer science, electrical and computer engineering, or operations management who want to acquire some knowledge about Markovian service systems necessary for performance evaluations of communi-cation systems, computer networks, or automated manufacturing systems they will find Chapters 2, 4, and 5 useful. In order to reach Section 5.8 on queueing　networks in a one-semester course, instructors may choose to skip Sections 5.5-5.7. For students in industrial and systems engineering and operation research who eventually will study queueing theory beyond Markovian models,knowing the materials in Chapter 6 would be helpful. To cover the entire book, a two-semester sequence can be considered with Chapters 1-4 in the first semester and Chapters 5-7 in the second. More difficult examples and problems are marked with an asterisk (*).
　　A solution manual is available from the publisher for instructors who adopt this text for a course. Readers are welcome to use the perforated card in the back of the book to contact Math Works, Inc. for a diskette containing the MATLAB programs listed in the appendices.

MATLAB
　　The software chosen for this text is MATLAB. MATLAB is easy to learn and numerically reliable. It is most suitable for solving problems involving matrices.In many homework problems, students are expected to experiment with their models and solution procedures with the aid of MATLAB. Of course, Mathemat-ica or Maple can also be used to accomplish the same for those who are con-versant with and have access to these software.
　　A brief tutorial on MATLAB is given at the end of the text. For more infor-marion about the MATLAB software, readers may contact: The Math Works, Inc.24 Prime Park Way, Natick, MA 01760-1500, E-Mail: info@mathworks.com, WWW: http://www, mathworks.com. We emphasize that the MATLAB programs　shown at the end of each chapter were for illustrative purposes and no attempts were made to optimize the codes.

Acknowledgments
　　I am deeply grateful to Professor Wayne L. Winston, Indiana University, who provided the initial encouragement and a continuing stream of comments and suggestions during the early development of the text. He generously shared his own class notes on Brownian motion and Ito's lemmas with me. In Chapter 7, the part relating to continuous-time finance was greatly influenced by his notes. The feedback from his use of the manuscript in the spring of 1994 in a course on
stochastic processes was very helpful. I would like to thank Professor Xiuli Chao,New Jersey Institute of Technology, for his help on queueing networks. My brother Dr. Peichuen Kao, AT&T Bell Labs, read many parts of the original manuscript and whose incisive remarks improved the clarity of a number of argu-ments. Many of my students at the University of Houston who have read prelim-inary versions of this text and offered numerous suggestions. In particular, I would like to thank Marvin A. Arostegui, Miguel A. Caceres, Calvin Chen, Jinhu Qian, Meng Rui, Nicola Secomandi, Bradley D. Silver, and Sandra D. Wilson for their many contributions.
　　Thanks to the reviewers of the manuscript: Professor Apostolos Burnetas,Case Western Reserve University; Professor Ralph L. Disney, Texas A&M Uni-versity; Professor Halina Frydman, New York University; Professor Carl M. Har-ris, George Mason University; Professor Vien Nguyen, Massachusetts Institute of Technology; Professor P. Simin Pulat, University of Oklahoma; Professor Shaler Stidham, Jr., University of North Carolina; Professor Wayne L. Winston, Indiana
University; and Professor Shelley Zacks, SUNY at Binghamton. Their thoughtful comments and suggestions played an important role in shaping the final version of the manuscript. Finally, I would like to express my appreciation to the staff at Duxbury Press: Editor Curt Hinrichs, Production Editor Jerry Holloway, Editorial Assistant Cynthia Mazow, and Project Development Editors Jennifer Burger and Julie McDonald. Thanks are also due to the staff at Shepherd, Inc. who did the edi- torial and composition work of the book, in particular, Editor Patricia Noble.
　　Christina Palumbo and Noami Bulock at The MathWorks, Inc. provided excellent support in my use of MATLAB. The MATLAB Tutorial shown at the end of the book benefited by expert feedback from the staff at The MathWorks.
　　While I was fortunate to receive the help from many people in writing and improving this text, I bear responsibility for any errors and would appreciate hearing about them.
　　Edward P. C. Kao Department of Decision and Information Science University of Houston

Houston, TX 77204-6282
E-mail: ekao@uh, edu
March 1996

1　　Introduction　1
1.0　Overview　2
1.1　Introduction　2
1.2　Discrete Random Variables and Generating Functions　6
1.3　Continuous Random Variables and Laplace Transforms　17
1.4　Some Mathematical Background　28
Problems　37
Bibliographic Notes　42
References　43
Appendix　43
2　Poisson Processes　47
2.0　Overview　47
2.1　Introduction　48
2.2　Properties of Poisson Processes　51
2.3　Nonhomogeneous Poisson Processes　56
2.4　Compound Poisson Processes　72
2.5　Filtered Poisson Processes　76
2.6　Two-Dimensional and Marked Poisson Processes　80
2.1　Poisson Arrivals See Time Averages (PASTA)　83
Problems　87
Bibliographic Notes　93
References　94
Appendix　95
enewal Processes　97
3　　　　
3.0　Overview　97
3.1　Introduction　98
3.2　Renewal-Type Equations　101
3.3　Excess Life, Current Life, and Total Life　107
3.4　Renewal Reward Processes　118
3.5　Limiting Theorems, Stationary and Transient Renewal Processes　128
3.6　Regenerative Processes　132
3.7　Discrete Renewal Processes　144
Problems　146
Bibliographic Notes　154
References　155
Appendix　156
iscrete-Time Markov Chains　160
4.0　Overview　160
4.1　Introduction　161
4.2　Classification of States　167
4.3　Ergodic and Periodic Markov Chains　175
4.4　Absorbing Markov Chains　188
4.5　Markov Reward Processes　203
4.6　Reversible Discrete-Tune Markov Chains　207
Problems 212
Bibliographic Notes　225
References　226
Appendix　227
ontinuous-Time Markov Chains　238
5.0　Overview　239
5.1　Introduction　239
5.2　The Kolmogorov Differential Equations　245
5.3　The Limiting Probabilities　252
5.4　Absorbing Continuous-Time Markov Chains　256
5.S　Phase-Type Distributions　264
5.6　Uniformization　273
5.7　Continuous-Time Markov Reward Processes　277
5.8　Reversible Continuous-Time Markov Chains　284
Problems　298
Bibliographic Notes　313
References　314
Appendix　316
arkov Renewal and Semi-Regenerative Processes　321
6.0　Overview　322
6.1　Introduction　322
6.2　Markov Renewal Functions and Equations　331
6.3　Semi-Markov Processes and Related Reward Processes　339
6.4　Semi-Regenerative Processes　348
Problems　363
Bibliographic Notes　367
References　367
Appendix　368
rownian Motion and Other Diffusion Processes　373
7.0　Overview　373
7.1　Introduction　374
7.2　Diffusion Processes　385
7.3　Ito's Calculus and Stochastic Differential Equations　396
7.4　Multidimensional Ito's Lemma　404
1.5　Control of Systems of Stochastic Differential Equations　409
Problems　417
Bibliographic Notes 419
References　420
Appendix　421
Appendix: Getting Started with MATLAB　427
Index　436