Table of Contents
- Introduction
- What Is a Stochastic Process?
- Index Sets and State Spaces
- Types of Stochastic Processes
- Stationarity and Ergodicity
- Markov Processes
- Discrete-Time Markov Chains
- Continuous-Time Markov Processes
- Poisson Processes
- Birth-Death Processes
- Brownian Motion and Wiener Process
- Stochastic Differential Equations (SDEs)
- Fokker–Planck and Langevin Equations
- Martingales and Filtration
- Applications in Physics, Finance, and Biology
- Conclusion
1. Introduction
A stochastic process is a mathematical model describing systems that evolve over time with an inherent randomness. Unlike deterministic systems, the future state of a stochastic process cannot be predicted exactly, only in terms of probability distributions.
Stochastic processes appear across physics, biology, finance, and engineering — from quantum measurements to stock prices and population dynamics.
2. What Is a Stochastic Process?
Formally, a stochastic process is a collection of random variables indexed by time (or space):
\[
\{X(t) : t \in T\}
\]
Where:
- \( T \) is the index set (e.g., time)
- \( X(t) \) is a random variable describing the system state at time \( t \)
3. Index Sets and State Spaces
- Index set: can be discrete (e.g., \( t = 0, 1, 2, \dots \)) or continuous (e.g., \( t \in [0, \infty) \))
- State space: the set of all possible values of \( X(t) \), which can be finite, countable, or continuous
4. Types of Stochastic Processes
- Discrete-time vs. Continuous-time
- Discrete-state vs. Continuous-state
- Markovian vs. non-Markovian
- Stationary vs. non-stationary
Classification helps choose appropriate models and solution techniques.
5. Stationarity and Ergodicity
- Stationary process: statistical properties (mean, variance) are invariant over time
- Ergodic process: time averages equal ensemble averages
Stationarity simplifies analysis, especially in signal processing and statistical mechanics.
6. Markov Processes
A process has the Markov property if the future depends only on the present, not on the past:
\[
P(X_{t+1} | X_t, X_{t-1}, \dots) = P(X_{t+1} | X_t)
\]
This memoryless property enables elegant mathematical treatment.
7. Discrete-Time Markov Chains
Described by a transition probability matrix \( P \):
\[
P_{ij} = P(X_{n+1} = j \mid X_n = i)
\]
Analysis focuses on:
- Transition probabilities
- Stationary distributions
- Absorbing states
- Recurrence and transience
8. Continuous-Time Markov Processes
Described by infinitesimal generators or rate matrices \( Q \):
\[
\frac{d}{dt} P(t) = QP(t)
\]
Applications include:
- Chemical reactions
- Queueing systems
- Epidemic models
9. Poisson Processes
A fundamental counting process:
- \( N(t) \): number of events by time \( t \)
- Inter-arrival times are exponential with rate \( \lambda \)
- Independent, memoryless increments
Used to model:
- Arrivals in queues
- Radioactive decay
- Network traffic
10. Birth-Death Processes
A class of continuous-time Markov processes where transitions occur between neighboring states:
\[
P_{n,n+1} = \lambda_n, \quad P_{n,n-1} = \mu_n
\]
Models:
- Population growth
- Queue lengths
- Chemical kinetics
11. Brownian Motion and Wiener Process
A continuous-time, continuous-state stochastic process:
- Starts at 0
- Has independent, normally distributed increments
- Continuous paths, nowhere differentiable
Mathematical model of diffusion:
\[
B(t) \sim \mathcal{N}(0, t)
\]
Foundation for stochastic calculus.
12. Stochastic Differential Equations (SDEs)
SDEs describe dynamics of systems influenced by noise:
\[
dX_t = \mu(X_t, t) dt + \sigma(X_t, t) dB_t
\]
Where \( dB_t \) is Brownian motion (Wiener process) noise.
Applications:
- Finance (Black–Scholes model)
- Physics (Brownian motion, Langevin dynamics)
13. Fokker–Planck and Langevin Equations
- Langevin equation: stochastic differential equation for velocity/momentum
- Fokker–Planck equation: governs evolution of probability density:
\[
\frac{\partial P(x,t)}{\partial t} = -\frac{\partial}{\partial x}[A(x)P] + \frac{1}{2} \frac{\partial^2}{\partial x^2}[B(x)P]
\]
Describes time evolution of stochastic systems in terms of densities.
14. Martingales and Filtration
A martingale is a process where the conditional expected future equals the present:
\[
\mathbb{E}[X_{t+1} \mid \mathcal{F}_t] = X_t
\]
Important in finance and gambling theory. Filtration represents growing information over time.
15. Applications in Physics, Finance, and Biology
- Physics: diffusion, statistical mechanics, quantum noise
- Finance: option pricing, risk modeling, interest rate models
- Biology: gene expression noise, population dynamics, neural activity
- Engineering: signal processing, queueing theory
16. Conclusion
Stochastic processes provide a rich mathematical framework to study dynamic systems influenced by chance. From Brownian motion to stock prices and genetic drift, they underpin modern scientific modeling across disciplines.
A deep understanding of these processes is essential for research in applied mathematics, theoretical physics, finance, and systems biology.
