**ST302 **

**Stochastic Processes**

Stochastic Processes代考 Identify the transient and recurrent states, and the irreducible closed sets for the Markov chains with following transiton matrices.

**Instructions to candidates **

This paper contains 5 questions. Answer **ALL **questions.

**Time allowed – Reading Time: ***None *

**Writing Time: ***3 hours *

**You are supplied with: ***Murdoch & Barnes Statistical Tables, 4th edition *

**You may also use: ***No additional materials *

**Calculators: ***Calculators are not allowed in this examination *

1. Identify the transient and recurrent states, and the irreducible closed sets for the Markov chains with following transiton matrices. [8 marks]

2.Consider a Markov chain (*X**n*)*n**≥*0 with the countable state space *{*0*, *1*, *2*, . . .**} *and the following transition probabilities:

*p*(*i, i *+ 1) = *p, i **≥ *0;

*p*(*i, i **− *1) = *q, i **≥ *1;

*p*(*i, i*) = 1 *− **p **− **q, i **≥ *1*, *

*p*(0*, *0) = 1 *− **p, *

where *p > *0 and *q > *0. Let *V**i *:= min*{**n **≥ *0 : *X**n *= *i**} *be the fifirst time that the chain visits *i*. Stochastic Processes代考

a) Explain why this Markov chain is irreducible. Is it also aperiodic?Show your reasoning. [5 marks]

b) Let *a, b *and *i *belong to the state space of *X *such that *a < i < b*.Without using the Optional Stopping Theorem show that

[8 marks]

c) Assume *p < q *and show that the limiting distribution *π *is given by

[8 marks]

### 3.Let *S *be a random walk adapted to (*F**n*)*n**≥*0 such that Stochastic Processes代考

*P*(*S**n*+1 = *S _{n}*

*+ 1*

*|F*

_{n}) =

*p,*

*P*(*S**n*+1 = *S _{n}*

*−*1

*|F*) =

_{n}*q,*and

*P*(*S**n*+1 = *S _{n}*

*|F*) = 1

_{n}*−*

*p*

*−*

*q,*

for some *p > *0 and *q > *0. Defifine

a) Show that *φ*(*S*) is a martingale with respect to (*F _{n}*)

*n*

*≥*0. [4 marks]

b) Defifine *M**n *= *S _{n}*

*−*

*n*(

*p*

*−*

*q*) for

*n*

*≥*Show that

*M*is martingale with respect to (

*F*)

_{n}*n*

*≥*0. [4 marks]

c) Assume *p≠**q *and let *T *= min*{**n **≥ *0 : *S**n *= *b *or *S**n *= 0*}*. Show that whenever 0 *≤ **i **≤ **b *we have

[8 marks]

4.Let *N *be a Poisson process with intensity *λ *and adapted to some fifiltration (*F _{t}*)

*t*

*≥*0.

a) Show that *N _{t}*

*−*

*λ*and (

_{t}*N*

_{t}*−*

*λ*) 2

_{t}*−*

*λ*are martingales with respect to (

_{t}*F*)

_{t}

_{t}*≥*0. [8 marks]

b) Consider the time of the *n*-th arrival *T**n *:= inf*{**t **≥ *0 : *N _{t}*

*=*

*n*

*}*and let

*m > n*. Show that

[8 marks]

c) Suppose there exists another adapted Poisson process *Z *with in-tensity *µ*, which is independent of *N*. Let *X _{t}*

*=*

*Z*

_{t}*+*

*N*

_{t}*and defifine*

*τ*:= inf

*{*

*t*

*≥*0 :

*X*

_{t}*= 1*

*}*. Show that

(Hint: Consider *P*(*T*1 *< S*1) where *T*1 and *S*1 are the fifirst arrivals for *N *and *Z *respectively and recall that the time of fifirst arrival for a Poisson process has exponential distribution.) [6 marks]

### 5.Let *B *denote a Brownian motion with *B*0 = 0. Stochastic Processes代考

a) State the defifinition of a Brownian motion. [4 marks]

b) Prove that expsin(*B _{t}*) and exp can be written as stochastic integrals with respect to

*B*. [6 marks]

c) Let *δ > *2 and consider *X*, which solves the following SDE:

Find a constant *α *for which * *can be written as a stochastic integral with respect to *B*. (Take that *X *never hits 0 for granted.)[5 marks]

d) Solve the following stochastic difffferential equation:

*dY _{t}*

*=*

*aY*

_{t}*d*+ (

_{t}*b*(

*) +*

_{t}*cY*)

_{t}*dB*

_{t}

*,*

where *Y*0 = 0. (Hint: Try a solution of the form *Z _{t}*

*H*

_{t}*where*

*Z*

_{t}*= expand*

*dH*

_{t}*=*

*F*(

*t*)

*d*+

_{t}*G*(

*t*)

*dB*

_{t}*for some adapted process*

*F*and

*G*which need to be determined.) [8 marks]

e) It is well known that for any deterministic function *f*(*t*) the ran-dom variable * *is normally distributed. Find its mean and variance. [2 marks]

f) Use Feynman-Kac representation result to fifind a function *F*(*t, x*) that solves

where *a, r *and *σ *are real constants. You may want to use the fact that for any *u **∈ *R where *Z *is a standard Normalrandom variable. [8 marks]