IT461 Assignment 3 Markov Chains

• Simulate the Land of Oz example given in class for 300 days.
• Simulate the stepping stone model of Example 11.2 of Laurie Snell book with n = 10 for two colors. Observe what happen’s at after a long time. Can you explain why this is happening?
• Simulate the drunkard’s walk with lattice points {`0,1,2,3,4`} with 0 and 4 being the absorbing states. Start the random walk in state 1. Estimate i)The expected number of times that the drunkard is in state 3 ii)The expected time to absorbtion iii)Probability that the chain will be absorbed in state 4 . Compare with the theoretical formula’s in section 11.2
• Simulate the rat’s maze example (Example 11.22) and keep track of the number of times that the rat is in state j for j = 1,2,.., Compare the empirical stationary distribution of this Markov chain to the theoretical one.
• A discrete time queueing system of capacity n = 50 consists of the person being served and those waiting to be served. The queue length x is observed each second. If 0 < x < n, then with probability p = 0.4, the queue size is increased by one by an arrival and, inependently, with probability r = 0.5, it is decreased by one because the person being served finishes service. If x = 0, only an arrival (with probability p) is possible. If x = 50, an arrival will depart without waiting for service, and so only the departure (with probability r)of the person being served is possible. This is a Markov chain with states given by the number of customers in the queue. Write a computer program to simulate the queue. Have your program keep track of the proportion of the time that the queue length is j for j = 0,1,...,n and the average queue length. Can you estimate the stationary vector ? Test what happens when p = 0.5 and r = 0.4?

Subjects List