Looking for GATE Preparation Material? Join & Get here now!

** Gate 2013 Question Papers.. ** CEED 2013 Results.. ** Gate 2013 Question Papers With Solutions.. ** GATE 2013 CUT-OFFs.. ** GATE 2013 Results.. **

Markov Chains

Markov Chains

In a previous page, we studied the movement between the city and suburbs. Indeed, if I are S are the initial population of the inner city and the suburban area, and if we assume that every year 40% of the inner city population moves to the suburbs, while 30% of the suburb population moves to the inner part of the city, then after one year the populations are given by

$\begin{displaymath}\left(\begin{array}{c} 0.6 I + 0.3 S\\ 0.4 I + 0.7 S\\ \end... ...ay}\right) \left(\begin{array}{c} I\\ S\\ \end{array}\right).\end{displaymath}$

The matrix

$\begin{displaymath}P = \left(\begin{array}{cc} 0.6&0.3\\ 0.4&0.7\\ \end{array}\right) \end{displaymath}$

is very special. Indeed, the entries of each column vectors are positive and their sum is 1. Such vectors are called probability vectors. A matrix for which all the column vectors are probability vectors is called transition or stochastic matrix. Andrei Markov, a russian mathematician, was the first one to study these matrices. At the beginning of this century he developed the fundamentals of the Markov Chain theory.
A Markov chain is a process that consists of a finite number of states and some known probabilities p_ij, where p_ij is the probability of moving from state j to state i. In the example above, we have two states: living in the city and living in the suburbs. The number p_ij represents the probability of moving from state i to state j in one year. We may have more than two states. For example, political affiliation: Democrat, Republican, and Independent. For example, p_ij represents the probability of a son belonging to party i if his father belonged to party j.
Of particular interest is a probability vector p such that $A \mbox{\bf p} = \mbox{\bf p}$ , that is, an eigenvector of A associated to the eigenvalue 1. Such vector is called a steady state vector. In the example above, the steady state vectors are given by the system

$\begin{displaymath}\left(\begin{array}{cc} 0.6&0.3\\ 0.4&0.7\\ \end{array}\rig... ...} -0.4&0.3\\ 0.4&-0.3\\ \end{array}\right)\cdot X = {\cal O}.\end{displaymath}$

This system reduces to the equation -0.4 x + 0.3 y = 0. It is easy to see that, if we set $x = 0.3 \alpha$ , then

$\begin{displaymath}X = \left(\begin{array}{c} x\\ y\\ \end{array}\right) = \alpha \left(\begin{array}{c} 0.3\\ 0.4\\ \end{array}\right).\end{displaymath}$

So the vector $\mbox{\bf p}_1 = \displaystyle \left(\begin{array}{c} 0.3\\ 0.4\\ \end{array}\right)$ is a steady state vector of the matrix above. So if the populations of the city and the suburbs are given by the vector $\mbox{\bf p}_1$ , after one year the proportions remain the same (though the people may move between the city and the suburbs).

Let us discuss another example on population dynamics.

Example: Age Distribution of Trees in a Forest
Trees in a forest are assumed in this simple model to fall into four age groups: b(k) denotes the number of baby trees in the forest (age group 0-15 years) at a given time period k; similarly y(k),m(k) and o(k) denote the number of young trees (16-30 years of age), middle-aged trees (age 31-45), and old trees (older than 45 years of age), respectively. The length of one time period is 15 years.
How does the age distribution change from one time period to the next? The model makes the following three assumptions:

A certain percentage of trees in each age group dies.
Surviving trees enter into the next age group; old trees remain old.
Lost trees are replaced by baby trees.

Note that the total tree population does not change over time.

We obtain the following difference equations:

b(k+1)	=	$\displaystyle d_b\cdot b(k)+d_y \cdot y(k) +d_m\cdot m(k) + d_o\cdot o(k)$	(1)
y(k+1)	=	(1-d_b) b(k)	(2)
m(k+1)	=	(1-d_y) y(k)	(3)
o(k+1)	=	(1-d_m) m(k) + (1-d_o) o(k)	(4)

Here 0 < d_b,d_y,d_m,d_o <1 denote the loss rates in each age group in percent.

Let

$\begin{displaymath}{\bf x}(k)=\left(\begin{array}{c}b(k)\\ y(k)\\ m(k)\\ o(k)\end{array}\right)\end{displaymath}$

be the ``age distribution vector". Consider the matrix

$\begin{displaymath}A = \left(\begin{array}{cccc} d_b&d_y&d_m&d_o\\ 1-d_b&0&0&0\\ 0&1-d_y&0&0\\ 0&0&1-d_m&1-d_o\\ \end{array}\right).\end{displaymath}$

Then we have

$\begin{displaymath}{\bf x}(k+1)=A\cdot {\bf x}(k).\end{displaymath}$

Note that the matrix A is a stochastic matrix!
If, d_b=.1,d_y=.2,d_m=.3 and d_o=.4, then

$\begin{displaymath}A = \left(\begin{array}{cccc} 0.1&0.2&0.3&0.4\\ 0.9&0&0&0\\ 0&0.8&0&0\\ 0&0&0.7&0.6\\ \end{array}\right).\end{displaymath}$

After easy calculations, we find the steady state vector for the age distribution in the forest:

$\begin{displaymath}\mbox{\bf p} = \left(\begin{array}{cccc} 1/3.88\\ 0.9/3.88\\... ...n{array}{cccc} 1\\ 0.9\\ 0.72\\ 1.26\\ \end{array}\right) .\end{displaymath}$

Assume a total tree population of 50,000 trees. Suppose the forest is newly planted, i.e.

$\begin{displaymath}{\bf x}(0)=\left(\begin{array}{c}50,000\\ 0\\ 0\\ 0\end{array}\right)\end{displaymath}$

After 15 years, the age distribution in the forest is given by

$\begin{displaymath}{\bf x}(1) = \left(\begin{array}{cccc} 0.1&0.2&0.3&0.4\\ 0.9... ...\begin{array}{cccc} 0.1\\ 0.9\\ 0\\ 0\\ \end{array}\right).\end{displaymath}$

After 30 years, we have

$\begin{displaymath}{\bf x}(2) = \left(\begin{array}{cccc} 0.1&0.2&0.3&0.4\\ 0.9... ...in{array}{cccc} 0.19\\ 0.09\\ 0.72\\ 0\\ \end{array}\right)\end{displaymath}$

and after 45 years

$\begin{displaymath}{\bf x}(3) = 50,000\left(\begin{array}{cccc} 0.19\\ 0.09\\ 0.72\\ 0\\ \end{array}\right)\end{displaymath}$

After 15n years, where $n=1,2,\cdots$ , the age distribution in the forest is given by

$\begin{displaymath}{\bf x}(n) = \left(\begin{array}{cccc} 0.1&0.2&0.3&0.4\\ 0.9... ...eft(\begin{array}{cccc} 1\\ 0\\ 0\\ 0\\ \end{array}\right).\end{displaymath}$

So the problem is to find the n^th-power of the matrix A. We have seen that diagonalization technique may be helpful to solve this problem. Another problem deals with the long term behavior of the sequence x(n) when n gets large.

The calculations on the example above becomes tedious. Let us illustrate the problem on a small matrix.

Example. Consider the stochastic matrix

$\begin{displaymath}A = \left(\begin{array}{cccc} 0.8&0.2\\ 0.2&0.8\\ \end{array}\right).\end{displaymath}$

Note this is a symmetric matrix. The characteristic polynomial of A is

$\begin{displaymath}p(\lambda) = (0.8 - \lambda)^2-0.2^2 = (1-\lambda)(0.6 - \lambda)\end{displaymath}$

An eigenvector associated to 1 is

$\begin{displaymath}\left(\begin{array}{cccc} 1\\ 1\\ \end{array}\right)\end{displaymath}$

and an eigenvector associated to 0.6 is

$\begin{displaymath}\left(\begin{array}{cccc} 1\\ -1\\ \end{array}\right).\end{displaymath}$

If we set

$\begin{displaymath}P = \left(\begin{array}{rr} 1&1\\ 1&-1\\ \end{array}\right),\end{displaymath}$

then we have

$\begin{displaymath}P^{-1}AP = D = \left(\begin{array}{cc} 1&0\\ 0&0.6\\ \end{array}\right).\end{displaymath}$

So, we have

$\begin{displaymath}A^n = P D^nP^{-1} = P\left(\begin{array}{cc} 1&0\\ 0&(0.6)^n\\ \end{array}\right)P^{-1}.\end{displaymath}$

When n gets large, the matrices Aⁿ get closer to the matrix

$\begin{displaymath}P\left(\begin{array}{cc} 1&0\\ 0&0\\ \end{array}\right)P^{-1}.\end{displaymath}$

So the sequence of vectors defined by

$\begin{displaymath}X(n+1) = A X(n),\;\;\mbox{given $X(0)$}\end{displaymath}$

will get closer to

$\begin{displaymath}X(\infty) = P\left(\begin{array}{cc} 1&0\\ 0&0\\ \end{array}\right)P^{-1}X(0)\end{displaymath}$

when n gets large. If $X(0) = \displaystyle \left(\begin{array}{cc} a\\ b\\ \end{array}\right)$ , then we have

$\begin{displaymath}X(\infty) = \left(\begin{array}{rr} 1&1\\ 1&-1\\ \end{array... ...ac{a+b}{2}\left(\begin{array}{cc} 1\\ 1\\ \end{array}\right).\end{displaymath}$

Note that the vector $X(\infty)$ is proportional to the unique steady state vector of A

$\begin{displaymath}{\bf p} = \frac{1}{2}\left(\begin{array}{cc} 1\\ 1\\ \end{array}\right).\end{displaymath}$

This is not surprising. In fact there is a general result similar to the one above for any stochastic matrix.

Discussion Center

Discuss/
Query
Papers/
Syllabus
Feedback/
Suggestion
Yahoo
Groups
Sirfdosti
Groups
Contact
Us

MEMBERS LOGIN

INTERVIEW EBOOK

Get 9,000+ Interview Questions & Answers in an eBook. Interview Question & Answer Guide

9,000+ Interview Questions
All Questions Answered
5 FREE Bonuses
Free Upgrades

GATE RESOURCES

Gate Books

Training Institutes

Gate FAQs

GATE BOOKS

Mechanical Engineeering Books

Robotics Automations Engineering Books

Civil Engineering Books

Chemical Engineering Books

Environmental Engineering Books

Electrical Engineering Books

Electronics Engineering Books

Information Technology Books

Software Engineering Books

GATE Preparation Books

Exciting Offers

GATE Exam, Gate 2009, Gate Papers, Gate Preparation & Related Pages GATE Overview \| GATE Eligibility \| Structure Of GATE \| GATE Training Institutes \| Colleges Providing M.Tech/M.E. \| GATE Score \| GATE Results \| PG with Scholarships \| Article On GATE \| GATE Forum \| GATE 2009 Exclusive \| GATE 2009 Syllabus \| GATE Organizing Institute \| Important Dates for GATE Exam \| How to Apply for GATE \| Discipline / Branch Codes \| GATE Syllabus for Aerospace Engineering \| GATE Syllabus for Agricultural Engineering \| GATE Syllabus for Architecture and Planning \| GATE Syllabus for Chemical Engineering \| GATE Syllabus for Chemistry \| GATE Syllabus for Civil Engineering \| GATE Syllabus for Computer Science / IT \| GATE Syllabus for Electronics and Communication Engineering \| GATE Syllabus for Engineering Sciences \| GATE Syllabus for Geology and Geophysics \| GATE Syllabus for Instrumentation Engineering \| GATE Syllabus for Life Sciences \| GATE Syllabus for Mathematics \| GATE Syllabus for Mechanical Engineering \| GATE Syllabus for Metallurgical Engineering \| GATE Syllabus for Mining Engineering \| GATE Syllabus for Physics \| GATE Syllabus for Production and Industrial Engineering \| GATE Syllabus for Pharmaceutical Sciences \| GATE Syllabus for Textile Engineering and Fibre Science \| GATE Preparation \| GATE Pattern \| GATE Tips & Tricks \| GATE Compare Evaluation \| GATE Sample Papers \| GATE Downloads \| Experts View on GATE \| CEED 2009 \| CEED 2009 Exam \| Eligibility for CEED Exam \| Application forms of CEED Exam \| Important Dates of CEED Exam \| Contact Address for CEED Exam \| CEED Examination Centres \| CEED Sample Papers \| Discuss GATE \| GATE Forum of OneStopGATE.com \| GATE Exam Cities \| Contact Details for GATE \| Bank Details for GATE \| GATE Miscellaneous Info \| GATE FAQs \| Advertisement on GATE \| Contact Us on OneStopGATE \|
Copyright © 2024. One Stop Gate.com. All rights reserved	Testimonials \|Link To Us \|Sitemap \|Privacy Policy \| Terms and Conditions\|About Us
Our Portals : Academic Tutorials \| Best eBooksworld \| Beyond Stats \| City Details \| Interview Questions \| India Job Forum \| Excellent Mobiles \| Free Bangalore \| Give Me The Code \| Gog Logo \| Free Classifieds \| Jobs Assist \| Interview Questions \| One Stop FAQs \| One Stop GATE \| One Stop GRE \| One Stop IAS \| One Stop MBA \| One Stop SAP \| One Stop Testing \| Web Hosting \| Quick Site Kit \| Sirf Dosti \| Source Codes World \| Tasty Food \| Tech Archive \| Software Testing Interview Questions \| Free Online Exams \| The Galz \| Top Masala \| Vyom \| Vyom eBooks \| Vyom International \| Vyom Links \| Vyoms \| Vyom World C Interview Questions \| C++ Interview Questions \| Send Free SMS \| Placement Papers \| SMS Jokes \| Cool Forwards \| Romantic Shayari