5. Poisson Probability Distribution

Recall (from module 4) the Binomial Probability Distribution W[(Subscript[n, R])] of a drunk taking (Subscript[n, R]) steps to the right (and (Subscript[n, L]) steps to the left)

\(\(\(\ \ \)\(W[
        n\_R] = \ \(N!\)\/\(\(n\_R!\)\ \(n\_L!\)\)\ \ p\^n\_R\ \ q\^n\_L\)\)\)

N is the total number of steps in the drunk takes and p is the probability the drunk takes a step to the right (and q is the probability the drunk takes as step to the left).  We know that the total number of steps N=(Subscript[n, R])+(Subscript[n, L]) and also that p+q=1 because there are only two possibilities, either the drunk takes a step to the right or to the left.
	We are not so much interested in the (random) walk of drunks as we are in laboratory situations where the probability of the random errors must be included in the report of a measurement.  Many laboratory situations involve cases where one of the probabilities, say p is quite small (p<<1) and as a consequence q≈1 (since q=1-p).  This limit of the Binomial probability distribution (for N large) is called the Poisson probability distribution and this is the subject of this module.
	We simplify the notation in equation (1) slightly by using n instead of (Subscript[n, R]) so that (Subscript[n, L])=N-(Subscript[n, R])=N-n  instead.  The Binomial distribution thus appears in the simpler form

\(\(\(\ \ \ \)\(W[
        n] = \ \(N!\)\/\(\(n!\) \(\((\ N - n)\)!\)\)\ \ p\^n\ \ q\^\(N - n\)\)\)\)

Observation about the case p<<1

A typical case of the Binomial distribution for p<<1  is

Num = 100;
p = 0.1;
q = 1 - p; 
W[n_] :=  ((Num!)/(n! (Num - n)!))  (p^n)   (q^(Num - n))

data = Table[{n, W[n]}, {n, 0, Num}];
ListPlot[data, PlotRange -> {0, 0.2}, 
      PlotStyle -> PointSize[0.01], 
      AxesLabel -> {"n", "W[n]"}]

The range of the probability distribution is 0≤n≤N but notice that for most values of n , W[n] is zero.  The only place where W[n]is not very small is when n<< N (say  0≤n≤20  in the above example). Another way to say this is that n is small compared with N.   We will use this fact to obtain the Poisson approximation to the Binomial distribution.

Some Approximations for p small and N large

We must make two approximations of equation (2) to get the Poisson limit distribution.  The first approximation involves

((N!)/( (N - n)!)) =  
      N (N - 1) (N - 
            2) (N - 3)··· (N - n + 
              1)

The second equality in equation (3) involves just writing out the terms of the two factorial functions in N! and (N - n)! and then performing cancellations.   Recall that when p<<1, the only place were  W[n] is NOT small is when (Subscript[n, R]) is small compared with N.  This means each term on the RHS of equation (3) is approximately the same size as N in the case where  W[n] is not small.  We may approximate equation (3) by

((N!)/( (N - n)!)) =  N · 
          N· N · N· N · 
          N · N · N = 
        (N^( n))

where on the RHS there are n numbers N multiplied by each other and this is otherwise known as (N^( n)).
	The second approximation we make to equation (2) involves (q^(N - n)) where q≈1.  Taking the natural Logarithm yields

Log[
        (q^(N - n))] = (N - n) Log q  =  (N - n) 
          Log [1 - p]

Since p<<1 is small we may approximate the Log function by a Taylor series expansion of the form

\(\(\(\ \)\(Log\ [
        1 - p] = \ \(-p\)\  + \ \(\(·\)\(·\)\(·\)\)\)\)\)

The approximation (6) is discussed in the appendix but for now notice say for p=0.1

Log[1 - 0.1]

so -p is a good approximation (within about 5%).  Using approximation (6) in equation (5) yields

Log[
        (q^(N - n))] ≈  -p (N - 
            n)

and since n<< N when W[n] is NOT zero

Log[
        (q^(N - n))] ≈  -p N

Finally using a property of the Log function we may write

\(\(\(\ \)\(q\^\(N - n\) ≈ \ e\^\(\(-\ p\)\ N\)\)\)\)

This is the second approximation we need to get the Poisson limit of the Binomial distribution (3).

The Poisson Limit

Using approximations (4) and (9) in the Binomial distribution (2) yields

\(\(\(\ \ \ \ \)\(W[n] ≈ \ 
      N\^\(\(\ \)\(n\)\)\/\(n!\)\ \ p\^n\ \ e\^\(-\ Np\)\)\)\)

and after a slight rearrangement

\(\(\(\ \)\(W[
        n] ≈ \ \((Np)\)\^n\/\(n!\)\ \ e\^\(-\ Np\)\)\)\)

Defining μ via

μ ≡ Np

the Poisson distribution equation (11) appears in standard form

\(\(\(\ \ \)\(W[
        n] ≈ \ \(\(μ\)\(\ \)\)\^n\/\(n!\)\ \ e\^\(-\ μ\)\)\)\)

The Normalization in the Poisson Distribution

The normalization condition

(∑_(n = 0)^N) W[n] = 1

holds because the sum over all the cases n must yield certainty or probability 1.  Using equation (13) in equation (14) yields

\(\(\(\ \)\(∑\+\(n = 0\)\%N W[
          n] = \ \(∑\+\(n = 0\)\%N\( \(μ\)\(\ \)\)\^n\/\(n!\)\ \ e\^\(-\ μ\) = \ e\^\(-\ μ\)\ \ \(∑\+\(n = 0\)\%N\( \(μ\)\(\ \)\)\^n\/\(n!\)\)\)\)\)\)

The appendix will show that the Taylor series of (e^μ) is

\(\(\(\ \ \ \)\(e\^μ\  = \ ∑\+\(n = 0\)\%N\( \(μ\)\(\ \)\)\^n\/\(n!\)\)\)\)

so combining this in equation (15) we obtain equation (1) because (e^(- μ))(e^( μ))=1.

The Average in the Poisson Distribution

The average number of steps  \(TraditionalForm\`n\&_\)  is defined

\(n\&_\ \  = \ ∑\+\(n = 0\)\%N\ n\ W[n]\)

where the n=0 term in equation (17) is absent since n=0 in the product  n W[n].  Using the Poisson distribution (13) we obtain

\(\(\(\ \)\(n\&_\ \  = \ \(∑\+\(n = 1\)\%N\ 
          n\ \(\(μ\)\(\ \)\)\^n\/\(n!\)\ \ e\^\(-\ μ\)\  = \ e\^\(-\ μ\)\ \(∑\+\(n = 1\)\%N\ \(\(n\)\(\ \)\(\(\(μ\)\(\ \)\)\^n\/\(n!\)\)\(\ \)\)\)\)\)\)\)

where now the sum starts at n=1 (since the n=0 term is absent).  The summation in equation (18) can be rearranged to yield

\(\(\(\ \)\(∑\+\(n = 1\)\%N\ 
        n\ \(\(μ\)\(\ \)\)\^n\/\(n!\) = \ \(∑\+\(n = 1\)\%N\ \(\(μ\)\(\ \)\)\^n\/\(\((n - 1)\)!\) = \ μ\ \(∑\+\(n = 1\)\%N\ \(\(μ\)\(\ \)\)\^\(n - 1\)\/\(\((n - 1)\)!\)\)\)\)\)\)

It helps to change the summation index on the RHS of equation (19) using m≡n-1 so that

\(\(\(\ \ \)\(∑\+\(n = 1\)\%N\ \(\(μ\)\(\ \)\)\^\(n - 1\)\/\(\((n - 1)\)!\) = \ ∑\+\(m = 0\)\%\(N - 1\)\ \(\(μ\)\(\ \)\)\^m\/\(m!\)\)\)\)

Again recall that in the appendix we show (for N->∞)

\(\(\(\ \ \)\(∑\+\(m = 0\)\%∞\ \(\(μ\)\(\ \)\)\^m\/\(m!\) = e\^μ\)\)\)

Combining equation (21) in (18) and  (19) yields

\(\(\(\ \ \)\(n\&_\ \  = \ \(e\^\(-\ μ\)\ \ μ\ \ e\^\(\(\ \)\(μ\)\)\  = \ μ\)\)\)\)

So the parameter μ≡Np in the Poisson distribution is just the average number of step \(TraditionalForm\`n\&_\).  The workshop allows you to discover this relationship as would an experimentalist.

The Standard Deviation

The standard deviation (or variance) (σ^2) is defined as the average of the square of the deviation from the average \(TraditionalForm\`n\&_\) namely

\(\(\(\ \)\(σ\^2\ \  = \ ∑\+\(n = 0\)\%N\ \((n - n\&_\ )\)\^2\ W[n]\)\)\)

It is straightforward to show that

(σ^2)   =  N p

for the Poisson distribution. Beginning with equation (19) we multiply the binomial on the RHS to obtain

\(\(\(\ \)\(σ\^2\ \  = \ ∑\+\(n = 0\)\%N\ \(\((\ 
            n\^2 - \ 2\ n\ n\&_\  + \ n\&_\^2\ )\)\(\ \)\(W[
            n]\)\(\ \ \)\)\)\)\)

and using the distributive principle

\(\(\(\ \ \)\(σ\^2\ \  = \ ∑\+\(n = 0\)\%N\ \((\ \(n\^2\) 
              W[n] - \ 2\ n\ n\&_\ W[n] + \ 
            n\&_\^2\ W[n])\)\)\)\)

\(TraditionalForm\`n\&_\)  does not involve the sum over n (it already has been summed) so equation (26) appears

\(\(\(\ \)\(σ\^2\ \  = \ ∑\+\(n = 0\)\%N\ \(n\^2\) W[n] - \ 
        2\ n\&_\ \(∑\+\(n = 0\)\%N 
              n\ \ W[n]\) + \ \(n\&_\^2\) \(∑\+\(n = 0\)\%N W[
              n]\)\)\)\)

The second summation in (27) is just the average \(TraditionalForm\`n\&_\) and the third summation is just the normalization so equation (27) reduces to

\(\(\(\ \ \)\(σ\^2\ \  = \ \(∑\+\(n = 0\)\%N\ \(n\^2\) W[n] - 
          2\ n\&_\^2\  + \ 
          n\&_\^2 = \ ∑\+\(n = 0\)\%N\ \(n\^2\) W[n] - \ 
          n\&_\^2\)\)\)\)

So all we really need to compute is the first sum in order to obtain (σ^2).  Explicitly the sum appears

\(\(\(\ \ \)\(\(n\^2\)\&_\  = \ \(∑\+\(n = 0\)\%N\ \(n\^2\) 
            W[n] = \ \(∑\+\(n = 0\)\%N\ \(n\^2\) \(\(μ\)\(\ \)\)\^n\/\(n!\)\ \ e\^\(-\ μ\) = \ e\^\(-\ μ\)\ \(\(\(μ\^2\) ∑\)\+\(n = 0\)\%N\ \(\(n\^2\) \(\(μ\)\(\ \)\)\^\(n - 2\)\/\(n!\)\)\)\)\)\)\)\)

Recall the infinite series, identity equation (21) and differentiate w.r.t. μ   once to obtain

\(\(\(\ \ \)\(∑\+\(n = 0\)\%∞\ \(n\ \(\(μ\)\(\ \)\)\^\(n - 1\)\)\/\(n!\) = e\^μ\)\)\)

and once again

\(\(\(\ \)\(∑\+\(n = 0\)\%∞\ \(n \((n - 1)\)\ \(\(μ\)\(\ \)\)\^\(n - 2\)\)\/\(n!\) = e\^μ\)\)\)

Rearranging the LHS of equation (21) yields

\(\(\(\ \)\(∑\+\(n = 0\)\%∞\ \(\((\ n\^2 - n\ )\)\ \(\(μ\)\(\ \)\)\^\(n - 2\)\)\/\(n!\) = e\^μ\)\)\)

and further

\(\(\(\ \)\(∑\+\(n = 0\)\%∞\ \(\(\ \)\(n\^2\ \(\(μ\)\(\ \)\)\^\(n - 2\)\)\)\/\(n!\) - \(\(1\/μ\^2\) ∑\)\+\(n = 0\)\%∞\ \(n\ \(\(μ\)\(\ \)\)\^n\)\/\(n!\) = e\^μ\)\)\)

and using equation (18)

\(\(\(\ \)\(∑\+\(n = 0\)\%∞\ \(\(\ \)\(n\^2\ \(\(μ\)\(\ \)\)\^\(n - 2\)\)\)\/\(n!\) = \(\(1\/μ\^2\) 
            n\&_\ e\^μ + e\^μ = 
        1\/μ\ e\^μ + e\^μ\)\)\)\)

since μ=\(TraditionalForm\`n\&_\).  Using (34) in equation (29) yields

\(\(\(\ \ \ \)\(\(n\^2\)\&_\  = \(e\^\(-\ μ\)\ \(\(μ\^2\) ∑\)\+\(n = 0\)\%N\ \(n\^2\) \(\(μ\)\(\ \)\)\^\(n - 2\)\/\(n!\) = \(e\^\(-\ μ\)\ μ\^2\ \ \((1\/μ\ e\^μ + e\^μ)\) = μ\^2 + μ\)\)\)\)\)

and finally equation (35) in equation (28) yields

\(\(\(\ \)\(σ\^2\ \  = \ \(μ\^2 + μ\  - \ 
          n\&_\^2 = \ μ\)\)\)\)

since again \(TraditionalForm\`n\&_\) = μ.  Equations is a very simple result.  σ = (Sqrt[μ])  characterizes the width of the Poisson distribution. This will be explored in a workshop.

The Relative Width of the Poisson Distribution

The width of the Poisson distribution compared with the average is

\(σ\/n\&_ = \ \(\@μ\/μ = \ \(1\/\@μ = \ 
          1\/\@Np\)\)\)

and this is the relative width of the Poisson distribution.  Since p is fixed and small and so as N->∞ the relative width goes to zero (or rather becomes small).

Appendix-Taylor Series

The basic assumption of Taylor series is that a function f[x] can be approximated by a power series of the form

f[x] =  
      (Subscript[a, 0]) +  (Subscript[a, 1]) x  +  (Subscript[a, 2]) (x^2) +  
          (Subscript[a, 3]) (x^3) +   ...

and it is easy compute the coefficients (Subscript[a, 0]), (Subscript[a, 1]), (Subscript[a, 2]), etc.   To obtain (Subscript[a, 0]), just set x=0 on both sides of equation (1) and obtain

f[0] =  (Subscript[a, 0])

Obtain (Subscript[a, 1]) is not much harder.  Just differentiate equation (1) once obtaining

(d/dx) 
        f[x] =    (Subscript[a, 1])  +  2 (Subscript[a, 2]) x + 
          3  (Subscript[a, 3]) (x^2) +   ...

and then set x=0 yielding

(f^((1)))[0] =  (Subscript[a, 1])

where we used the definition

\(\(\(\ \ \)\(\(f\^\((1)\)\)[
        0] = \(\(\(d\/dx\) 
          f[x]\)\( | \_\(x = 0\)\)\)\)\)\)

It should be clear how to proceed further.  For example, differentiation of equation (40) and setting x=0 yields

(f^((2)))[
        0] = 2 (Subscript[a, 2])     or    (Subscript[a, 2]) =  (1/2) (f^((2)))[
            0]

and in general

\(\(\(\ \)\(\(f\^\((n)\)\)[
        0] = \(\(n!\)\ a\_n\ \ \ \ \ or\ \ \ \ a\_n = \ \(1\/\(n!\)\) \(f\^\((n)\)\)[0]\)\)\)\)

Utilization of the (Subscript[a, n])in the Taylor series equation (38) yields

f[x] =  
      f[0] +  (x/(n!)) (f^((1)))[
              0]   +  ((x^2)/(n!)) (f^((2)))[
              0] +  ((x^3)/(n!)) (f^((3)))[
              0] +   ...

Examples of Taylor Series

Suppose f[x]=Log[1-x] then f[0]=Log[1]=0 and

(f^((1)))[
        x] = (d/dx) 
          Log[1 - x] =  -(1/(1 - 
                  x]))

so setting x=0 we obtain simply

(f^((1)))[0] = -1

Utilization of these results in the first two terms of the Taylor series (45) yields

Log[1 - x] ≈  
        0  -  1 x  +   ...

which is just approximation equation (6).

Another example of the Taylor series is exponential function f[x]=(e^x) .  This is quite easy since

\(\(\(\ \)\(\(f\^\((n)\)\)[
        x] = \(\(d\^n\/dx\^n\) e\^x = \ e\^x\)\)\)\)

and so

(f^((n)))[0] = (e^0) = 
        1.

The Taylor series (45) for the exponential function with equation (50) reduces to

(e^x) =  
      1 +  (x/(n!))  +  ((x^2)/(n!)) +  
          ((x^3)/(n!)) +   ...

which is the same as equation (21) above.

© Rodney L. Varley (2001).