8.Experimental Correlations

Introduction

Sometimes you have reason to expect a relationship exists between two experimental quantities x and y.  For example, you might expect on the basis of theory that there is a linear relationship of the form y=mx+b.  You may proceed to collect N data points {(Subscript[x, 1]),  (Subscript[y, 1])}, {(Subscript[x, 2]),  (Subscript[y, 2])}, ... , {(Subscript[x, N]),  (Subscript[y, N])} and use the Method of Least Square Fit to determine the values of the slope m and intercept b that gives the best straight line though your data.
	An alternative experimental situation might exist where you might suspect a linear relationship exists between x and y but you are not quite sure.  So after you have collected your data, you wish to determine whether or not your experiment confirms your hypothesis of a linear relationship between x and y.  The correlation coefficient (Subscript[σ, x y]) is a quantity that can be computed from your N data points and the value of the correlation coefficient (Subscript[σ, x y]) gives the probability that your hypothesis (that there is a linear relationship between x and y) is correct or not.

Introduction to Correlations

Given the N data points we can calculate the averages \(TraditionalForm\`x\&_\) and \(TraditionalForm\`y\&_\) as usual

\(\(\(\ \ \ \ \ \ \ \ \ \ \ \ \)\(x\&_ = \ \(\(1\/N\) \(∑\+\(n = 1\)\%N x\_n\ \ \ \ \ and\ \ \ \ \ y\&_\) = \ \(1\/N\) \(∑\+\(n = 1\)\%N y\_n\)\)\)\)\)

and the standard deviations (Subscript[σ, x]) and (Subscript[σ, y]) are

\(\(\(\ \ \ \ \ \ \ \ \ \ \ \)\(σ\_x\^2 = \ \(\(1\/N\) \(∑\+\(n = 1\)\%N\((x\_n - \ x\&_)\)\^2\ \ \ \ \ and\ \ \ \ \ σ\_y\^2\) = \ \(1\/N\) \(∑\+\(n = 1\)\%N\((y\_n - \ y\&_)\)\^2\)\)\)\)\)

Suppose x and y determine a calculated theoretical quantity q=q(x,y).For example, suppose q=xy the sum of the two experimental variables.  The averages \(TraditionalForm\`x\&_\) and \(TraditionalForm\`y\&_\)  determine the average \(TraditionalForm\`q\&_\)  using

\(\(\(\ \ \ \ \ \ \ \ \ \ \ \)\(q\&_ = 
      q \((x\&_, y\&_)\)\)\)\)

For the example, q=xy we have \(TraditionalForm\`q\&_\)=\(TraditionalForm\`x\&_\) \(TraditionalForm\`y\&_\).  We can also use each data point, for example {(Subscript[x, n]),  (Subscript[y, n])} to calculate the value of q associated with that data point namely

(Subscript[q, n]) = 
      q ((Subscript[x, n]), (Subscript[y, n]))

Since the the {(Subscript[x, n]),  (Subscript[y, n])} are close to the averages \(TraditionalForm\`x\&_\) and \(TraditionalForm\`y\&_\) it makes sense to expand equation (4) in a Taylor series in the form

\(\(\(\ \ \ \ \ \ \ \ \)\(q \((x\_n, y\_n)\) = \ 
      q \((x\&_, 
              y\&_)\) + \ \(∂\ q\/∂x\) \((x\_n\  - \ 
                x\&_)\) + \ \(∂\ q\/∂y\) \((y\_n\  - \ 
                y\&_)\) + \  ... \)\)\)

We will keep only the lowest order terms in equation (5).  The derivatives are both evaluated at \(TraditionalForm\`x\&_\) and \(TraditionalForm\`y\&_\).  Using the example q=xy before,

\(\(\(\ \ \ \ \ \ \)\(∂\ q\/∂x = \ \(∂\ \((xy)\)\/∂x = \ \(\(y ∂\ x\)\/∂x = \ 
          y\)\)\)\)\)

and

\(\(\(\ \ \ \ \ \ \)\(∂\ q\/∂y = \ \(∂\ \((xy)\)\/∂y = \ \(\(x ∂\ y\)\/∂y = \ 
          x\)\)\)\)\)

Both equation (6) are evaluated at \(TraditionalForm\`x\&_\) and \(TraditionalForm\`y\&_\) so equation (5) becomes in this case

\(\(\(\ \ \ \ \ \ \ \)\(q \((x\_n, y\_n)\) = \ 
      q \((x\&_, 
            y\&_)\)\  + \ \(y\&_\) \((x\_n\  - \ 
              x\&_)\) + \ \(x\&_\) \((y\_n\  - \ 
              y\&_)\)\)\)\)

The standard deviation of the quantity q(x, y) is defined as usual

\(\(\(\ \ \ \ \ \ \ \)\(σ\_q\^2 = \ \(1\/N\) \(∑\+\(n = 1\)\%N\((q\_n - \ q\&_)\)\^2\)\)\)\)

where (Subscript[q, n])= q((Subscript[x, n]),  (Subscript[y, n])) and \(TraditionalForm\`q\&_\)= q(\(TraditionalForm\`\(\(x\&_, \ y\&_\)\()\)\)\).  Utiliztion of equation (5) in equation (9) yields

\(\(\(\ \ \)\(σ\_q\^2 = \ \(1\/N\) \(∑\+\(n = 1\)\%N\((\(∂\ q\/∂x\) \((x\_n\  - \ x\&_)\) + \ \(∂\ q\/∂y\) \((y\_n\  - \ x\&_)\))\)\^2\)\)\)\)

Multiplication of the bionomial yields

\(\(\(\ \ \)\(σ\_q\^2 = \ \(1\/N\) \((∂\ q\/∂x)\)\^2\ \(∑\+\(n = 1\)\%N\((x\_n\  - \ x\&_)\)\^2\) + \(1\/N\) \((∂\ q\/∂y)\)\^2\ \(∑\+\(n = 1\)\%N\((y\_n\  - \ y\&_)\)\^2\) + 
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \(2\/N\) \((∂\ q\/∂x)\) \((∂\ q\/∂y)\)\ \(∑\+\(n = 1\)\%N\((x\_n\  - \ x\&_)\) \((y\_n\  - \ 
                  y\&_)\)\)\)\)\)

Recall the definitions equation (2) for the standard deviations and define the covariance (Subscript[σ, xy]) via

\(\(\(\ \ \ \ \ \)\(σ\_xy = \ \(1\/N\) \(∑\+\(n = 1\)\%N\((x\_n - \ x\&_)\) \((y\_n - \ y\&_)\)\)\)\)\)

then equation (11) reduces to the much simpler form

\(\(\(\ \)\(σ\_q\^2 = \ \((∂\ q\/∂x)\)\^2\ σ\_x\^2 + \((∂\ q\/∂y)\)\^2\ σ\_y\^2 + \ 
        2 \((∂\ q\/∂x)\) \((∂\ q\/∂y)\)\ \ σ\_xy\)\)\)

Quite often there is no statistical correlation between the quantities x and y in which case (Subscript[σ, xy])=0 and equation (13) becomes the simpler result

\(σ\_q\^2 = \ \((∂\ q\/∂x)\)\^2\ σ\_x\^2 + \((∂\ q\/∂y)\)\^2\ σ\_y\^2\)

However, there are many cases where (Subscript[σ, xy]) is not zero and correlations exist between the measured values of x and y.

Condition of Zero Correlation

(Subscript[σ, x]) is positive in equation (2) because it is the sum of squares (which are all positivie).  However,  (Subscript[σ, xy]) can be either positve or negative since when \(\((x\_n - \ x\&_)\)\)is positive then \(\((y\_n - \ y\&_)\)\) might be negative.  There is nothing to assure the experimenter that both \(\((x\_n - \ x\&_)\)\) and  \(\((y\_n - \ y\&_)\)\) are positive (or negative).  
	However under some experimental circumstances, where a positive deviation in \(\((x\_n - \ x\&_)\)\) is  always accompanied by a positive deviation in \(\((y\_n - \ y\&_)\)\)and under this circumstance, (Subscript[σ, xy]) is positive.  (Similary, there are some experimental circumstances, where a negative deviation in \(\((x\_n - \ x\&_)\)\) is  always accompanied by a negative deviation in \(\((y\_n - \ y\&_)\)\)and again, (Subscript[σ, xy]) is positive.)  The contrary case also occurs where when \(\((x\_n - \ x\&_)\)\)is measured positive positive, then \(\((y\_n - \ y\&_)\)\) is always negative and as a result the correlation (Subscript[σ, xy]) is negative.   In all these case we would say x and y are correlated.
	There also occurs experimental situations where the measurement of x is independent of the measurement of y and in this case should \(\((x\_n - \ x\&_)\)\)be positive, it is equally likely that \(\((y\_n - \ y\&_)\)\) be positive or negative.  In this case, the sum in equation (12) is zero on average and we have (Subscript[σ, xy])=0.   In this case, we say x and y are uncorrelated.

Definition of Correlation Coefficient r

The covariance (Subscript[σ, xy]) can have any value positive or negative and depends upon the experiment involved.  It is useful to define a normalized correlation coefficient r using

\(\(\(\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \)\(r\  = \ σ\_xy\/\(σ\_x\ σ\_y\)\)\)\)

It is easy to prove that r must lie in the range

1  ≥  
      r  ≥  -1

Homework #1

Show that

\(\(\(\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \)\(σ\_x\ σ\_y\  ⩾ σ\_xy\  ≥ \ \(-\ σ\_x\)\ σ\_y\)\)\)

from which relationship (16) follows immediately.  HINT:  Start with the definition of A[α] below

\(\(\(\ \ \ \)\(A[α]\  ≡ \ \(1\/N\) \(∑\+\(n = 1\)\%N\((\((x\_n - \ x\&_)\) + \ α\ \((y\_n - \ y\&_)\))\)\^2\) ≥ \ 0\)\)\)

α is just some parameter.  A[α] is positive as it is the sum of squares.  The size of A[α] varies as  α changes but regardless A[α] is positive.  It is also reasonable to expect A[α] is not constant so it will have a minimum (maybe zero) which can be computed by

(d/d)α A[α] = 
      0

Use condition (19) to calculate the minimum of equation (18) (Subscript[A, min]).  Note that (Subscript[A, min])≥0 because of (18) and relationship (17) follows.

An Example of Calculating (Subscript[σ, xy])

Suppose an instructor has 10 students in the class and each has a total homework grade and final exam grade as indicated in the table below:

({{Student n, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, {Homework (Subscript[x, n]), 90, 60, 45, 100, 15, 23, 52, 30, 71, 88}, {Final Grade (Subscript[y, n]), 90, 71, 65, 100, 45, 60, 75, 85, 100, 80}})

The instructor also suspects there is a statistical relationship between a given student doing well (or poorly) on the homework and doing well (or poorly) on the final exam. The instructor wants to test this hypothesis by calculating the correlation coefficient.

Homework #2

Use the above data to show r=0.8  Show all your work.

Meaning of Specific Values of r

The correlation you just calculated is positive and closer to 1 than 0 so you might conclude that the grade a student gets on the homework is (somewhat) closely correlated to the final grade the student gets in the course.  There are ways of concluding a bit more.  Since there are N=10 data points and since r=0.8, you can conclude that there is a 0.5% probability that the homework and final grades are not correlated.  A table with required probabilties appears below:

({{r, 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0}, {3, 100, 94, 87, 81, 74, 67, 59, 51, 41, 29, 0}, {6, 100, 85, 70, 56, 43, 31, 21, 12, 6, 1, 0}, {10, 100, 78, 58, 40, 25, 14, 7, 2, 0.5, , 0}, {20, 100, 67, 40, 20, 8, 2, 05, 0.1, , , 0}, {50, 100, 49, 16, 3, 04, , , , , , 0}})

The left hand column in the above table are the N values (N=10 in the above student example) and the r values on in the top row (r=0.8 in the student example).  The 0.5% in red is at the intersection of the r=0.8 column and the N=10 row.  If you take 10 data points and calculate r=0.8 (as in the above example) then there is only a 0.5% chance that uncorrelated data points would give this result.  In other words, there is a 99.5% chance that student homegrades and student final course grades are correlated.

Homework #3

Suppose you have N=3 data points and you calculate r=0.7  What is the chance that {(Subscript[x, n]),  (Subscript[y, n])} are uncorrelated?

Homework #4

Suppose you have N=6 data points and you calculate r=0.7  What is the chance that {(Subscript[x, n]),  (Subscript[y, n])} are uncorrelated?

Homework #5

Suppose you have N=10 data points and you calculate r=0.7  What is the chance that {(Subscript[x, n]),  (Subscript[y, n])} are uncorrelated?

©  Rodney L. Varley (2001).