Partial correlation

Not to be confused with Coefficient of partial determination.

In probability theory and statistics, partial correlation measures the degree of association between two random variables, with the effect of a set of controlling random variables removed.

Formal definition

Formally, the partial correlation between X and Y given a set of n controlling variables Z = {Z₁, Z₂, ..., Z_n}, written ρ_XY·Z, is the correlation between the residuals R_X and R_Y resulting from the linear regression of X with Z and of Y with Z, respectively. The first-order partial correlation (i.e. when n=1) is the difference between a correlation and the product of the removable correlations divided by the product of the coefficients of alienation of the removable correlations. The coefficient of alienation, and its relation with joint variance through correlation are available in Guilford (1973, pp. 344–345).^[1]

Computation

Using linear regression

A simple way to compute the sample partial correlation for some data is to solve the two associated linear regression problems, get the residuals, and calculate the correlation between the residuals. Let X and Y be, as above, random variables taking real values, and let Z be the n-dimensional vector-valued random variable. If we write x_i, y_i and z_i to denote the ith of N i.i.d. samples of some joint probability distribution over real random variables X, Y and Z, solving the linear regression problem amounts to finding n-dimensional coefficient vectors $\mathbf {w} _{X}^{*}$ and $\mathbf {w} _{Y}^{*}$ such that

\mathbf {w} _{X}^{*}=\arg \min _{\mathbf {w} }\left\{\sum _{i=1}^{N}(x_{i}-\langle \mathbf {w} ,\mathbf {z} _{i}\rangle )^{2}\right\}

\mathbf {w} _{Y}^{*}=\arg \min _{\mathbf {w} }\left\{\sum _{i=1}^{N}(y_{i}-\langle \mathbf {w} ,\mathbf {z} _{i}\rangle )^{2}\right\}

with N being the number of samples and $\langle \mathbf {v} ,\mathbf {w} \rangle$ the scalar product between the vectors v and w. Note that in some formulations the regression includes a constant term, so the matrix $\mathbf {z}$ would have an additional column of ones.

The residuals are then

r_{X,i}=x_{i}-\langle \mathbf {w} _{X}^{*},\mathbf {z} _{i}\rangle

r_{Y,i}=y_{i}-\langle \mathbf {w} _{Y}^{*},\mathbf {z} _{i}\rangle

and the sample partial correlation is then given by the usual formula for sample correlation , but between these new derived values.

{\hat {\rho }}_{XY\cdot \mathbf {Z} }={\frac {N\sum _{i=1}^{N}r_{X,i}r_{Y,i}-\sum _{i=1}^{N}r_{X,i}\sum _{i=1}^{N}r_{Y,i}}{{\sqrt {N\sum _{i=1}^{N}r_{X,i}^{2}-\left(\sum _{i=1}^{N}r_{X,i}\right)^{2}}}~{\sqrt {N\sum _{i=1}^{N}r_{Y,i}^{2}-\left(\sum _{i=1}^{N}r_{Y,i}\right)^{2}}}}}.

Using recursive formula

It can be computationally expensive to solve the linear regression problems. Actually, the nth-order partial correlation (i.e., with |Z| = n) can be easily computed from three (n - 1)th-order partial correlations. The zeroth-order partial correlation ρ_XY·Ø is defined to be the regular correlation coefficient ρ_XY.

It holds, for any $Z_{0}\in \mathbf {Z}$ :

\rho _{XY\cdot \mathbf {Z} }={\frac {\rho _{XY\cdot \mathbf {Z} \setminus \{Z_{0}\}}-\rho _{XZ_{0}\cdot \mathbf {Z} \setminus \{Z_{0}\}}\rho _{Z_{0}Y\cdot \mathbf {Z} \setminus \{Z_{0}\}}}{{\sqrt {1-\rho _{XZ_{0}\cdot \mathbf {Z} \setminus \{Z_{0}\}}^{2}}}{\sqrt {1-\rho _{Z_{0}Y\cdot \mathbf {Z} \setminus \{Z_{0}\}}^{2}}}}}.

Naïvely implementing this computation as a recursive algorithm yields an exponential time complexity. However, this computation has the overlapping subproblems property, such that using dynamic programming or simply caching the results of the recursive calls yields a complexity of ${\mathcal {O}}(n^{3})$ .

Note in the case where Z is a single variable, this reduces to:

\rho _{XY\cdot Z}={\frac {\rho _{XY}-\rho _{XZ}\rho _{ZY}}{{\sqrt {1-\rho _{XZ}^{2}}}{\sqrt {1-\rho _{ZY}^{2}}}}}.

Using matrix inversion

In ${\mathcal {O}}(n^{3})$ time, another approach allows all partial correlations to be computed between any two variables X_i and X_j of a set V of cardinality n, given all others, i.e., $\mathbf {V} \setminus \{X_{i},X_{j}\}$ , if the correlation matrix (or alternatively covariance matrix) Ω = (ω_ij), where ω_ij = ρ_{X_iX_j}, is positive definite and therefore invertible. If we define P = Ω⁻¹, we have:

\rho _{X_{i}X_{j}\cdot \mathbf {V} \setminus \{X_{i},X_{j}\}}=-{\frac {p_{ij}}{\sqrt {p_{ii}p_{jj}}}}.

Interpretation

Geometrical

Let three variables X, Y, Z (where Z is the "control" or "extra variable") be chosen from a joint probability distribution over n variables V. Further let v_i, 1 ≤ i ≤ N, be N n-dimensional i.i.d. samples taken from the joint probability distribution over V. We then consider the N-dimensional vectors x (formed by the successive values of X over the samples), y (formed by the values of Y) and z (formed by the values of Z).

It can be shown that the residuals R_X coming from the linear regression of X on Z, if also considered as an N-dimensional vector r_X, have a zero scalar product with the vector z generated by Z. This means that the residuals vector lies on an (N–1)-dimensional hyperplane S_z that is perpendicular to z.

The same also applies to the residuals R_Y generating a vector r_Y. The desired partial correlation is then the cosine of the angle φ between the projections r_X and r_Y of x and y, respectively, onto the hyperplane perpendicular to z.^[2]^{:ch. 7}

As conditional independence test

Semipartial correlation (part correlation)

The semipartial (or part) correlation statistic is similar to the partial correlation statistic. Both compare variations of two variables after certain factors are controlled for, but to calculate the semipartial correlation one holds the third variable constant for either X or Y but not both, whereas for the partial correlation one holds the third variable constant for both.^[6] The semipartial correlation compares the unique variation of one variable (having removed variation associated with the Z variable(s)), with the unfiltered variation of the other, while the partial correlation compares the unique variation of one variable to the unique variation of the other.

The semipartial (or part) correlation can be viewed as more practically relevant "because it is scaled to (i.e., relative to) the total variability in the dependent (response) variable." ^[7] Conversely, it is less theoretically useful because it is less precise about the role of the unique contribution of the independent variable.

The absolute value of the semipartial correlation of X with Y is always less than or equal to that of the partial correlation of X with Y. The reason is this: Suppose the correlation of X with Z has been removed from X, giving the residual vector r_x . In computing the semipartial correlation, Y still contains both unique variance and variance due to its association with Z. But r_x , being uncorrelated with Z, can only explain some of the unique part of the variance of Y and not the part related to Z. In contrast, with the partial correlation, only r_y (the part of the variance of Y that is unrelated to Z) is to be explained, so there is less variance of the type that r_x cannot explain.

Use in time series analysis

In time series analysis, the partial autocorrelation function (sometimes "partial correlation function") of a time series is defined, for lag h, as

\varphi (h)=\rho _{X_{0}X_{h}\,\cdot \,\{X_{1},\,\dots \,,X_{h-1}\}}.

This function is used to determine the appropriate lag length for an autoregression.

References

↑ Guilford J. P., Fruchter B. (1973). Fundamental statistics in psychology and education. Tokyo: McGraw-Hill Kogakusha, LTD.
↑ Rummel, R. J. (1976). "Understanding Correlation".
↑ Baba, Kunihiro; Ritei Shibata; Masaaki Sibuya (2004). "Partial correlation and conditional correlation as measures of conditional independence". Australian and New Zealand Journal of Statistics. 46 (4): 657–664. doi:10.1111/j.1467-842X.2004.00360.x.
↑ Kendall MG, Stuart A. (1973) The Advanced Theory of Statistics, Volume 2 (3rd Edition), ISBN 0-85264-215-6, Section 27.22
↑ Fisher, R.A. (1924). "The distribution of the partial correlation coefficient". Metron. 3 (3–4): 329–332.
↑ http://luna.cas.usf.edu/~mbrannic/files/regression/Partial.html. Missing or empty |title= (help)
↑ StatSoft, Inc. (2010). "Semi-Partial (or Part) Correlation", Electronic Statistics Textbook. Tulsa, OK: StatSoft, accessed January 15, 2011.

External links

Wikiversity has learning materials about Partial correlation

Prokhorov, A.V. (2001), "Partial correlation coefficient", in Hazewinkel, Michiel, Encyclopedia of Mathematics, Springer, ISBN 978-1-55608-010-4
What is a partial correlation?
Mathematical formulae in the "Description" section of the IMSL Numerical Library PCORR routine
A three-variable example

Statistics

Descriptive statistics

Continuous data

Center	Mean arithmetic geometric harmonic Median Mode

Dispersion	Variance Standard deviation Coefficient of variation Percentile Range Interquartile range

Shape	Moments Skewness Kurtosis L-moments

Count data

Index of dispersion

Summary tables

Dependence

Graphics

Data collection

Study design	Population Statistic Effect size Statistical power Sample size determination Missing data

Survey methodology	Sampling Standard error stratified cluster Opinion poll Questionnaire

Controlled experiments	Design control optimal Controlled trial Randomized Random assignment Replication Blocking Interaction Factorial experiment

Uncontrolled studies	Observational study Natural experiment Quasi-experiment

Statistical inference

Statistical theory

Frequentist inference

Point estimation	Estimating equations Maximum likelihood Method of moments M-estimator Minimum distance Unbiased estimators Mean-unbiased minimum-variance Rao–Blackwellization Lehmann–Scheffé theorem Median unbiased Plug-in

Interval estimation	Confidence interval Pivot Likelihood interval Prediction interval Tolerance interval Resampling Bootstrap Jackknife

Testing hypotheses	1- & 2-tails Power Uniformly most powerful test Permutation test Randomization test Multiple comparisons

Parametric tests	Likelihood-ratio Wald Score

Specific tests

Z (normal) Student's t-test F

Goodness of fit	Chi-squared Kolmogorov–Smirnov Anderson–Darling Normality (Shapiro–Wilk) Likelihood-ratio test Model selection Cross validation AIC BIC

Rank statistics	Sign Sample median Signed rank (Wilcoxon) Hodges–Lehmann estimator Rank sum (Mann–Whitney) Nonparametric anova 1-way (Kruskal–Wallis) 2-way (Friedman) Ordered alternative (Jonckheere–Terpstra)

Bayesian inference

Correlation	Pearson product–moment Partial correlation Confounding variable Coefficient of determination

Regression analysis	Errors and residuals Regression model validation Mixed effects models Simultaneous equations models Multivariate adaptive regression splines (MARS)

Linear regression	Simple linear regression Ordinary least squares General linear model Bayesian regression

Non-standard predictors	Nonlinear regression Nonparametric Semiparametric Isotonic Robust Heteroscedasticity Homoscedasticity

Generalized linear model	Exponential families Logistic (Bernoulli) / Binomial / Poisson regressions

Partition of variance	Analysis of variance (ANOVA, anova) Analysis of covariance Multivariate ANOVA Degrees of freedom

Categorical / Multivariate / Time-series / Survival analysis

Categorical

Multivariate

Time-series

General	Decomposition Trend Stationarity Seasonal adjustment Exponential smoothing Cointegration Structural break Granger causality

Specific tests	Dickey–Fuller Johansen Q-statistic (Ljung–Box) Durbin–Watson Breusch–Godfrey

Time domain	Autocorrelation (ACF) partial (PACF) Cross-correlation (XCF) ARMA model ARIMA model (Box–Jenkins) Autoregressive conditional heteroskedasticity (ARCH) Vector autoregression (VAR)

Frequency domain	Spectral density estimation Fourier analysis Wavelet

Survival

Survival function	Kaplan–Meier estimator (product limit) Proportional hazards models Accelerated failure time (AFT) model First hitting time

Hazard function	Nelson–Aalen estimator

Test	Log-rank test

Applications

Biostatistics	Bioinformatics Clinical trials / studies Epidemiology Medical statistics

Engineering statistics	Chemometrics Methods engineering Probabilistic design Process / quality control Reliability System identification

Social statistics	Actuarial science Census Crime statistics Demography Econometrics National accounts Official statistics Population statistics Psychometrics

Spatial statistics	Cartography Environmental statistics Geographic information system Geostatistics Kriging

Category
Portal
Commons
WikiProject

This article is issued from Wikipedia - version of the 12/3/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.