Correlation and Simple Linear Regression

Correlation and Simple Linear Regression

Why These Tools Matter

Data scientists constantly ask: Do two variables move together, and can one variable predict another? Correlation and simple linear regression are the foundational tools for answering those questions. They appear in every major data analytics credential — Google Data Analytics, CompTIA Data+, IBM Data Science Professional, and Microsoft PL-300 workflows all expect you to interpret and communicate these statistics accurately.

---

Pearson Correlation Coefficient

The Pearson correlation coefficient (r) measures the strength and direction of the linear relationship between two continuous variables. Its value always falls between –1 and +1.

• r = +1: perfect positive linear relationship (as X rises, Y rises proportionally)
• r = 0: no linear relationship
• r = –1: perfect negative linear relationship

A common rule of thumb: |r| < 0.3 is weak, 0.3–0.7 is moderate, and |r| > 0.7 is strong, though context always matters.

The formula is:

```

r = Σ[(xᵢ − x̄)(yᵢ − ȳ)] / √[Σ(xᵢ − x̄)² · Σ(yᵢ − ȳ)²]

```

In practice you will use `pandas` or `scipy.stats.pearsonr()` rather than computing this by hand for large datasets.

Important limitation: Correlation measures only linear association. Two variables can have a strong curved relationship yet an r near zero. Always pair a numeric result with a scatter plot.

---

Correlation Matrix and Heat Map

When you have several variables, a correlation matrix shows every pairwise r value in a table. A heat map (seaborn's `heatmap()`) colors each cell so that strong positive correlations appear in one color and strong negative correlations in another. This visualization lets you quickly spot which pairs of variables deserve further analysis and which are essentially unrelated.

```python

import seaborn as sns

corr_matrix = df[cols].corr() # Pearson by default

sns.heatmap(corr_matrix, annot=True, cmap='coolwarm', fmt='.2f')

```

---

Simple Linear Regression

Simple linear regression fits a straight line through data to model the relationship between one predictor (X) and one outcome (Y):

```

ŷ = β₀ + β₁x

```

• β₀ (intercept): the predicted value of Y when X = 0
• β₁ (slope): the expected change in Y for each one-unit increase in X

The Ordinary Least Squares (OLS) method finds the line that minimizes the sum of squared differences between observed values (y) and predicted values (ŷ). These differences are called residuals (e = y − ŷ).

In Python you can use either `statsmodels` (for a detailed summary table) or `scikit-learn` (for prediction pipelines).

```python

import statsmodels.formula.api as smf

model = smf.ols('SalePrice ~ GrLivArea', data=df).fit()

print(model.summary())

```

---

Interpreting the Summary Table

The OLS summary provides several key statistics:

• Coefficient (slope): "For each additional square foot of living area, the predicted sale price increases by $\[slope\]." Always state units.
• p-value: If p < 0.05, the slope is statistically significant — unlikely to be zero by chance.
• R-squared (R²): The proportion of variance in Y explained by X. R² = 0.72 means 72 % of the variation in sale price is explained by living area. It ranges from 0 to 1; higher is better, but context determines what counts as useful.
• Confidence intervals: The range within which the true slope likely falls.

---

Residual Diagnostics

OLS regression rests on several assumptions. Violating them can make coefficients unreliable. Always check residuals after fitting.

1. Linearity: Residuals vs. fitted plot should show points scattered randomly around zero — no curve.

2. Constant variance (homoscedasticity): The spread of residuals should be roughly equal across all fitted values. A fan shape signals heteroscedasticity.

3. Normality of residuals: A QQ-plot (quantile-quantile plot) compares residual quantiles to a normal distribution. Points following the diagonal line indicate normality; heavy tails mean departures.

4. Independence: Residuals should not show patterns by time or group.

Common remediations:

• Log-transform Y or X to reduce skew or a fan shape
• Remove or investigate influential outliers (check Cook's distance)
• Add a polynomial term if a curve is present

---

Data Safety and Integrity Notes

Data science does not involve physical hazards like lockout/tagout, but professional safety still applies:

• Never publish or share data containing personally identifiable information (PII) unless you have confirmed authorization. The Ames housing dataset used in this lab is public domain and contains no PII.
• Validate your dataset before analysis. Missing values, duplicate rows, or mis-coded entries can silently distort correlation and regression results. Always run `df.info()` and `df.describe()` first.
• Document your steps. Reproducibility is an ethical obligation in data work — another analyst must be able to verify your results.

---

Lab Workflow Summary

1. Load and inspect the Ames housing dataset.

2. Select five variable pairs, compute Pearson r, build a heat map.

3. Fit OLS regression of `SalePrice` on `GrLivArea`.

4. Interpret slope, intercept, R², and p-value in plain English.

5. Plot residuals vs. fitted values and a QQ-plot; write your findings.

Refer to OpenIntro Statistics Chapter 8 for additional examples and derivations.