Probability Fundamentals

What Is Probability?

Probability is the numerical measure of how likely an event is to occur. It is expressed as a number between 0 and 1, where 0 means the event cannot happen and 1 means it is certain to happen. In data science, probability underpins everything from predictive models to anomaly detection to algorithmic fairness audits. Mastering a few core rules lets you calculate the likelihood of complex events from simpler building blocks.

Sample Spaces and Events

A sample space (S) is the complete set of all possible outcomes of an experiment. Rolling a standard six-sided die produces S = {1, 2, 3, 4, 5, 6}. An event is any subset of the sample space. For example, event A = "roll an even number" = {2, 4, 6}.

The complement of event A, written A′ or Aᶜ, contains every outcome in S that is not in A. Because every outcome must either be in A or not in A:

> P(A′) = 1 − P(A)

If a manufactured batch has a 0.08 defect rate, the probability of a non-defective item is 1 − 0.08 = 0.92.

Union and Intersection

• Union (A ∪ B): the event that A or B (or both) occurs.
• Intersection (A ∩ B): the event that A and B both occur.

The Addition Rule

The addition rule gives the probability that at least one of two events occurs:

> P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

Subtracting P(A ∩ B) corrects for double-counting outcomes that fall in both events.

Mutually exclusive events (also called disjoint events) cannot occur at the same time, so P(A ∩ B) = 0. The rule simplifies to:

> P(A ∪ B) = P(A) + P(B) [mutually exclusive only]

Example: drawing a King or a Queen from a standard deck. These are mutually exclusive, so P(King ∪ Queen) = 4/52 + 4/52 = 8/52 ≈ 0.154.

The Multiplication Rule

The multiplication rule gives the probability that two events both occur:

> P(A ∩ B) = P(A) × P(B|A)

where P(B|A) is the conditional probability of B given A (explained below).

Independent events are events where the occurrence of one does not affect the probability of the other. When A and B are independent:

> P(A ∩ B) = P(A) × P(B) [independent events only]

Do not assume independence without justification. In data contexts, features are rarely truly independent, and mistakenly treating them as such leads to incorrect model assumptions.

Conditional Probability

Conditional probability measures the probability of event B given that event A has already occurred:

> P(B|A) = P(A ∩ B) / P(A), provided P(A) > 0

This concept is central to Bayesian reasoning, decision trees, and fairness analysis. When you evaluate whether an algorithm treats groups differently, you are essentially comparing conditional probabilities across subgroups — for example, P(loan approved | applicant in group X) versus P(loan approved | applicant in group Y).

Mutually Exclusive vs. Independent: A Common Confusion

These two concepts are often mixed up.

• Mutually exclusive: the events cannot both happen; if one occurs, the other cannot. P(A ∩ B) = 0.
• Independent: knowing one occurred gives no information about the other. P(B|A) = P(B).

If two events are mutually exclusive and each has nonzero probability, they are not independent — knowing A occurred tells you B definitely did not.

Probability in Data Quality and Ethics

Probability calculations appear throughout the data analytics workflow:

• Quality control: estimating the chance that a sampled batch contains defective units.
• Risk scoring: computing the joint probability of multiple risk indicators.
• Algorithmic fairness: conditional probabilities reveal whether a model's decisions differ systematically across demographic groups. Disparities in P(outcome | group) can signal bias that must be investigated before a model is deployed.

When working with sensitive data in these analyses, always follow your organization's data governance policies and applicable privacy regulations. Protect individual-level data; use aggregated or anonymized tables whenever the task permits.

Connecting Theory to Simulation

Theoretical probability tells you what should happen in the long run; observed (empirical) frequency tells you what did happen in a finite number of trials. As the number of trials increases, empirical frequency converges toward theoretical probability — a principle called the Law of Large Numbers. In today's simulation activity you will see this directly: pooling the class data produces a result closer to the theoretical value than any single pair's 50 trials alone.

Reading Recommendations

For additional depth, read Chapters 2 and 3 of OpenIntro Statistics (Diez et al.), available as a free PDF. Pay particular attention to the section on probability rules and the worked examples using contingency tables.