Integrated Python/R Mini-Project: End-to-End Analytical Script

Integrated Python/R Mini-Project: End-to-End Analytical Script

Why End-to-End Thinking Matters

A data analyst's job does not stop at writing a clever line of code. Employers expect you to move a dataset from raw form all the way through cleaning, analysis, and communication — without anyone else having to fix your work in between. This lab asks you to do exactly that: pick a real dataset, ask a focused question, write a script that answers it, and explain your reasoning to a peer audience. That sequence mirrors what Google, IBM, and CompTIA all test: the ability to plan, execute, and communicate an analytical workflow.

Step 1 — Write the Problem Statement First

Before you open a notebook or an RStudio session, write one paragraph that answers three questions: What is the analytical question? Who will read the output? What form will the output take (table, chart, report)? This document is not busywork — it is your contract with yourself. Analysts who skip this step often produce technically correct code that answers the wrong question. Keep the problem statement in a plain text or Markdown file and commit it to your repository before writing a single line of code.

Example structure: "Using the public transit ridership dataset, this script will determine whether weekday ridership on the three busiest routes increased or decreased between the first and last full months in the dataset. The output is intended for a transit operations manager and will be delivered as a CSV summary table and a short formatted text report."

Step 2 — Import and Inspect the Data

Load your chosen dataset using `pandas.read_csv()` in Python or `read.csv()` / `readr::read_csv()` in R. Immediately inspect shape, column names, and data types. Look for obvious problems before you start cleaning:

• Missing values (`df.isnull().sum()` in Python; `colSums(is.na(df))` in R)
• Columns stored as the wrong type (dates as strings, numeric amounts as objects)
• Unexpected whitespace or inconsistent capitalization in text columns

Document every issue you find in your lab log. You are required to fix at least two column-level issues. A "column-level issue" means something that affects the values in one specific column — for example, converting a date string column to a proper datetime type, or stripping leading/trailing spaces from a text field.

Step 3 — Write a Named Function with a Docstring

Your main transformation step must live inside a function that has a docstring. A docstring explains what the function does, what arguments it accepts, and what it returns. This is not optional style — it is how professional code is documented and how peer reviewers and future maintainers understand your logic without reading every line.

In Python, the docstring sits immediately inside the `def` block, enclosed in triple quotes. In R, a roxygen2-style comment block above the function serves the same purpose. Keeping transformation logic in a function also makes your script testable: you can call the function on a small sample and verify the output before running it on the full dataset.

Step 4 — Compute Summary Statistics and Export Outputs

Compute at least three summary statistics relevant to your question. These might include counts, means, medians, percent changes, or group-level aggregations — whatever the problem statement calls for. Round final values to a reasonable number of decimal places and label every column clearly.

Export two outputs:

1. A CSV file — use `df.to_csv('output.csv', index=False)` in Python or `write.csv(df, 'output.csv', row.names = FALSE)` in R.

2. A formatted text report — a plain `.txt` file that a non-technical stakeholder can open and read without knowing anything about code. Use `f-strings` or `format()` in Python, or `cat()` / `sprintf()` in R to build readable sentences around your numbers.

Both files must be written programmatically by your script — not manually created after the fact.

Step 5 — Version Control with Git/GitHub

Commit your work at logical checkpoints: after the problem statement, after the cleaning step, after the function is written, and after the final outputs are generated. Each commit message should be a short, honest description of what changed. When you submit the final revision after peer review, include a changelog comment at the top of your script or notebook explaining what you changed and why. This trains the habit employers look for: traceable, explainable work.

Step 6 — Peer Walkthrough and Revision

Deliver a five-minute walkthrough to two classmates acting as stakeholder reviewers. Walk through the problem statement, the key cleaning decisions, the function, and the final outputs. Invite at least two clarifying questions and answer them directly. After the session, note the feedback in your lab log and decide what to revise. Even if a reviewer's suggestion does not change your code, document why you accepted or declined it. Documented reasoning is evidence of professional judgment.

Data Safety and Responsible Handling

Even public datasets can contain personally identifiable information (PII) — names, addresses, transit card IDs, or account numbers included by accident. Before you begin analysis, scan for any columns that could identify individuals and confirm with your instructor that the dataset has been cleared for student use as provided. Do not post raw datasets to public GitHub repositories without instructor approval. Handle data files on school systems only, following your program's acceptable-use policy.

Pass Criteria

Your script must run end-to-end on the instructor's machine without manual edits, meaning no hard-coded file paths that only work on your computer. Use relative paths. The problem statement, the docstring function, the CSV export, and the peer-review documentation must all be present. Every element is required — meeting most of them is not a pass.