Data Science Case Studies

Case studies assess the ability to apply analytical skills to business problems. These questions evaluate problem framing, hypothesis generation, and communication of analytical approaches.

Case Study vs Technical Interview Comparison

Aspect	Technical Interview	Case Study Interview
Question type	Implementation focused	Open-ended business problem
Format	Coding or calculation	Discussion and analysis
Answer type	Single correct solution	Multiple valid approaches
Duration	30-45 minutes	45-60 minutes

Standard Framework

Step 1: Problem Clarification

Establish precise understanding before analysis.

Questions to address:

Category	Example Questions
Metric definition	What metric? How is it calculated?
Time period	When did the change occur? What is the comparison period?
Scope	All users or specific segment? All platforms?
Business context	Why does this metric matter? What decisions depend on it?
Baseline	What is the normal range or expected value?

Step 2: Hypothesis Generation

Develop potential explanations organized by category.

Category	Examples
Product changes	Feature release, bug, UI modification
External factors	Seasonality, competition, news events
User composition shift	Different user segments, acquisition changes
Technical issues	Tracking bug, data pipeline failure
Organic trends	Market saturation, behavioral evolution

Step 3: Data Requirements

Identify data needed to test each hypothesis.

Hypothesis	Required Data
Product change impact	Metric by feature version, release dates
Seasonal effect	Year-over-year comparison, historical patterns
Tracking issue	Raw event counts, pipeline logs
User mix shift	Metric by segment, new vs existing users

Step 4: Analysis Approach

Structure the analytical workflow:

1. Address highest-likelihood hypotheses first
2. Examine aggregate trends before segment-level analysis
3. Segment data to identify concentration of effect
4. Account for confounding variables

Step 5: Implications

Connect findings to business decisions:

• Define recommendations for each hypothesis outcome
• Specify confidence threshold for action
• Evaluate risk of action vs inaction

Case Study Categories

Metric Change Investigation

Example: DAU decreased 15% last week.

Standard approach:

1. Confirm the metric

   • Verify metric definition and calculation
   • Assess statistical significance vs normal variance

2. Check technical causes first

   • Tracking implementation changes
   • Data pipeline status
   • Compare against secondary data sources

3. Segment the change

   • Platform (iOS, Android, Web)
   • Geography
   • User tenure (new vs existing)
   • Acquisition source

4. Identify correlated events

   • Product releases
   • Marketing campaign changes
   • Competitor actions
   • External events

5. Investigate largest-impact segment

   • What distinguishes affected users?
   • What changed for this segment?

Diagnostic indicators:

Observation	Implication
Change coincides with specific date	Product or external event
Change isolated to single segment	Segment-specific cause
Percentage change but not absolute	Denominator change

Feature Impact Analysis

Example: New feature launched. Assess effectiveness.

Framework:

1. Define success criteria

   • Target metric and expected magnitude
   • Measurement time period

2. Establish baseline

   • Pre-launch metric values
   • Historical variance

3. Measure impact

   • Treatment vs control comparison (A/B test)
   • Before vs after analysis (with limitations)
   • Novelty effect assessment

4. Identify confounds

   • Concurrent changes
   • Selection bias in feature adoption
   • Cannibalization effects

5. Formulate recommendation

   • Continue, iterate, or discontinue
   • Confidence level in conclusion

Root Cause Analysis

Example: Support tickets increased 50%.

Systematic approach:

1. Quantify the problem

   • Onset timing and growth rate
   • Absolute volume

2. Categorize tickets

   • Topic distribution (billing, bugs, how-to)
   • Identify high-growth categories

3. Segment by user

   • New vs existing users
   • Platform or version
   • Geographic concentration

4. Correlate with changes

   • Product releases
   • Marketing campaigns
   • Policy changes

5. Quantify impact

   • Support resource cost
   • User impact (churn risk)

Opportunity Sizing

Example: Evaluate potential value of building feature X.

Framework:

1. Define the opportunity

   • User problem addressed
   • Target user segment

2. Size the market

   • Addressable user count
   • Current behavior baseline

3. Estimate impact

   • Target metric improvement
   • Magnitude estimate using comparables

4. Account for costs

   • Development time
   • Ongoing maintenance
   • Opportunity cost

5. Calculate expected value

   • ROI estimate
   • Confidence interval
   • Decision reversal criteria

Worked Example: Engagement Drop

Prompt: Weekly active users (WAU) decreased 8% month-over-month.

Clarification Phase

Question	Assumed Answer
WAU definition	Users opening app at least once
Normal variance	+/- 2% MoM
Change pattern	Gradual over month
Recent changes	New onboarding flow launched 3 weeks ago

Hypothesis Generation

1. New onboarding flow causing new user drop-off
2. Seasonal effect (end of summer)
3. Technical issue with app or tracking
4. Competitive product launch

Analysis Plan

Step 1: Segment by user type

• Determine if drop concentrated in new vs existing users
• If new users affected, onboarding hypothesis gains support

Step 2: Compare onboarding versions

• Compare completion rates between versions
• Analyze Day 7 retention by version

Step 3: Technical verification

• Check for crashes in new onboarding
• Verify tracking coverage

Step 4: Seasonal adjustment

• Compare to same period previous year
• Check competitor trends

Step 5: Identify specific drop-off points

• Funnel analysis within onboarding
• User research on abandonment reasons

Example Findings

Metric	Old Onboarding	New Onboarding
Completion rate	65%	40%
Drop-off point	Various	Email verification
WAU contribution to drop	20%	80%

Root cause: Mandatory email verification in new flow creates friction before users experience product value.

Recommendation: Make email verification optional or delayed. Validate with A/B test before full rollout.

Worked Example: Opportunity Sizing

Prompt: Evaluate adding dark mode feature.

Analysis Framework

1. Define opportunity

• User need: Low-light usage comfort
• Value: Improved experience, potential increased evening usage

2. Size the market

Data Point	Source
Dark mode feature requests	Support tickets, surveys
Evening session percentage	Usage analytics
Competitor dark mode adoption	Market research

3. Estimate impact

Metric	Estimate	Basis
Current DAU	1,000,000	Internal data
Evening sessions	30%	Internal data
Dark mode adoption rate	40%	Industry benchmarks
Evening session increase	10%	Conservative estimate
DAU impact	1-2%	Calculated

4. Assess costs

• Engineering: ~4 weeks
• Design: All screens require dark variants
• Ongoing: Additional testing surface

5. Recommendation

The feature provides modest impact (~1-2% DAU lift) but addresses user expectations and competitive parity. Priority depends on alternative uses of engineering resources.

Common Errors

Error	Description
Premature solution	Proposing solutions before understanding problem
Ignoring confounds	Attributing causation without controlling for other factors
Missing business context	Technical analysis without decision relevance
Vague analysis	General approach without specific steps
Unquantified opportunity	Qualitative assessment without numbers
Overcomplicating	Using complex methods when simple analysis suffices

Communication Structure

Answer Organization

1. State approach: "I will analyze this in four steps: clarify the problem, generate hypotheses, propose analysis, and discuss implications."
2. Verbalize reasoning throughout the discussion
3. Use visual organization (lists, diagrams, timelines)
4. Check for alignment: "Does this approach make sense? Should I elaborate on any part?"
5. Acknowledge uncertainty: "This is an estimate; validation with historical data would be needed."

Practice Questions

Question	Assessment Focus
Conversion rate dropped 5%. Investigate.	Metric diagnosis, segmentation
Should we launch in a new country?	Opportunity sizing, market analysis
Users report slow app performance.	Root cause analysis, prioritization
New feature increased engagement but decreased revenue.	Trade-off analysis, metric relationships
How would you measure recommendation system success?	Metric design, experimentation
Competitor launched similar feature. How to respond?	Competitive analysis, strategic thinking

Evaluation Criteria

Case studies assess the following competencies:

Competency	Evaluation Criteria
Problem clarification	Asks relevant scoping questions
Hypothesis generation	Structures potential explanations systematically
Analytical translation	Converts business questions to data problems
Communication	Explains reasoning clearly
Decision making	Reaches conclusions with incomplete information