Econ Statistics & Data Lab
"Extracting signal from noise: The grammar of economic reality."
The Architecture of Measurement
Economic theory without data is merely philosophy. Realm 17 is the laboratory where reality is quantified. It details how we collect observations, organise them into coherent distributions, extract their central tendencies and dispersions, and guard against the profound biases that can turn numbers into lies.
The 87 Nodes of Realm 17
Deconstructing The Data Laboratory
Data
- Observations or facts collected for analysis.
- Income, prices, population, employment.
Primary Data
- Collected directly by investigator for specific purpose.
- Surveys, interviews, questionnaires.
Secondary Data
- Data already collected by someone else.
- Census, RBI publications, World Bank databases.
Primary vs Secondary
- Primary: Tailored, first-hand.
- Secondary: Cheaper/faster, but requires evaluation.
Data Sources
- Census, NSO, RBI, Govt ministries, SEBI.
- Exact source should always be recorded.
Census
- Attempts to collect info from EVERY unit of population.
- E.g., Population census.
Sample
- Subset selected from population for study.
- Representative fraction instead of entire group.
Population
- Complete set of units relevant to investigation.
- E.g., All relevant households in a city.
Census vs Sample
- Sampling reduces time, cost, collection burden.
- But introduces sampling-error considerations.
Sampling Methods
- Random, systematic, stratified, cluster.
- Objective: obtain representative sample.
Random Sampling
- Each unit has specified probability of selection.
- Reduces selection bias.
Sampling Error
- Diff between sample estimate and population value.
- Larger sample can reduce this uncertainty.
Non-Sampling Error
- Incorrect responses, data-entry mistakes.
- Large sample ≠ automatically high-quality data.
Variable
- Characteristic taking different values.
- Income, age, price, employment status.
Qual vs Quant Data
- Qualitative: Categories (gender, occupation).
- Quantitative: Numerical (income, age, output).
Discrete Variable
- Takes distinct, countable values.
- E.g., Number of children, number of employees.
Continuous Variable
- Can take values over a continuous range.
- E.g., Height, weight, temperature, income.
Raw Data
- Data in original collected form.
- Before systematic organisation.
Classification of Data
- By qualitative/quantitative traits.
- By time, location, or other criteria.
Frequency
- How many times a value/category occurs.
- E.g., 25 households in the 20k-30k income group.
Frequency Distribution
- Organizes observations into classes/values.
- Shows their respective frequencies.
Grouped Data
- Observations organized into intervals/classes.
- Useful for large numbers of observations.
Class Interval
- Interval between lower and upper boundaries.
- E.g., 20–30 is a class interval.
Class Width
- Size of the class interval.
- E.g., 20–30 → width = 10.
Cumulative Frequency
- Adds frequencies progressively.
- Less-than vs More-than Ogives.
Tabulation
- Presents data systematically in rows/columns.
- Should identify title, units, source, time period.
Diagrams
- Visual presentation of economic data.
- Bar, pie, histograms, line graphs.
Bar Graph
- For discrete or categorical data.
- Bars are generally separated. E.g., GDP by sector.
Histogram
- For frequency distribution of continuous data.
- Bars touch each other (no gaps).
Pie Chart
- Represents components of a whole (360°).
- Angle = (Component / Total) × 360°.
Line Graph
- Useful for showing changes over time.
- E.g., Inflation over years 2020 to 2023.
Arithmetic Mean
- Sum of observations ÷ Number of obs.
- Uses every observation in the dataset.
Weighted Mean
- Used when observations have different importance.
- Crucial for economic indexes.
Median
- Middle value when data is ordered.
- Better for highly skewed income distributions.
Mode
- Value occurring most frequently.
- Depends purely on frequency peaks.
Mean vs Median vs Mode
- Mean uses all data, sensitive to outliers.
- Median ignores extremes, good for income.
Dispersion
- How spread out are the observations?
- Range, Quartiles, Mean Dev, Std Dev, Variance.
Range
- Maximum value − Minimum value.
- Simple but highly sensitive to extreme observations.
Quartiles
- Divide ordered data into four parts.
- Q1 (25%), Q2 (Median), Q3 (75%).
Interquartile Range
- IQR = Q3 − Q1.
- Measures spread of the middle 50% of data.
Standard Deviation
- Dispersion of observations around their mean.
- Higher SD = greater dispersion.
Variance
- Squared measure of dispersion.
- Variance = (Standard Deviation)².
Coefficient of Variation
- Relative measure of dispersion.
- Useful for comparing datasets with different scales.
Skewness
- Asymmetry in a distribution.
- Right skew: long tail to higher values (Income).
Correlation
- Degree and direction of association.
- E.g., Income ↔ Consumption.
Positive Correlation
- Variables move in the same direction.
- X ↑ → Y ↑.
Negative Correlation
- Variables move in opposite directions.
- X ↑ → Y ↓.
Zero Correlation
- Little or no linear association.
- Scattered randomly.
Correlation Coeff (r)
- Value lies between -1 and +1.
- +1 = Perfect positive linear correlation.
Corr ≠ Causation
- Most important data literacy rule.
- X & Y moving together doesn't mean X causes Y.
Time Series
- Observations across time (GDP by year).
- Trend, seasonality, cycles, irregular.
Trend
- Underlying long-term direction.
- E.g., Real GDP increasing over decades.
Seasonality
- Regular patterns repeating during periods.
- E.g., Festival demand, agri cycles.
Index Numbers
- Relative change compared to base period.
- Base is assigned 100.
Simple Price Index
- Price Relative = (Current / Base) × 100.
Price Index
- Tracks changes across a basket (CPI, WPI).
- Connects Realm 17 with Realm 11.
Quantity Index
- Measures changes in quantities of goods.
Value Index
- Changes in total monetary value of a basket.
Weighted Index
- Items get weights based on importance.
- More meaningful than equal weighting.
CPI as Econ Index
- Major example of index number.
- Tracks cost of consumer basket over time.
Nominal vs Real
- Nominal: Current prices.
- Real: Adjusted for price-level changes.
Percentage Change
- (New − Old) ÷ Old × 100.
Growth Rate
- Rate at which a variable changes over time.
Compound Growth
- FV = PV × (1 + g)ⁿ.
- Useful for long-term economic projections.
Data Comparability
- Check definition, time period, units, base year.
- Otherwise comparisons are misleading.
Absolute vs Relative
- 10M unemployed vs 6% unemployment rate.
- Population size dictates interpretation.
Rate vs Level
- Level: Absolute quantity.
- Rate: Quantity relative to denominator.
Per Capita Data
- Adjusts an aggregate by population.
- Improves comparisons but ignores inequality.
Real PC Income
- Accounts for both population AND price changes.
- Better for long-term living standards.
Base Effect
- Growth rates highly influenced by previous value.
- 100→200 (100%), then 200→220 (10%).
Index Base Effect
- Applies to inflation and indices.
- High/low previous base distorts current growth rate.
Data Bias
- Data ≠ automatically truth.
- Biased by sampling, selection, reporting.
Sample Selection Bias
- Systematically excluding certain groups.
- Sample ≠ representative population.
Survivorship Bias
- Analyzing only entities that survive.
- E.g., Studying successful startups, ignoring failed.
Cherry-Picking
- Selecting only data supporting a conclusion.
- Good analysis uses the full dataset.
Data Vis Literacy
- Check axes, units, time period, base.
- Is axis truncated? Scale consistent?
Meaningful Source
- Identify source, methodology, definition, units.
- Makes data reproducible and interpretable.
Official Statistics
- Published by govt statistical agencies.
- E.g., MoSPI/NSO in India.
Data & Policy
- Diagnosis → Design → Monitor → Evaluate.
- Data → Evidence → Policy.
Data → Theory
- Reality → Data → Stats → Model → Theory.
- Built into the Measurement Engine.
Statistical Misinterp
- Confusing correlation with causation.
- Confusing nominal with real, level with rate.
Pct vs Pct Point
- Change of 5% to 6% is 1 percentage point.
- But it is a 20% increase. Not the same.
Real Data Lab
- If Nom GDP ↑ 10% and Infl = 5%...
- Real GDP growth ≠ 10%. Adjust for prices.
Stats & Decision
- Question → Data → Clean → Measure.
- Visualize → Analyze → Interpret → Decide.
Master Data Lab Chain
- Question → Sample → Raw Data → Tables/Graphs.
- Mean/Dispersion/Correl → Econ Conclusion.
Master Literacy Chain
- Number → Unit? → Denominator? → Base?
- Collection Method? → Bias? → Actual Proof?
Formula Bank
- Mean = Σx/n. Range = Max - Min. CV = SD/Mean*100.
- Correlation: -1 ≤ r ≤ +1.
- Pct Change = (New-Old)/Old*100.
Exit Portal
We have forged the instruments of observation. We know how to extract the signal of reality from the noise of raw numbers, and how to defend against the cognitive traps of bias and misrepresentation. Yet, even perfect measurement of the current system is not enough if the system itself is on a collision course with physical reality. We must now turn our analytical lens upon the limits of the planet itself: the environment, natural capital, and the urgent economics of sustainability.