Quant Research Guide
Machine Learning for Financial Prediction
Applying ML to financial data requires completely different instincts than applying it to images or text. The signal-to-noise ratio is ~100ร lower, the data is non-stationary, and standard cross-validation leaks future information. Master these failure modes before writing a single model.
Core Theory
- 1Low signal-to-noise ratio: financial IC โ 0.02โ0.08. For comparison, image classification accuracy โ 99%. A model with 50.5% directional accuracy on next-day returns is exceptional in finance โ the same accuracy in any other domain would be considered a failure. Overfitting is catastrophic because the true signal is so small.
- 2Non-stationarity: financial data distributions change over time as regimes shift (QE era vs rate hike era, pre vs post-GFC). A model trained on 2010-2019 may fail completely in 2020-2023. Standard train/test splits that assume stationarity are inappropriate. Always use walk-forward or time-series cross-validation.
- 3Purged K-Fold cross-validation (Lopez de Prado 2018): in standard K-Fold, a test sample at time t may have training samples within h periods that share information (overlap in features). Purging removes training samples within h periods of each test sample. The embargo adds a further buffer of e periods after the test window. This is the correct CV for any time-series ML.
- 4SHAP (SHapley Additive exPlanations): assigns each feature a contribution to the prediction for each sample. ฯแตข(x) is the Shapley value for feature i โ it satisfies efficiency (sum = prediction - baseline), symmetry, and null player axioms. More reliable than split-based feature importance in tree models, which is biased toward high-cardinality features.
- 5XGBoost for cross-sectional returns: gradient boosted trees are state-of-the-art for tabular financial data. Key hyperparameters: max_depth (3-6 for finance, resist deep trees), learning_rate (0.01-0.05), n_estimators (300-1000 with early stopping), subsample (0.7-0.9), colsample_bytree (0.7-0.9). Regularization via L1/L2 on leaf weights (reg_alpha, reg_lambda).
- 6Feature engineering for cross-sectional ML: momentum features (1m, 3m, 6m, 12m returns), volatility normalization (divide returns by rolling std), sector neutralization (subtract sector mean), quality features (profitability ratios, accruals), volume features (dollar volume z-score, turnover ratio). Always cross-sectionally z-score continuous features before feeding to tree models.
- 7Model stacking and ensemble: train multiple base models (XGBoost, LightGBM, Ridge regression) on different training windows or feature subsets. Meta-learner (Ridge) combines base model predictions. Stacking improves OOS IC by 10-20% vs single model โ uncorrelated errors cancel out.
- 8Multiple comparison bias in ML: if you tune 100 hyperparameter configurations on the same validation set, the best configuration has inflated OOS expectations. Use a true holdout set never touched during tuning. Report OOS performance on the holdout, not the cross-validation score.
Key Formulas
| 1 | import numpy as np |
| 2 | import pandas as pd |
| 3 | from sklearn.base import BaseEstimator |
| 4 | from sklearn.model_selection import KFold |
| 5 | |
| 6 | np.random.seed(42) |
| 7 | T, N = 60, 200 # 60 months, 200 stocks per month |
| 8 | |
| 9 | # --- Build panel: (T*N) rows, features + target --- |
| 10 | rows = [] |
| 11 | for t in range(T): |
| 12 | for i in range(N): |
| 13 | rows.append({ |
| 14 | 'date': t, |
| 15 | 'stock': i, |
| 16 | 'mom_1m': np.random.randn(), # 1-month momentum |
| 17 | 'mom_12m': np.random.randn(), # 12-month momentum |
| 18 | 'vol_ratio': np.abs(np.random.randn()), # vol ratio |
| 19 | 'bp_ratio': np.random.randn(), # book/price |
| 20 | 'target': np.random.randn() * 0.05, # next-month return |
| 21 | }) |
| 22 | |
| 23 | df = pd.DataFrame(rows) |
| 24 | features = ['mom_1m', 'mom_12m', 'vol_ratio', 'bp_ratio'] |
| 25 | |
| 26 | # --- Purged K-Fold (simplified) --- |
| 27 | class PurgedKFold: |
| 28 | """Time-series CV that removes samples near test boundary.""" |
| 29 | def __init__(self, n_splits=5, purge_periods=2): |
| 30 | self.n_splits = n_splits |
| 31 | self.purge_periods = purge_periods |
| 32 | |
| 33 | def split(self, dates): |
| 34 | unique_dates = np.sort(np.unique(dates)) |
| 35 | n = len(unique_dates) |
| 36 | fold_size = n // self.n_splits |
| 37 | for k in range(self.n_splits): |
| 38 | test_start = k * fold_size |
| 39 | test_end = (k + 1) * fold_size |
| 40 | test_dates = set(unique_dates[test_start:test_end]) |
| 41 | purge_dates = set(unique_dates[max(0, test_start - self.purge_periods):test_start]) |
| 42 | train_dates = set(unique_dates) - test_dates - purge_dates |
| 43 | train_idx = np.where(np.isin(dates, list(train_dates)))[0] |
| 44 | test_idx = np.where(np.isin(dates, list(test_dates)))[0] |
| 45 | yield train_idx, test_idx |
| 46 | |
| 47 | # --- Cross-sectional z-score features (no look-ahead) --- |
| 48 | for feat in features: |
| 49 | df[feat + '_z'] = df.groupby('date')[feat].transform( |
| 50 | lambda x: (x - x.mean()) / (x.std() + 1e-8) |
| 51 | ) |
| 52 | |
| 53 | feat_z = [f + '_z' for f in features] |
| 54 | X = df[feat_z].values |
| 55 | y = df['target'].values |
| 56 | dates = df['date'].values |
| 57 | |
| 58 | # --- Purged K-Fold CV --- |
| 59 | from sklearn.linear_model import Ridge # use Ridge as simple stand-in for XGBoost |
| 60 | pkf = PurgedKFold(n_splits=5, purge_periods=2) |
| 61 | oos_ics = [] |
| 62 | |
| 63 | for train_idx, test_idx in pkf.split(dates): |
| 64 | model = Ridge(alpha=1.0) |
| 65 | model.fit(X[train_idx], y[train_idx]) |
| 66 | preds = model.predict(X[test_idx]) |
| 67 | # IC: Spearman correlation on each test date separately |
| 68 | test_df = pd.DataFrame({'pred': preds, 'ret': y[test_idx], 'date': dates[test_idx]}) |
| 69 | ic = test_df.groupby('date').apply( |
| 70 | lambda g: g['pred'].corr(g['ret'], method='spearman') |
| 71 | ).mean() |
| 72 | oos_ics.append(ic) |
| 73 | |
| 74 | print(f"OOS IC per fold: {[f'{x:.4f}' for x in oos_ics]}") |
| 75 | print(f"Mean OOS IC: {np.mean(oos_ics):.4f}") |
| 76 | print(f"OOS ICIR: {np.mean(oos_ics)/np.std(oos_ics):.4f}") |
| 77 | print("\nNote: IC near 0 expected โ features are random noise in this simulation") |
| 78 | print("Real financial features typically yield IC = 0.02-0.06 after costs") |
Implements the full ML-in-finance workflow: cross-sectional z-scoring of features (within each date, no look-ahead), a simplified Purged K-Fold that removes samples near test boundaries, and computing OOS IC per fold rather than accuracy โ because IC is the right metric for financial prediction, not classification accuracy.
Worked Interview Problems
3 problemsProblem
You have 60 months of stock data. You run standard 5-fold K-Fold CV on a momentum signal. Fold 3 train set contains months 1-24 and 37-60; test set is months 25-36. What is the information leakage?
Solution
The momentum signal for test month 25 is computed using returns from months 13-24 (12-month lookback). Months 13-24 also appear in the training set.
The model trained on months 37-60 has 'seen' return patterns in 37-60. But months 37-60 come AFTER the test period 25-36, so the model has look-ahead into the future relative to the test period.
This violates the fundamental time-series constraint: training data must be strictly before test data. Standard K-Fold randomly shuffles folds without respecting time order.
Fix 1 โ Time-series split: train on months 1-K, test on months K+1 to K+12, slide forward. This preserves time order but doesn't purge overlapping features.
Fix 2 โ Purged K-Fold: additionally remove training samples within h=12 periods of the test start boundary (because their features overlap with the test features). Add an embargo of e=1-2 periods after test end.
Answer
Standard K-Fold leaks in two ways: (1) training samples after the test period (future information), (2) training samples just before the test with overlapping feature windows. Purged K-Fold fixes both by removing contaminated training samples and adding post-test embargo.
Common Mistakes
Using standard K-Fold or even TimeSeriesSplit without purging. TimeSeriesSplit respects time order but doesn't remove the overlapping feature window near the test boundary. Always use PurgedKFold for financial ML.
Feature selection or hyperparameter tuning using the full dataset before CV splits. This leaks target information into both training and validation folds, inflating CV metrics dramatically.
Using accuracy or AUC-ROC as the primary metric for financial return prediction. The correct metric is OOS IC (Spearman rank correlation between predicted and realized returns). Accuracy measures directional correctness but ignores magnitude; IC captures the full rank ordering.
Treating XGBoost feature importance (split counts or gain) as reliable. Gain-based importance is biased toward high-cardinality continuous features. SHAP values are the correct, consistent measure of feature contribution.
Not performing cross-sectional z-scoring of features before training. Without normalization, the model may learn that a feature value of 0.5 is 'high' in some periods and 'low' in others due to distributional drift. Always normalize features within each cross-section (date) before feeding to the model.