Quant Research Guide
Portfolio Optimization
Turning signals into an investable portfolio requires balancing expected return, risk, constraints, and transaction costs simultaneously. Mean-variance optimization is the theoretical foundation, but its practical failures (error maximization, corner solutions) led to a rich toolkit of robust alternatives that every quant researcher must understand.
Core Theory
- 1Mean-variance optimization (Markowitz 1952): maximize μ'w - (λ/2)w'Σw subject to 1'w = 1 (and other constraints). The solution w* = (1/λ)Σ⁻¹μ (unconstrained). The efficient frontier traces all risk-return Pareto optimal portfolios. The problem is a convex QP and is solved exactly with CVXPY.
- 2Estimation error dominates MVO in practice: the inputs μ (expected returns) and Σ (covariance) are estimated from data with large errors. Michaud (1989) called MVO an 'error maximizer' — small errors in expected returns lead to extreme, concentrated portfolios. This is why practitioners use regularization, constraints, and shrinkage.
- 3Ledoit-Wolf shrinkage (2004): instead of the noisy sample covariance S, use Σ̃ = α·F + (1-α)·S where F is a structured target (e.g., diagonal, single-factor, constant correlation) and α is the optimal shrinkage intensity. The Oracle shrinkage minimizes expected Frobenius distance. LW provides a closed-form formula for α. Empirically reduces out-of-sample portfolio variance by 10-30%.
- 4Risk parity / Equal Risk Contribution (ERC): instead of equal weight or MVO, allocate so each asset contributes equal total portfolio variance. ERC condition: wᵢ × (∂σ_p/∂wᵢ) = σ_p/N for all i. There is no closed-form solution — solve numerically. ERC is less sensitive to expected return estimation errors than MVO.
- 5Black-Litterman (1990): combines a market equilibrium prior (CAPM implied returns: μ_eq = δΣw_mkt) with investor views (P·μ = q + ε, ε~N(0,Ω)) using Bayes' theorem. Posterior: μ_BL = [(τΣ)⁻¹ + P'Ω⁻¹P]⁻¹[(τΣ)⁻¹μ_eq + P'Ω⁻¹q]. BL produces more stable, diversified portfolios than using raw α estimates in MVO.
- 6Transaction cost penalty in portfolio optimization: add -λ_TC·‖w - w_prev‖₁ to the MVO objective. This penalizes turnover, leading to a 'no-trade zone' around the current portfolio. The optimal rebalancing frequency balances the cost of tracking the signal against the cost of transacting. Grinold (2006) showed the optimal position is a weighted average of signal and previous position.
- 7Convex constraints via CVXPY: position limits (|wᵢ| ≤ w_max), long-only (wᵢ ≥ 0), sector limits (Σᵢ∈sector wᵢ ≤ s_max), tracking error (‖w - w_bm‖_Σ ≤ TE_max), and beta neutrality (w'β = 0). CVXPY handles all of these natively as convex constraints on a QP.
- 8Factor risk model for portfolio optimization: portfolio variance = w'Σw = w'(XFX' + D)w = ‖F^{1/2}X'w‖² + w'Dw where X are factor exposures, F is factor covariance, D is specific risk diagonal. This decomposition is essential for risk attribution and enables fast computation for large universes (N=500 stocks, K=50 factors).
Key Formulas
| 1 | import numpy as np |
| 2 | import cvxpy as cp |
| 3 | from sklearn.covariance import LedoitWolf |
| 4 | |
| 5 | np.random.seed(42) |
| 6 | N = 10 # assets |
| 7 | T = 252 # daily returns for covariance estimation |
| 8 | |
| 9 | # --- Simulate returns --- |
| 10 | true_cov = np.random.randn(N, N) |
| 11 | true_cov = true_cov @ true_cov.T / N + np.eye(N) * 0.01 |
| 12 | returns = np.random.multivariate_normal(np.zeros(N), true_cov, T) |
| 13 | mu_est = returns.mean(axis=0) * 252 # annualized expected returns |
| 14 | sigma_sample = np.cov(returns.T) # noisy sample covariance |
| 15 | |
| 16 | # --- Ledoit-Wolf shrinkage --- |
| 17 | lw = LedoitWolf().fit(returns) |
| 18 | sigma_lw = lw.covariance_ |
| 19 | |
| 20 | # --- Mean-Variance Optimization via CVXPY --- |
| 21 | def mvo(mu, sigma, lam=2.0, w_max=0.25, long_only=True): |
| 22 | w = cp.Variable(N) |
| 23 | risk = cp.quad_form(w, sigma) |
| 24 | ret = mu @ w |
| 25 | obj = cp.Maximize(ret - lam/2 * risk) |
| 26 | constr = [cp.sum(w) == 1, cp.abs(w) <= w_max] |
| 27 | if long_only: |
| 28 | constr.append(w >= 0) |
| 29 | prob = cp.Problem(obj, constr) |
| 30 | prob.solve(solver=cp.CLARABEL, verbose=False) |
| 31 | return w.value |
| 32 | |
| 33 | w_sample = mvo(mu_est, sigma_sample) |
| 34 | w_lw = mvo(mu_est, sigma_lw) |
| 35 | |
| 36 | # --- Risk Parity (iterative algorithm) --- |
| 37 | def risk_parity(sigma, tol=1e-8, max_iter=1000): |
| 38 | n = sigma.shape[0] |
| 39 | w = np.ones(n) / n |
| 40 | for _ in range(max_iter): |
| 41 | mrc = sigma @ w # marginal risk contribution |
| 42 | rc = w * mrc # risk contribution |
| 43 | target = rc.sum() / n # equal risk target |
| 44 | w_new = w * target / rc # update (multiplicative) |
| 45 | w_new /= w_new.sum() # normalize |
| 46 | if np.max(np.abs(w_new - w)) < tol: |
| 47 | break |
| 48 | w = w_new |
| 49 | return w |
| 50 | |
| 51 | w_rp = risk_parity(sigma_lw) |
| 52 | |
| 53 | # --- Evaluate out-of-sample risk --- |
| 54 | returns_oos = np.random.multivariate_normal(np.zeros(N), true_cov, 252) |
| 55 | def portfolio_vol(w, rets): |
| 56 | pnl = rets @ w |
| 57 | return pnl.std() * np.sqrt(252) |
| 58 | |
| 59 | print(f"{'Portfolio':<20} | {'Ann Vol':>8} | {'Sum w':>7}") |
| 60 | print("-" * 42) |
| 61 | print(f"{'Equal Weight':<20} | {portfolio_vol(np.ones(N)/N, returns_oos):>8.4f} | {1.0:>7.4f}") |
| 62 | print(f"{'MVO (sample cov)':<20} | {portfolio_vol(w_sample, returns_oos):>8.4f} | {w_sample.sum():>7.4f}") |
| 63 | print(f"{'MVO (Ledoit-Wolf)':<20} | {portfolio_vol(w_lw, returns_oos):>8.4f} | {w_lw.sum():>7.4f}") |
| 64 | print(f"{'Risk Parity':<20} | {portfolio_vol(w_rp, returns_oos):>8.4f} | {w_rp.sum():>7.4f}") |
| 65 | |
| 66 | # --- Verify ERC condition --- |
| 67 | mrc = sigma_lw @ w_rp |
| 68 | rc = w_rp * mrc |
| 69 | print(f"\nRisk contributions (should be equal): {rc/rc.sum()}") |
Implements the three main portfolio construction approaches: (1) MVO with CVXPY using both sample and Ledoit-Wolf covariance — demonstrating that LW reduces out-of-sample volatility; (2) Risk Parity via the iterative multiplicative updates algorithm; (3) verification of the ERC condition. Compares all portfolios against equal-weight as a baseline.
Worked Interview Problems
3 problemsProblem
You have 5 assets. Asset A has estimated return 8% and true return 6%. Asset B has estimated return 5% and true return 5%. MVO heavily overweights A. Explain why and how shrinkage fixes it.
Solution
MVO solution: w* = (1/λ)Σ⁻¹μ. Σ⁻¹ amplifies differences in μ — a 2% overestimate in asset A's return gets multiplied by the precision matrix entries, potentially creating a 20%+ overweight in A.
The error in μ̂ for A (overestimated by 2%) is well within estimation uncertainty for any realistic sample size. With 60 monthly returns, SE of mean return ≈ σ/√60 ≈ 5%/7.75 ≈ 0.65%/month ≈ 7.7%/year — larger than the 2% error.
MVO treats μ̂ as if it were true μ with no uncertainty. It maximizes the objective using potentially wrong inputs — hence 'error maximizer' (Michaud 1989).
Shrinkage fix: use a prior on μ — for example, shrink toward the cross-sectional mean return. This reduces the variance of μ̂ at the cost of introducing bias, improving out-of-sample portfolio performance.
Practical fix: use Black-Litterman instead of raw μ̂. BL combines the CAPM equilibrium returns (a prior with low estimation error) with your views (α signals), weighted by their respective confidence. The result is a more stable, diversified portfolio.
Answer
MVO amplifies errors in expected return estimates because it takes a precision-matrix-weighted combination of μ̂. Fix: Ledoit-Wolf for Σ, Black-Litterman for μ, or add maximum position constraints to limit concentration.
Common Mistakes
Using the sample covariance matrix directly in MVO for large universes (N > 100 stocks). For N=500, the sample covariance has N×(N+1)/2 = 125,250 parameters estimated from maybe 252 data points. It is severely noisy. Always use Ledoit-Wolf or another shrinkage estimator.
Forgetting to add transaction cost penalties when rebalancing. Optimizing to the signal-optimal portfolio each period ignores that getting there requires trading. The cost-aware optimal portfolio is between the current portfolio and the signal-optimal one.
Solving MVO with raw expected return estimates (μ̂ from historical means) without any shrinkage or equilibrium prior. Historical mean returns are extremely noisy. Use Black-Litterman equilibrium returns as the baseline and add views carefully.
Confusing risk parity (equal risk contribution) with equal volatility weighting. Equal vol weighting sets position sizes as 1/σᵢ (inversely proportional to individual asset vol) ignoring correlations. ERC properly accounts for covariance structure — they are equivalent only if all pairwise correlations are equal.
Treating the CVXPY solution as exact when the covariance matrix is near-singular. Add a small regularization term: σ_reg = Σ + εI where ε ≈ 0.0001. This ensures the matrix is positive definite and the QP has a unique solution.