mlsynth: A Practical Guide2025-11-11
Find the data here.
| state | year | cigsale | Proposition 99 |
|---|---|---|---|
| Alabama | 1970 | 89.8 | False |
| Alabama | 1971 | 95.4 | False |
| Alabama | 1972 | 101.1 | False |
| Alabama | 1973 | 102.9 | False |
| Alabama | 1974 | 108.2 | False |
| state | year | cigsale | Proposition 99 |
|---|---|---|---|
| Wyoming | 1996 | 110.3 | False |
| Wyoming | 1997 | 108.8 | False |
| Wyoming | 1998 | 102.9 | False |
| Wyoming | 1999 | 104.8 | False |
| Wyoming | 2000 | 90.5 | False |
import pandas as pd
from mlsynth.utils.helperutils import sc_diagplot
url = "https://raw.githubusercontent.com/jgreathouse9/mlsynth/main/basedata/smoking_data.csv"
df = pd.read_csv(url)
print(df.head().to_markdown(index=False))
print(df.tail().to_markdown(index=False))
# Configure the SC_DIAGPLOT method
plotconfig = {
"df": df,
"outcome": df.columns[2],
"treat": df.columns[-1],
"unitid": df.columns[0],
"time": df.columns[1],
"display_graphs": True,
"save": False,
"counterfactual_color": "red"
}
sc_diagplot([plotconfig])| state | year | cigsale | Proposition 99 |
|:--------|-------:|----------:|:-----------------|
| Alabama | 1970 | 89.8 | False |
| Alabama | 1971 | 95.4 | False |
| Alabama | 1972 | 101.1 | False |
| Alabama | 1973 | 102.9 | False |
| Alabama | 1974 | 108.2 | False |
| state | year | cigsale | Proposition 99 |
|:--------|-------:|----------:|:-----------------|
| Wyoming | 1996 | 110.3 | False |
| Wyoming | 1997 | 108.8 | False |
| Wyoming | 1998 | 102.9 | False |
| Wyoming | 1999 | 104.8 | False |
| Wyoming | 2000 | 90.5 | False |
Parallel trends does not appear to hold for California with respect to its donor pool. California has a steeper downward sloping trend of cigarette consumption compared to the average of the donor pool. No matter what scalar constant we would shift the mean of the donor pool by, the mean difference would not be constant with respect to California. This suggests one of two solutions is viable: either parallel trends would hold with some control units, or we simply need to re-weight the donor pool to better match the pre-intervention trajectory.
Here is the results of the CLUSTERSC class. This class implements the Robust Synthetic Control Method. We can see that this model fits the pre-treatment trajectory very well, without needing to use any of the seven covariates that the original paper uses.
import pandas as pd
from mlsynth import CLUSTERSC
stub = "https://raw.githubusercontent.com/jgreathouse9/mlsynth/"
trail = "refs/heads/main/basedata/"
file = "smoking_data.csv"
data = pd.read_csv(stub + trail + file)
SCconfig = {
"df": data,
"outcome": data.columns[2],
"treat": data.columns[-1],
"unitid": data.columns[0],
"time": data.columns[1],
"display_graphs": True,
"save": False,
"counterfactual_color": ["blue"], "method": "PCR", "cluster": False
}
SCResult = CLUSTERSC(SCconfig).fit()
Here, the weights are allowed to be negative and there is no summation constraint: PCA is used to denoise the donor pool, we select the number of singular values via a singular value thresholding, and we reconstruct the donor matrix to learn the weights via OLS. We may also use the Bayesian version of this estimator if we set Frequentist to False
FDID is an iterative, data-driven approach that selects the control group for a treated unit.
By design, it is agnostic to which units to include or how many controls to use. The algorithm adds controls sequentially based on pre-treatment fit.
\(\widehat{U}_0 = \emptyset \qquad \mathcal{U} = \emptyset\)
For each candidate control \(i \in \mathcal{N}_0\):
\[ \underset{\beta_{\widehat{U}_0 \cup \{i\}}}{\operatorname*{argmin}} \big\| \mathbf{y}_1 - \mathbf{Y}_{\widehat{U}_0 \cup \{i\}} \beta_{\widehat{U}_0 \cup \{i\}} - \beta_\text{cons}\mathbf{1} \big\|_2^2, \quad \beta_{\widehat{U}_0 \cup \{i\}} = \frac{1}{|\widehat{U}_0| + 1} \]
Select the control with highest \(R^2\):
\[ i_1^\ast = \underset{i \in \mathcal{N}_0}{\operatorname*{argmax}} R^2_i, \quad \widehat{U}_1 = \{ i_1^\ast \}, \quad \mathcal{U} = \{\widehat{U}_1\}. \]
For each remaining candidate \(i \notin \widehat{U}_{k-1}\):
\[ i_k^\ast = \underset{i \in \mathcal{N}_0 \setminus \widehat{U}_{k-1}}{\operatorname*{argmax}} R^2_{\widehat{U}_{k-1} \cup \{i\}}, \quad \widehat{U}_k = \widehat{U}_{k-1} \cup \{ i_k^\ast \}, \quad \mathcal{U} = \mathcal{U} \cup \{ \widehat{U}_k \}. \]
\[ \widehat{U} := \underset{\widehat{U}_k \in \mathcal{U}}{\operatorname*{argmax}} R^2(\widehat{U}_k) \]
The selected control set \(\widehat{U}\) maximizes pre-treatment fit (\(R^2\)) among all candidate sets.
import pandas as pd
from mlsynth import FDID
url = "https://raw.githubusercontent.com/jgreathouse9/mlsynth/refs/heads/main/basedata/smoking_data.csv"
data = pd.read_csv(url)
FDIDconfig = {
"df": data,
"outcome": data.columns[2],
"treat": data.columns[-1],
"unitid": data.columns[0],
"time": data.columns[1],
"display_graphs": True,
"save": False,
"counterfactual_color": ["blue", "red"]
}
FDIDResult = FDID(FDIDconfig).fit()
Here all selected units are given equal weight, adjusted only by an intercept.
Basque Treated in 1975 with the other 16 Spanish communities under control.
#| fig-align: center
import pandas as pd
from mlsynth.utils.helperutils import sc_diagplot
stub = "https://raw.githubusercontent.com/jgreathouse9/mlsynth/"
trail = "refs/heads/main/basedata/"
file = "basque_data.csv"
df = pd.read_csv(stub + trail + file)
plotconfig = {
"df": df,
"outcome": df.columns[2],
"treat": df.columns[-1],
"unitid": df.columns[0],
"time": df.columns[1],
"display_graphs": True,
"save": False,
"counterfactual_color": "red"
}
sc_diagplot([plotconfig])
A feature of Classic SCM is that the weights are often very sparse, which may overfit to certain donor units and produce unstable results.
Relaxed Balanced SCM introduces two modifications:
Let
\[ \mathbf{G} \operatorname{:=} \mathbf{Y}_0^\top \mathbf{Y}_0 \in \mathbb{R}^{N_0 \times N_0}, \quad \mathbf{g} \operatorname{:=} \mathbf{Y}_0^\top \mathbf{y}_1 \in \mathbb{R}^{N_0}. \]
\[ \begin{aligned} \mathbf{w}^{\mathrm{RBSCM}} &= \underset{\mathbf{w} \in \mathcal{W}_{\mathrm{conv}}}{\operatorname*{argmin}} \; \|\mathbf{w}\|_2^2, \\ \text{s.t.} \quad & \big\| \mathbf{G}\mathbf{w} - \mathbf{g} \big\|_\infty \le \tau, \end{aligned} \]
where
\[ \mathcal{W}_{\mathrm{conv}} \operatorname{:=} \Big\{ \mathbf{w} \in (\{0\} \cup \mathbb{R}_+)^{N_0} : \sum_{j=1}^{N_0} w_j = 1 \Big\}. \]
\(\tau \ge 0\) controls the allowed relaxation of the pre-treatment fit.
The \(\ell_2\)-norm penalty encourages dense weights, distributing contribution across multiple donor units instead of concentrating on a few.
import pandas as pd
from mlsynth import RESCM
stub = "https://raw.githubusercontent.com/jgreathouse9/mlsynth/"
trail = "refs/heads/main/basedata/"
file = "basque_data.csv"
data = pd.read_csv(stub + trail + file)
Relconfig = {
"df": data,
"outcome": data.columns[2],
"treat": data.columns[-1],
"unitid": data.columns[0],
"time": data.columns[1],
"display_graphs": True,
"save": False,
"counterfactual_color": ["blue", "red"]
}
RelResult = RESCM(Relconfig).fit()
The ATT for ADH-SCM and the Relaxed Balanced SC are very similar, as is their pretreatment fits. However we can see that they select very different unit weights. The relaxed SC, with \(\tau\) chosen by cross validation, diversifies the prediction risk.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from mlsynth import FDID
stub = "https://raw.githubusercontent.com/jgreathouse9/mlsynth/"
trail = "refs/heads/main/basedata/"
file = "basque_data.csv"
data = pd.read_csv(stub + trail + file)
FDIDconfig = {
"df": data,
"outcome": data.columns[2],
"treat": data.columns[-1],
"unitid": data.columns[0],
"time": data.columns[1],
"display_graphs": True,
"save": False,
"counterfactual_color": ["blue", "red"]
}
FDIDResult = FDID(FDIDconfig).fit()
Aragon and Catalonia (weighted at 0.5), counterfactual similar to the relaxed SCM.
