Synthesizing Minority Samples: A Formal Analysis of Linear Interpolation in Imbalanced Classification

In supervised learning, class imbalance presents a significant topological challenge, often biasing classifiers toward the majority distribution. The Synthetic Minority Over-sampling Technique (SMOTE) addresses this by transcending simple replication (Random Oversampling) and instead performing linear interpolation in the feature space. This paper formalizes the mechanics of SMOTE and its impact on the structural geometry of minority class clusters.

---

1. Mathematical Formalism of Synthetic Generation

The SMOTE algorithm operates by augmenting the minority class through the generation of synthetic samples along the line segments connecting existing minority samples and their $k$-nearest neighbors.

Step 1: Neighborhood Selection

For each minority sample $x_i$, the algorithm identifies a set of $k$ nearest neighbors $\{x_{z_1}, x_{z_2}, \dots, x_{z_k}\}$ within the same minority class. Proximity is typically determined using the Euclidean distance metric: $d(x_i, x_{zi}) = \sqrt{\sum_{j=1}^{n} (x_{ij} - x_{zij})^2}$

Step 2: Linear Interpolation

A specific neighbor $x_{zi}$ is randomly selected from the $k$-nearest set. The synthetic sample $x_{new}$ is then computed by interpolating between the anchor point $x_i$ and the selected neighbor $x_{zi}$: $x_{new} = x_i + \lambda \cdot (x_{zi} - x_i)$

Where $\lambda$ (the "gap") is a random scalar sampled from a uniform distribution: $\lambda \sim U(0, 1)$

This operation ensures that the vector difference $(x_{zi} - x_i)$ is scaled by a random factor, effectively placing the new sample $x_{new}$ at a stochastic position on the segment $\overline{x_i x_{zi}}$.

---

2. Geometric Interpretation: Decision Boundary Expansion

The fundamental advantage of SMOTE over Random Oversampling lies in its ability to increase the topological diversity of the minority class.

Feature	Random Oversampling	SMOTE
Mechanism	Exact duplication of minority samples.	Linear interpolation between neighbors.
Topological Effect	Increases density at existing locales (high risk of overfitting).	Expands the convex hull and decision boundaries.
Statistical Diversity	Zero; maintains original variance.	High; introduces new, plausible observations.

By populating the space between existing samples, SMOTE forces the classifier to learn a larger, more generalized region for the minority class, rather than memorizing individual points.

---

3. Numerical Validation: Feature Space Mapping

Consider a network traffic analysis scenario where we evaluate Packet Size ($f_1$) and Duration ($f_2$).

• Anchor Point ($P_i$): $\begin{bmatrix} 120 \\ 0.4 \end{bmatrix}$
• Selected Neighbor ($N_1$): $\begin{bmatrix} 130 \\ 0.5 \end{bmatrix}$

The Difference Vector is calculated as: $\vec{V} = N_1 - P_i = \begin{bmatrix} 10 \\ 0.1 \end{bmatrix}$

Applying a randomly sampled $\lambda = 0.7$: $P_{new} = \begin{bmatrix} 120 \\ 0.4 \end{bmatrix} + 0.7 \cdot \begin{bmatrix} 10 \\ 0.1 \end{bmatrix} = \begin{bmatrix} 127 \\ 0.47 \end{bmatrix}$

The resultant synthetic sample $P_{new}$ maintains the structural characteristics of the minority class while introducing numerical variance necessary for robust model convergence.

---

4. Topological Hazards and Performance Constraints

Despite its utility, SMOTE is sensitive to the underlying data distribution and can introduce noise if not properly calibrated:

1. Overgeneralization (Over-smoothing): If $k$ is too large, SMOTE may interpolate between samples that are far apart, potentially spanning across majority class regions and creating "spy samples" (noise).
2. Sensitivity to Outliers: If an anchor point is a minority outlier (noise), SMOTE will generate synthetic points between that outlier and its neighbors, creating a bridge of noise through the majority class.
3. Categorical Constraints: In its vanilla form, SMOTE is unsuitable for discrete features. For datasets containing categorical variables, variants such as SMOTE-NC must be employed to handle non-continuous manifolds.

Advanced Variants

To mitigate these risks, architectures like Borderline-SMOTE focus exclusively on samples located at the edge of the decision boundary, where the risk of misclassification is highest, thereby optimizing the sampling efficiency.

---

5. Architectural Conclusion

SMOTE represents a formal shift from data replication to data synthesis. By leveraging the geometric properties of the feature space, it provides a rigorous framework for balancing datasets. When integrated into complex pipelines—such as CNN-LSTM architectures for temporal traffic analysis—SMOTE ensures that the model learns the underlying patterns of rare events without sacrificing performance on the majority class.

[!CAUTION] Implementation Warning: Practitioners must apply SMOTE only to the training partition. Applying oversampling to validation or test sets leads to "data leakage" and artificially inflated performance metrics that do not reflect real-world generalization.