SpectralContrastiveLoss

class kooplearn.torch.nn.SpectralContrastiveLoss

Bases: Module

Spectral contrastive loss based originally introduced by HaoChen et al. [1], and adopted for evolution operators in Turri et al. [2], Jeong et al. [3]

\[\mathcal{L}(x, y) = \frac{1}{N(N-1)}\sum_{i \neq j}\langle x_{i}, y_{j} \rangle^2 - \frac{2}{N}\sum_{i=1}\langle x_{i}, y_{i} \rangle.\]

Hint

Check out the Ordered MNIST example for a practical use of this loss function.

Methods

forward(x: Tensor, y: Tensor) → Tensor

Forward pass of the L2 contrastive loss.

Parameters:

• x (Tensor) – Input features.

• y (Tensor) – Output features.

Shape:

x: \((N, D)\), where \(N\) is the batch size and \(D\) is the number of features.

y: \((N, D)\), where \(N\) is the batch size and \(D\) is the number of features.

[1]

Jeff Z HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma. Provable guarantees for self-supervised deep learning with spectral contrastive loss. Advances in neural information processing systems, 34:5000–5011, 2021.

[2]

Giacomo Turri, Luigi Bonati, Kai Zhu, Massimiliano Pontil, and Pietro Novelli. Self-supervised evolution operator learning for high-dimensional dynamical systems. arXiv preprint arXiv:2505.18671, 2025.

[3]

Minchan Jeong, J Jon Ryu, Se-Young Yun, and Gregory W Wornell. Efficient parametric svd of koopman operator for stochastic dynamical systems. arXiv preprint arXiv:2507.07222, 2025.