In a recent paper1, we have been exploring the role of nonlinear feature maps in learning from few examples. Specifically, suppose that we have some previously trained classifier $F : \mathbb{R}^n \to \mathcal{L}$ (where $\mathcal{L}$ is some set of data labels), which works well on an existing dataset. However, we now wish to update this classifier to also be able to classify data from some new data class, for which we have very few available examples. Rather than retraining the whole system with this extra data (and risking losing any existing behaviour and overfitting to the few available new data points), we just want to be able to cheaply ‘update’ our existing classifier.

A simple way of achieving this is to use a binary linear classifier in some appropriate feature space: if we decide that a sample belongs to the old classes then we use the old classifier, otherwise we just assign the new label. This approach has also been studied elsewhere2 3, and indeed in3 it was shown that for high dimensional data, this approach can be extremely effective.

[…then the story continues…]

References

  1. O. J. Sutton, A. N. Gorban, and I. Y. Tyukin (2022). Towards a mathematical understanding of learning from few examples with nonlinear feature maps, arXiv, 2211.03607. ↩

  2. A. N. Gorban, B. Grechuk, E. M. Mirkes, S. Stasenko V, and I. Y. Tyukin (2021). High-Dimensional Separability for One- and Few-Shot Learning, Entropy, 23(8). ↩

  3. I. Y. Tyukin, A. N. Gorban, M. H. Alkhudaydi, and Q. Zhou (2021). Demystification of Few-shot and One-shot Learning, 2021 International Joint Conference On Neural Networks (IJCNN). ↩ ↩2