Home

Anonymous Steven Van Vaerenbergh

The Kernel Methods Toolbox (KMBOX) is a collection of MATLAB programs that implement kernel-based algorithms, with a focus on regression algorithms and online algorithms. It can be used for nonlinear signal processing and machine learning.

KMBOX includes implementations of algorithms such as kernel principal component analysis (KPCA), kernel canonical correlation analysis (KCCA) and kernel recursive least-squares (KRLS).

The goal of this distribution is to provide easy-to-analyze algorithm implementations, which reveal the inner mechanics of each algorithm and allow for quick modifications. Therefore, the focus of these implementations is on readability rather than speed or memory usage.

The current version is v0.9, updated on May 21st, 2013.

Included algorithms:

  1. Kernel Ridge Regression (KRR).
  2. Principal Component Analysis (PCA).
  3. Kernel Principal Component Analysis (KPCA), as proposed in B. Scholkopf, A. Smola and K.R. Muller, "Nonlinear component analysis as a kernel eigenvalue problem", Neural Computation, volume 10, no. 5, pages 1299-1319, 1998.
  4. Approximate Linear Dependency Kernel Recursive Least-Squares (ALD-KRLS), as proposed in Y. Engel, S. Mannor, and R. Meir. "The kernel recursive least-squares algorithm", IEEE Transactions on Signal Processing, volume 52, no. 8, pages 2275–2285, 2004.
  5. Sliding-Window Kernel Recursive Least-Squares (SW-KRLS), as proposed in S. Van Vaerenbergh, J. Via, and I. Santamaria. "A sliding-window kernel RLS algorithm and its application to nonlinear channel identification", 2006 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Toulouse, France, 2006.
  6. Naive Online R_reg Minimization Algorithm (NORMA), as proposed in J. Kivinen, A. Smola and C. Williamson. "Online Learning with Kernels", IEEE Transactions on Signal Processing, volume 52, no. 8, pages 2165-2176, 2004.
  7. Fixed-Budget Kernel Recursive Least-Squares (FB-KRLS), as proposed in S. Van Vaerenbergh, I. Santamaria, W. Liu and J. C. Principe, "Fixed-Budget Kernel Recursive Least-Squares", 2010 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2010), Dallas, Texas, U.S.A., March 2010.
  8. Incomplete Cholesky Decomposition (ICD), as proposed in Francis R. Bach and Michael I. Jordan. "Kernel Independent Component Analysis", Journal of Machine Learning Research, volume 3, pages 1-48, 2002.
  9. Kernel Recursive Least-Squares Tracker (KRLS-T), as proposed in M. Lazaro-Gredilla, S. Van Vaerenbergh and I. Santamaria, "A Bayesian Approach to Tracking with Kernel Recursive Least-Squares", 2011 IEEE International Workshop on Machine Learning for Signal Processing (MLSP 2011), Beijing, China, September, 2011.
  10. Kernel Canonical Correlation Analysis (KCCA), as proposed in D. R. Hardoon, S. Szedmak and J. Shawe-Taylor, "Canonical Correlation Analysis: An Overview with Application to Learning Methods", Neural Computation, Volume 16 (12), Pages 2639--2664, 2004.
  11. Quantized Kernel Least Mean Squares (QKLMS), as proposed in Chen B., Zhao S., Zhu P., Principe J.C. " Kernel Least Mean Square Algorithm," IEEE Transactions on Neural Networks and Learning Systems, vol.23, no.1, Jan. 2012, pages 22-32.
  12. Alternating Kernel Canonical Correlation Analysis for blind equalization of single-input multiple-output Wiener systems, as proposed in S. Van Vaerenbergh, J. Via and I. Santamaria, "Blind Identification of SIMO Wiener Systems based on Kernel Canonical Correlation Analysis", accepted for publication in IEEE Transactions on Signal Processing, 2013.
  13. Kernel density estimation (KDE).