Skip to content

Correntropy RKHS and space of expectations


Correntropy is a statistical quantity that captures nonlinear similarity between an indexed set of random variables, defined as V(i, j) = E[\kappa(X_i, X_j)], where X_i are the random variables, and \kappa(\cdot, \cdot) is a symmetric positive-semi-definite kernel [1]. Due to the reproducing kernel Hilbert space (RKHS) theory, any symmetric positive semi-definite kernel induces an RKHS, that is it can be used as an inner product in an extended space [2]. It is easy to show that correntropy is symmetric positive-definite, and hence it defines the so-called correntropy RKHS.

Each element in the correntropy RKHS is a functional that maps from an index of a random variable to a real value. In general, due to representer theorem, any functional in the space can be represented as, g(j) = \sum_i \alpha_i E[\kappa(X_i, X_j)], a linear combination of the mapped random variable vectors. When the random variables that constitute the space is rich enough, then the one can approximate expectations by estimating \alpha_i‘s. However, unfortunately, this has little practical value, because the richness has to come from the joint of pairs of random variables. It is an interesting subject to ponder.

Note: correntropy RKHS should not be confused with the correntropy space which is an analogue of Parzen’s random process where the covariance is equivalent to an arbitrary Gaussian process.

  1. I. Santamaria, P. Pokharel, J. C. Príncipe. Generalized Correlation Function: Definition, Properties, and Application to Blind Equalization. IEEE Transactions on Signal Processing, Vol. 54, No. 6. (2006), pp. 2187-2197.
  2. Nachman Aronszajn. Theory of Reproducing Kernels. Transactions of the American Mathematical Society, Vol. 68, No. 3. (1950), pp. 337-404.
No comments yet

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: