Skip to content

Hellinger divergence between two homogeneous Poisson processes

2009/07/14

Hellinger divergence/distance/metric/dissimilarity measures how different two probability measures are. I am interested in comparing spike train observations, hence I apply the Hellinger divergence to probability measures over all possible spike trains. The squared Hellinger divergence is defined as D^2(P,Q) = \int \left(\sqrt{P} - \sqrt{Q}\right)^2 where P and Q are probability measures.

In case of  the simplest point process, homogeneous Poisson process, the spike train space as well as the probability measure can be partitioned according to the number of events (action potentials) in each spike train. Since, the location statistic (probability given the total number of events) is exactly same for all homogeneous Poisson process, the integral in Hellinger divergence can be simply written as the following summation,

D^2(P,Q) = \sum_{n=0}^\infty \left( \sqrt{p_n} - \sqrt{q_n} \right)^2 = 2 - 2 \sum_{n=0}^\infty \sqrt{p_n q_n}

where p_n and q_n are the probability that n events occur. Let \Lambda_P and \Lambda_Q be the corresponding mean number of events, then p_n = \frac{1}{n!} \Lambda_P^n \exp(-\Lambda_P) and similarly for q_n; they follow the Poisson distribution. Substituting and rearranging the terms, we obtain,

D^2(P,Q) = 2 - 2 \exp\left(-\frac{1}{2} \left(\sqrt{\Lambda_P}-\sqrt{\Lambda_Q}\right)^2\right).

This is essentially same as just the divergence between two Poisson distributions. Let’s plot how it looks (using Matlab).

The divergence between $latex \Lambda = 0,4,8,12$ and other rates between 0 to 16

Squared Hellinger divergence as we defined it earlier ranges from 0 to 2. From the figure above, we can see that it is zero when the rates coincide and tends to 2 as the difference increases. I recently proposed a nonparameteric estimator for this measure, and it is currently under review. I would like to empirically verify the asymptotic consistency of the estimator with this result.

Note that previously proposed mCI based distance between point processes [Paiva et. al. 2009], related to van Rossum’s distance of spike trains, has a quadratic form (\Lambda_P - \Lambda_Q)^2. It only applies to Poisson process when no realization is given, which is a major limitation. The Hellinger divergence can be applied to arbitrary point processes.

Advertisements
2 Comments leave one →
  1. memming permalink*
    2010/05/23 3:10 pm

    Around the point where P = Q, most divergences show quadratic behavior. Therefore for optimization purposes any divergence will do fine, near the optima. The question is how they behave when they are not close to each other.

Trackbacks

  1. Point process spaces « Memming

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: