HGR

HGR stands for Hirschfeld–Gebelein–Rényi maximum correlation. It is a correlation metric in statistics. Compared with commonly-used Pearson's correlation coefficient, it has the advantage to handle non-linear statistical dependency. The benefit for such generalization comes with computational cost. While correlation coefficient can be computed by definition, HGR maximum correlation should be approximated by ACE ( Alternating Conditional Expectations) algorithm. The application of HGR and its extension has been found in the field of information theory, machine learning and so on.

History

HGR maximum correlation is independently proposed by three mathematicians in 20th century. Hans proposed that the correlation can be computed by series expansion, which is not efficient and replaced by ACE latterly.

Mathematical Description

Let X and Y be random variables, f and g be smooth transformation of X, Y respectively, HGR is defined as

$\rho =\mathop {\max _{\mathbb {E} [f(X)]=0,\mathbb {E} [g(Y)]=0}} _{{\textrm {Var}}[f(X)]=1,{\textrm {Var}}[g(Y)]=1}\mathbb {E} [f(X)g(Y)]$

Applications

Information Theory

The connection between HGR and other information-theoretic metric is discussed thoroughly with local assumption.