Understanding of Hilbert-Schmidt Independence Criterion (HSIC)

Problem
Given (x1,y1),…,(xn,yn)∈P(x,y),(x_1, y_1), …, (x_n, y_n) \in P(x, y),(x1,y1),…,(xn,yn)∈P(x,y),
determing whether P(x,y)=P(x)×P(y).P(x, y) = P(x) \times P(y).P(x,y)=P(x)×P(y).
Or measure degree of dependence.
Below is the formal problem setting:
Let Pxy be a Borel probability measure defined on a domain X x Y, and let Px and Py be the respective marginal distributions on X and Y. Given an I.I.D sample Z := (X, Y) = {(x1, y1), …, (xm, ym)} of size m drawn independently and identically distributed according to Pxy, does Pxy factorize as PxPy.
Applications
- Independent component analysis;
- Dimsionality reduction and feature extraction;
- Statistical modeling.
Indirect Approach
- Perform density estimate of P(x, y)
- Check whether the estimate approximately factorizes
Direct Approach
- Check properties of factorizing distributions
- e.g. kurtosis, covariance operators.

HSIC is defined as the squared Hilbert-Schmidt (HS) norm of the associated cross-covariance operator Cxy:
HSIC(Pxy,F,G)=∣∣Cxy∣∣HS2.{\rm HSIC}(P_{xy}, F, G) = || C_{xy} || ^2 _{HS}.HSIC(Pxy,F,G)=∣∣Cxy∣∣HS2.
Above definition of HSIC involves two other definitions. One is cross-covariance operator which is defined:
Cxy:=Exy([f(x)−Ex(f(x))][g(y)−Ey(g(y))]).C_{xy} := {\rm E}_{xy}([f(x) - {\rm E}_{x}(f(x))] [g(y) - {\rm E}_{y}(g(y))]).Cxy:=Exy([f(x)−Ex(f(x))][g(y)−Ey(g(y))]).
It is obvious that Cxy is a linear opertor that maps from G to F. The operator itself can be writen in: Cxy:=Exy[(f(x)−μx)⊗(g(y)−μy)],C_{xy} := {\rm E}_{xy}[(f(x) - \mu_x) \otimes (g(y) - \mu_y)],Cxy:=Exy[(f(x)−μx)⊗(g(y)−μy)],
where it denotes the tensor product.

Understanding of Hilbert-Schmidt Independence Criterion (HSIC)相关推荐

论文阅读《Expert Training Task Hardness Aware Meta-Learning for Few-Shot Classification》
原文链接:https://arxiv.org/abs/2007.06240 摘要在一些深度神经网络的优化工作中发现,训练数据的顺序对于分类器的性能有一定影响,合理地安排顺序使分类器更快.更准.受此启 ...
迁移学习一——TCA和SSTCA
本篇文章是对论文<Domain Adaptation via Transfer Component Analysis>进行学习 1 背景首先我们引入域的概念,通常认为域由两部分组成:输入 ...
干货！重新思考希尔伯特-施密特信息瓶颈在对抗性鲁棒性中的应用
点击蓝字关注我们 AI TIME欢迎每一位AI爱好者的加入! 本文首次提出将希尔伯特-施密特信息瓶颈(HSIC Information Bottleneck)用于神经网络优化目标中的正则项来提升模型 ...
使用new bing随机生成的综述【无聊的实验】
随机过程是描述随机现象的数学模型,它可以用来刻画不确定性.变化性和动态性.在机器学习中,随机过程有着广泛的应用,例如对时序数据.空间数据.图结构数据等进行建模和推理.本文将介绍一种常用的随机过程模型: ...
图像处理与机器视觉行业分析
一行业分析数字图像处理是对图像进行分析.加工.和处理,使其满足视觉.心理以及其他要求的技术.图像处理是信号处理在图像域上的一个应用.目前大多数的图像是以数字形式存储,因而图像处理很多情况下指数字 ...
我爱机器学习机器学习干货站- 资源
我爱机器学习机器学习干货站跳至正文机器学习 Deep Learning 所有主题关注我们 DeepMind团队的<Deep Reinforcement Learning in Large ...
机器学习（Machine Learning）入门科普
=======================国外==================== Machine Learning 大家(1):M. I. Jordan (http://www.cs.ber ...
Transformer 眼中世界 Vs. CNN 眼中世界
点击上方"3D视觉工坊",选择"星标" 干货第一时间送达作者丨Andy 来源丨Andy的写作间编辑丨极市平台导读谷歌大脑团队近日的新工作,回答了关于Vi ...
国外AI界牛人主页及资源链接
感觉好博客要收集,还是贴在自己空间里难忘!!! 原文链接:http://blog.csdn.net/hitwengqi/article/details/7907366 http://people.c ...

Understanding of Hilbert-Schmidt Independence Criterion (HSIC)

Understanding of Hilbert-Schmidt Independence Criterion (HSIC)相关推荐

最新文章

热门文章