Knowledge Graph Embedding: A Survey of Approaches and Applications 摘要
想看原文戳这里
1. introduction
(1) the definition of KG
A KG is a multi-relational graph composed of entities (nodes) and relations (different types of edges).
(2) examples
Freebase, DBpedia, YAGO, NELL
(3) key idea of KGE
The key idea is to embed components of a KG including entities and relations into continuous vector spaces, so as to simplify the manipulation while preserving the inherent structure of the KG.
(4) traditional procedure of KGE
Given a KG, such a technique first represents entities and relations in a continuous vector space, and defines a scoring function on each fact to measure its plausibility. Entity and relation embeddings can then be obtained by maximizing the total plausibility of observed facts.
2. notations
…
3. KG Embedding with Facts Alone
(1) facts
Facts observed in the KG are stored as a collection of triples D+={(h,r,t)}. Each triple is composed of a head entity h∈E, a tail entity t∈E, and a relation r∈R between them.
(2) steps
① representing entities and relations
Entities are usually represented as vectors, i.e., deterministic points in the vector space. Relations are typically taken as operations in the vector space.
② defining a scoring function
A scoring function fr(h,t) is defined on each fact (h,r,t) to measure its plausibility. Facts observed in the KG tend to have higher scores than those that have not been observed.
③ learning entity and relation representations
The third step solves an optimization problem that maximizes the total plausibility of observed facts.
(3) embedding techniques
① Translational Distance Models
exploit distance-based scoring functions; measure the plausibility of a fact as the distance between the two entities, usually after a translation carried out by the relation
(i) TransE
-represents both entities and relations as vectors in the same space, say Rd
-Given a fact (h,r,t), the relation is interpreted as a translation vector r so that the embedded entities h and t can be connected by r with low error, i.e., h+r≈t when (h,r,t) holds.
-
-TransE has flaws in dealing with 1-to-N, N-to-1, and N-to-N relations : given a 1-to-N relation, e.g., DirectorOf, TransE might learn very similar vector representations for Psycho, Rebecca, and RearWindow which are all films directed by AlfredHitchcock, even though they are totally different entities
(ii) TransE’s Extensions
Relation-Specific Entity Embeddings: allow an entity to have distinct representations when involved in different relations
- TransH
* relation-specific hyperplanes
* models entities as vectors
* models each relation r as a vector r on a hyperplane with wr as the normal vector
*
* By introducing the mechanism of projecting to relation-specific hyperplanes, TransH enables different roles of an entity in different relations.
- TransR
* relation-specific spaces
* entities are represented as vectors in an entity space Rd, and each relation is associated with a specific space Rk and modeled as a translation vector in that space
* Mr∈Rk×d is a projection matrix from the entity space to the relation space of r
*
- TransD
* simplifies TransR
*
- TranSparse (share)
* simplifies TransR
*
- TranSparse (separate)
* simplifies TransR
*
Relaxing Translational Requirement: relaxing the overstrict requirement of h+r≈t
- TransM
*
- ManifoldE
*
- TransF
* only requires t to lie in the same direction with h+r, and meanwhile h in the same direction with t-r
*
- TransA
*
(iii) Gaussian Embeddings
take into account entities’ uncertainties, and model them as random variables
- KG2E
*
*
*
- TransG
*
*
*
(iv) Other Distance Models
- UM (unstructured model)
* a naive version of TransE by setting all r=0
- SE (structured embedding)
*
② Semantic Matching Models
exploit similarity-based scoring functions; measure plausibility of facts by matching latent semantics of entities and relations
(i) RESCAL and Its Extensions
- RESCAL
*
- TATEC
* models not only the three-way interaction h⊤Mrt but also two-way interactions, e.g., those between an entity and a relation
*
- DistMult
* this over-simplified model can only deal with symmetric relations
*
- HolE
* combines the expressive power of RESCAL with the efficiency and simplicity of DistMult
*
*
- ComplEx
* extends DistMult by introducing complex-valued embeddings so as to better model asymmetric relations
* entity and relation embeddings h,r,t no longer lie in a real space but a complex space
* every ComplEx has an equivalent HolE
*
- ANALOGY
* extends RESCAL so as to further model the analogical properties of entities and relations
*
(ii) Matching with Neural Networks
- Semantic Matching Energy (SME)
- Neural Tensor Network (NTN)
- Multi-Layer Perceptron (MLP)
- Neural Association Model (NAM)
③ Model Training
(i) Training under Open World Assumption
- KGs contain only true facts and non-observed facts can be either false or just missing. D+ stores only positive examples.
- Given the positive set D+ and a negative set D- constructed accordingly, we can learn entity and relation representations Θ by minimizing the logistic loss/ pairwise ranking loss.
- logistic loss :
- pairwise ranking loss :
- Minimizing the pairwise ranking loss has an additional advantage: it does not assume that negative examples are necessarily false, just that they are more invalid than those positive ones.
- the logistic loss generally yields better results for semantic matching models such as DistMult and ComplEx, while the pairwise ranking loss may be more suitable for translational distance models like TransE
- steps
* Initializing Entity and Relation Embeddings
~ usually initialized randomly from uniform distributions or Gaussian distributions
~ also initialized with results of simple models such as TransE
~ also represent an entity as the average word vector of its name or description, and initialize word vectors with those pre-trained on a text corpus
* Generating Negative Training Examples
~
~
~ To reduce false-negative examples, (i) set different probabilities for replacing the head and the tail, i.e., to give more chance to replacing the head if the relation is 1-to-N and the tail if the relation is N-to-1 or (ii) given a positive fact, it corrupts a position (i.e., head or tail) using only entities that have appeared in that position with the same relation
~ 50 negatives per positive example is a good trade-off between accuracy and training time
(ii) Training under Closed World Assumption
- all facts that are not contained in D+ are false
- squared loss :
- other possible loss functions include the logistic loss and absolute loss
④ Model Comparison
⑤ Other Approaches
learn representations for head-tail entity pairs rather than individual entities
4. Incorporating Additional Information
(1) Entity Types
① a straight forward method : take IsA as an ordinary relation and the corresponding triples as ordinary training examples
② semantically smooth embedding (SSE)
③ type-embodied knowledge representation learning (TKRL)
(i) can handle hierarchical entity categories and multiple category labels
(ii) a translational distance model with type-specific entity projections
(2) Relation Paths
A relation path is typically defined as a sequence of relations r1→⋯→rℓ through which two entities can be connected on the graph. A key challenge then is how to represent such paths in the same vector space along with entities and relations.
① a straight forward method : represent a path as a composition of the representations of its constituent relations; typical composition operations include addition, multiplication, and recurrent neural network (RNN)
② path-based TransE (PTransE)
…
(3) Textual Descriptions
① align the given KG with an auxiliary text corpus, and then jointly conduct KG embedding and word embedding
② description-embodied knowledge representation learning (DKRL)
③ a text-enhanced KG embedding model, referred to as TEKE
(4) Logical Rules
systems such as WARMR, ALEPH, and AMIE can extract logical rules automatically from KGs
① utilize rules to refine embedding models during KG completion
② a joint model which embeds KG facts and logical rules simultaneously; a key ingredient of this approach, called KALE, is to represent and model facts and rules in a unified framework
③ vector embeddings are introduced for entity pairs rather than individual entities, making it particularly useful for relation extraction
(5) Other Information
Entity Attributes; Temporal Information; Graph Structures; Evidence from Other Relational Learning Methods
5. Applications in Downstream Tasks
…待补充
6. Concluding Remarks
…待补充
Knowledge Graph Embedding: A Survey of Approaches and Applications 摘要相关推荐
- Knowledge Graph Embedding: A Survey of Approaches and Applications (2)
本文是对知识图谱相关文章阅读翻译,<Knowledge Graph Embedding: A Survey of Approaches and Applications>只针对文字部分,图 ...
- Knowledge Graph Embedding: A Survey of Approaches and Applications【翻译】
原文地址:https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8047276 论文名称:Knowledge Graph Embed ...
- 论文阅读《Knowledge Graph Refinement: A Survey of Approaches and Evaluation Methods》
论文链接 一.谷歌的知识图谱 谷歌的知识图谱是在2012年向公众推出的,也是在这个时候,"知识图谱"这个术语被创造出来. 谷歌本身对于知识图谱的构建是相当保密的:只有少数外部来源讨 ...
- 《knowledge graph embedding：a survey of approaches and applications》论文阅读
发表于TKDE 2017. knowledge graph embedding:a survey of approaches and applications abstract 1. introduc ...
- 【实体对齐·BootEA】Bootstrapping Entity Alignment with Knowledge Graph Embedding
文章目录 0.总结 1.动机 2. 贡献+方法 3.应用场景 4.其他模型 5.数据集 6.效果 以下的是组内比较 BootEA: "Bootstrapping Entity Alignme ...
- 学习Knowledge Graph Embedding Based Question Answering代码笔记
前言 最近被导师安排学习一下[Knowledge Graph Embedding Based Question Answering] 这篇paper,这篇paper的重点在于运用了Knowledge ...
- AutoSF- Searching Scoring Functions for Knowledge Graph Embedding
AutoSF: Searching Scoring Functions for Knowledge Graph Embedding Abstract 衡量知识图(KG)中三元组合理性的得分函数(SFs ...
- 知识图谱-第三方工具：LibKGE（用于Knowledge Graph Embedding）【包含多种模型：TransE、DistMult、ComplEx、ConvE、Transformer等】
用于Knowledge Graph Embedding的向量表示库有很多,比如: LibKGE GraphVite AmpliGraph OpenKE pykeen pykg2vec LibKGE的主 ...
- 知识图谱论文阅读（十三）【2020 arXiv】Attentive Knowledge Graph Embedding for Personalized Recommendation
题目: Attentive Knowledge Graph Embedding for Personalized Recommendation 论文链接: 代码链接: 知识图谱特征学习在推荐系统中的应 ...
最新文章
- Service Mesh所应对的8项挑战 1
- 多线程循环输出abcc++_C ++循环| 查找输出程序| 套装4
- 二维码提升对比度文献调研(2)--Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement
- SSM项目使用example查询时多次查询条件相同
- web前端开发技术要求会什么
- 设计素材|剪纸风新年春节烫金PSD分层模板,牛气！
- [kubernetes] comands
- 谷歌输入法linux下载官网下载软件,linux下安装谷歌拼音输入法
- SIM卡电气特性详解
- TCP的短链接和长连接
- linux中文件夹删不掉,Linux无法删除文件夹 Device or resource busy
- 常微分方程各种类型方程表格汇总
- js如何实现侧边广告_如何用稳定器轻松实现低成本的商业级广告片制作？
- 集群搭建系列（八）Kafka
- 联想a2580 android 7,iOS卫垒战! 2013主流小尺寸平板年度横评
- 近视也可以用计算机来算,近视眼度数怎么算的 5个方法教你掌握
- 《SolidCAM+SolidWorks 2014中文版数控加工从入门到精通》——1.2 SolidCAM安装步骤
- scratch-blocks在Mac下本地编译
- VSCODE snippet的用法
- scratch模拟动态声波图/steam教育/输入输出/声音物理学