回顾知识图谱表示应用场景：实体推荐与知识推理过程简析

574次阅读

没有评论

今天是2023年11月26日，星期日，天津，天气晴。

我们回到知识图谱这个话题。

在以前的文章中我们介绍了知识图谱表示学习的一些经典方法，包括TransE、TransD、TransR、TransH、RotateE、pairRE等，通过这些方法，我们可以学习实体和关系的嵌入表示，并用于实体补全等操作。

之前有篇文章写到知识图谱的推理，《技术总结：知识图谱推理中的常用方法、关键问题、评测指标与开放数据总结》，就知识推理常用的方法和关键问题等方面进行了总结。

本文主要围绕知识图谱表示的落地应用这一主题展开论述，介绍其中的两个重要场景，并结合实际的案例出发进行展示，以加深理解。

一、知识图谱表示的应用场景

知识图谱表示学习，能够将实体和关系嵌入到一个低维空间，使得相似的实体和关系尽可能相近。基于这样一种假设，我们可以得到知识补全以及实体推荐这两个主要应用点。

1、实体补全

实体补全问题，是知识图谱推理的一个形式化表现，包括边的预测和实体或属性的预测两种。形式化出来就是给定<h,r,t>去掉其中的一个部分，然后利用其余2个预测其中缺失的部分，例如比如<姚明，老婆，叶莉>，把叶莉扣掉，变成<姚明，老婆，?>，然后任务就是预测这个叶莉。

在具体实现上，知识图谱补全任务用来预测三元组（h，r，t）中缺失实体h，t或r的任务，对于每个待测试三元组，每个评测三元组，移去头部实体（迭代的方式替换尾部实体）、轮流替换成词表中的其他实体，构建错误的三元组实体。利用关系函数计算头部实体和尾部实体的相似度，对于这个相似度来讲，正确的三元组的值应该比较小，而错误样本的相似度值会比较大。用关系函数对所有的三元组（包括正确的三元组和错误的三元组）进行计算，并按照升序排序。因此，针对这个结果，就延伸出来了hit@k、MRR等评估指标。

2、实体推荐

实体推荐问题，是搜索推荐业务中常用的一个板块，例如下图展示了百度搜索中的右侧推荐产品。系统通过用户输入的query”珠穆朗玛峰”，通过实体推荐策略的可以得到右侧的相关山体实体，包括西伯利亚山脉等。

回顾知识图谱表示应用场景：实体推荐与知识推理过程简析

这个在具体的实现上，可以基于预先训练好的实体向量，再加入相似度计算方式得到，也可以融合字面相似度，利用fasttext、word2vec等策略现。

二、基于TransE表示模型进行知识表示训练

下面以FB15k-237这一数据集，采用TransE表示学习训练的过程进行流程介绍。

该方法的输入包括三元组triples.txt，实体文件entity和关系文件relation三部分组成，后面两者可以从第一个文件中处理得到。如下表示了一个三元组文件：

/m/027rn /location/country/form_of_government /m/06cx9 /m/017dcd /tv/tv_program/regular_cast./tv/regular_tv_appearance/actor /m/06v8s0 /m/07s9rl0 /media_common/netflix_genre/titles /m/0170z3 /m/01sl1q /award/award_winner/awards_won./award/award_honor/award_winner /m/044mz_ /m/0cnk2q /soccer/football_team/current_roster./sports/sports_team_roster/position /m/02nzb8 /m/04nrcg /soccer/football_team/current_roster./soccer/football_roster_position/position /m/02sdk9v /m/07nznf /film/actor/film./film/performance/film /m/014lc_ /m/02qyp19 /award/award_category/nominees./award/award_nomination/nominated_for /m/02d413 /m/0q9kd /award/award_nominee/award_nominations./award/award_nomination/award_nominee /m/0184jc /m/03q5t /music/performance_role/regular_performances./music/group_membership/role /m/07y_7 /m/0gqng /award/award_category/winners./award/award_honor/ceremony /m/073hkh /m/0b76d_m /film/film/release_date_s./film/film_regional_release_date/film_release_distribution_medium /m/029j_ /m/014_x2 /award/award_winning_work/awards_won./award/award_honor/award_winner /m/012ljv /m/0ds35l9 /film/film/release_date_s./film/film_regional_release_date/film_release_region /m/05r4w /m/015qsq /film/film/language /m/02bjrlw /m/04bdxl /award/award_nominee/award_nominations./award/award_nomination/award_nominee /m/02s2ft /m/09c7w0 /location/location/contains /m/0rs6x /m/079vf /film/actor/film./film/performance/film /m/0d90m

基于这一内容，可以建立entity_dict，将头尾实体进行ID化，并形成字典形式，如relation_dict:

0 /organization/organization/headquarters./location/mailing_address/state_province_region 1 /education/educational_institution/colors 2 /people/person/profession 3 /film/film/costume_design_by 4 /film/film/genre 5 /celebrities/celebrity/celebrity_friends./celebrities/friendship/friend 6 /tv/tv_producer/programs_produced./tv/tv_producer_term/producer_type 7 /film/film/executive_produced_by 8 /sports/sports_team/roster./basketball/basketball_roster_position/position 9 /award/award_nominee/award_nominations./award/award_nomination/nominated_for 10 /award/award_category/winners./award/award_honor/award_winner 11 /award/award_winner/awards_won./award/award_honor/award_winner 12 /music/artist/origin 13 /food/food/nutrients./food/nutrition_fact/nutrient 14 /film/film/distributors./film/film_film_distributor_relationship/region 15 /time/event/instance_of_recurring_event 16 /sports/professional_sports_team/draft_picks./sports/sports_league_draft_pick/school 17 /film/film/language

又如，entity_dict中存储实体映射字典:

0 /m/0vm5t 1 /m/07fb6 2 /m/06nm1 3 /m/0fvr1 4 /m/0f6lx 5 /m/016cff 6 /m/01pk8b 7 /m/0456xp 8 /m/0f1kwr 9 /m/07c98 10 /m/0ql7q 11 /m/01mb87 12 /m/03l26m 13 /m/017323 14 /m/023fxp 15 /m/0s5cg

通过送入模型，使用得分函数进行计算，可以生成关系实体和嵌入的表示，得到关系向量文件relation_embeddibg.nparry，实体向量文件entitiy_embedding.nparray。

例如：relation_embedding.narray，大小为关系数量＊向量维度

/people/person/spouse_s./people/marriage/spouse [-0.24878718 -0.14749844 0.02170403 ... -0.09271624 -0.17970167 0.06479023] /people/cause_of_death/people [ 0.08820944 0.17076945 0.1551227 ... 0.20603253 0.09615939 -0.03900995] /organization/organization_founder/organizations_founded [-0.1254077 0.02902584 -0.19820833 ... -0.07185289 0.03939589 0.00487392] /government/government_office_category/officeholders./government/government_position_held/jurisdiction_of_office [ 0.05308013 0.09621259 0.06344923 ... 0.16410299 0.1449485 -0.04601951] /tv/tv_program/languages [ 0.02785942 0.12866454 -0.20449817 ... -0.06031425 0.29029676 -0.02030068] /base/popstra/location/vacationers./base/popstra/vacation_choice/vacationer [-0.01933523 0.12925176 0.0730862 ... 0.03981015 -0.09181329 0.18411466] /influence/influence_node/influenced_by [-0.13907708 -0.08791728 0.0683579 ... 0.09029175 -0.15296058 -0.02200152] /location/country/second_level_divisions [-0.03234593 -0.01131915 0.11482401 ... 0.02604388 0.03834524 0.16948576]

又如:entity_embedding.array，大小为实体数量＊向量维度。

/m/09wwlj [ 0.00865843 -0.00612095 0.01060241 -0.02893136 0.06107994 0.02354774 0.03058389 -0.00427339 0.01820436 -0.04203533] /m/05mph [-0.04250936 -0.03532703 -0.0092318 -0.05083899 0.03814617 -0.02377752 0.00620935 -0.04750924 0.01671227 0.00393458] /m/02g_6j [ 0.02165578 0.01128484 0.05618706 -0.00257486 0.00690633 0.01952383 -0.0080839 0.08011721 0.02630108 -0.0112354 ] /m/066m4g [-0.00283125 -0.00569218 0.02547518 0.04415546 0.05473727 0.00095892 -0.01951734 0.02279767 -0.02202196 0.049605 ] /m/011s0 [-0.0248879 -0.00882316 -0.03032782 0.03560867 0.02033721 -0.04790292 -0.00235457 0.05997594 -0.02088312 0.0002314 ] /m/0127ps [ 0.03237503 -0.03975173 0.02703967 -0.02258028 -0.00022754 -0.0013597 0.02303881 -0.00570352 -0.02743368 -0.01313969]

三、基于向量的实体与关系推荐

上面说到，基于训练好的向量，可以支持实体推荐和知识补全两个任务。下面就具体操作加以讲解：

首先，我们可以进一步形成向量文件以供相似检索使用，例如：将实体embedding转换为对应的向量文件

def trans_vec_ent(): entity_vec = np.load(ent_embed) out_vec = open(ent_vec_path, 'w+', encoding="utf-8") word_count = len(entity_vec) embedding_size = len(entity_vec[0]) out_vec.write(str(word_count) + ' ' + str(embedding_size) + 'n') for id, entity in enumerate(entity_vec): name = ent_id_dict[id] out_vec.write(name + ' ' + ' '.join([str(i) for i in entity]) + 'n') if id % 1000 == 0: print(id) out_vec.close() return

1、相似推荐

通过gensim内置的cosine相似度方法，可以快速的召回向量相似的实体或者关系类型，例如：

1）实体召回

/m/0vm5t ['/m/0nj7b', '/m/02dtg', '/m/0vm39', '/m/0vrmb', '/m/01mqh5', '/m/0nj07', '/m/0njdm', '/m/04rrx', '/m/0wh3', '/m/01s0_f'] /m/07fb6 ['/m/06s9y', '/m/06ryl', '/m/01n8qg', '/m/035yg', '/m/05qkp', '/m/0164b', '/m/03_r3', '/m/020p1', '/m/04hvw', '/m/06v36'] /m/06nm1 ['/m/02h40lc', '/m/064_8sq', '/m/0h5k', '/m/01r2l', '/m/02bjrlw', '/m/04306rv', '/m/06b_j', '/m/05zjd', '/m/0t_2', '/m/04h9h'] /m/0fvr1 ['/m/01n30p', '/m/0bwhdbl', '/m/0m2kd', '/m/02pw_n', '/m/01l_pn', '/m/0cqr0q', '/m/0bz3jx', '/m/09ps01', '/m/05ch98', '/m/0cmdwwg'] /m/0f6lx ['/m/0f0y8', '/m/01wz_ml', '/m/053yx', '/m/041mt', '/m/0tcj6', '/m/02mslq', '/m/0bdlj', '/m/01kvqc', '/m/01l7cxq', '/m/01wp8w7'] /m/016cff ['/m/015z4j', '/m/0r0ls', '/m/01my95', '/m/0162c8', '/m/03f3yfj', '/m/051q39', '/m/01w7nwm', '/m/01445t', '/m/069d71', '/m/02lm0t'] /m/01pk8b ['/m/01z215', '/m/094vf', '/m/058wp', '/m/0cw51', '/m/0dlm_', '/m/0bqyhk', '/m/01f08r', '/m/04s9n', '/m/013g3', '/m/0c7zf'] /m/0456xp ['/m/01r93l', '/m/01l2fn', '/m/05bnp0', '/m/04shbh', '/m/0f276', '/m/09wj5', '/m/013w7j', '/m/0184jc', '/m/02vntj', '/m/01kgv4'] /m/0f1kwr ['/m/03x6rj', '/m/04kn29', '/m/073hkx', '/m/0747k8', '/m/08pgl8', '/m/02b0zd', '/m/03mg3l', '/m/04lhft', '/m/09kzxt', '/m/03n5v'] /m/07c98 ['/m/0b24sf', '/m/0c8tk', '/m/03rk0', '/m/019fbp', '/m/0f1_p', '/m/019fc4', '/m/0yyh', '/m/05jrj4', '/m/019fm7', '/m/01c1nm'] /m/0ql7q ['/m/0j54b', '/m/0ql86', '/m/01hwkn', '/m/0c0wvx', '/m/0ql76', '/m/0py8j', '/m/0845v', '/m/03jqfx', '/m/01_3rn', '/m/0k4y6']

2）关系召回

/organization/organization/headquarters./location/mailing_address/state_province_region ['/soccer/football_player/current_team./sports/sports_team_roster/team', '/education/educational_institution_campus/educational_institution', '/location/hud_county_place/place', '/base/x2010fifaworldcupsouthafrica/world_cup_squad/current_world_cup_squad./base/x2010fifaworldcupsouthafrica/current_world_cup_squad/current_club', '/education/educational_institution/campuses', '/base/biblioness/bibs_location/state', '/sports/pro_athlete/teams./sports/sports_team_roster/team', '/food/food/nutrients./food/nutrition_fact/nutrient', '/award/award_winning_work/awards_won./award/award_honor/award_winner', '/sports/professional_sports_team/draft_picks./sports/sports_league_draft_pick/school'] /education/educational_institution/colors ['/soccer/football_player/current_team./sports/sports_team_roster/team', '/education/educational_institution/campuses', '/education/educational_institution_campus/educational_institution', '/location/hud_county_place/place', '/base/x2010fifaworldcupsouthafrica/world_cup_squad/current_world_cup_squad./base/x2010fifaworldcupsouthafrica/current_world_cup_squad/current_club', '/sports/pro_athlete/teams./sports/sports_team_roster/team', '/food/food/nutrients./food/nutrition_fact/nutrient', '/award/award_winning_work/awards_won./award/award_honor/award_winner', '/film/film/produced_by', '/sports/professional_sports_team/draft_picks./sports/sports_league_draft_pick/school'] /people/person/profession ['/soccer/football_player/current_team./sports/sports_team_roster/team', '/film/film/production_companies', '/location/hud_county_place/place', '/music/artist/track_contributions./music/track_contribution/role', '/music/group_member/membership./music/group_membership/role', '/education/educational_institution/campuses', '/organization/organization/headquarters./location/mailing_address/state_province_region', '/people/person/religion', '/location/location/time_zones', '/education/educational_institution_campus/educational_institution']