向量检索不够用?GraphRAG才是多跳推理的答案
05篇讲高级RAG架构时,GraphRAG只占了一小节——当时受限于篇幅,只给了LlamaIndex的基本用法。但GraphRAG是2024-2025年RAG领域最重要的突破之一,微软开源的GraphRAG项目已经拿了2万多star,值得专门一篇拆透。
你可能会问:向量检索已经够用了,为什么要搞知识图谱?我举个真实场景:
公司有500份文档,用户问"我们公司所有产品线之间的技术依赖关系是什么"。向量检索能做的,是找到跟"产品线"和"技术依赖"语义相似的文档片段。但这些片段是分散的——产品A用了技术X,产品B依赖产品A,产品C集成了产品B——这些跨文档的实体关联,向量检索根本串不起来。
GraphRAG的思路:先从文档里抽取实体和关系,构建知识图谱,然后用图的结构化信息做检索。向量检索找的是"像不像",GraphRAG找的是"有没有关系"。
这篇把GraphRAG从原理到实现拆透:实体抽取→关系构建→社区检测→全局/局部检索→工程落地。
GraphRAG vs 向量RAG:什么时候该换方案
先搞清楚一个问题:不是所有场景都需要GraphRAG。
| 维度
|
向量RAG
|
GraphRAG
擅长的问题
|
“X是什么”、“怎么做X”
|
“X和Y什么关系”、“整体趋势是什么”
| |
检索粒度
|
文本片段(chunk)
|
实体+关系+社区摘要
| |
全局性问题
|
差(只能找局部相似片段)
|
强(图结构天然支持全局推理)
| |
多跳推理
|
靠迭代检索,容易丢线索
|
图遍历一次到位
| |
建索引成本
|
低(切分+Embedding)
|
高(LLM抽取实体关系)
| |
索引耗时
|
500篇文档约5分钟
|
500篇文档约30-60分钟
| |
索引成本
|
Embedding API费用
|
LLM调用费用(抽取实体很费token)
| |
查询延迟
|
200-500ms
|
500-2000ms(图遍历+社区摘要)
|
判断标准很简单:如果你的用户经常问"A和B什么关系"、“整体情况怎么样”、“X影响了哪些东西"这类需要跨文档关联的问题,向量RAG搞不定,就该上GraphRAG。如果用户问的都是"退货流程是什么"这种单文档能回答的,向量RAG就够了,别折腾。
GraphRAG的核心流程:四步走
微软GraphRAG的流程分四个阶段:
<span leaf="">文档 → ① 文本切分 → ② 实体关系抽取 → ③ 社区检测 → ④ 社区摘要</span><span leaf=""><br></span><span leaf=""> ↓</span><span leaf=""><br></span><span leaf="">查询 → 局部检索(实体+关系) 或 全局检索(社区摘要)</span><span leaf=""><br></span>
每个阶段都有坑,咱们逐个拆。
阶段一:文本切分
GraphRAG的切分和向量RAG不一样。向量RAG追求chunk大小均匀(200-500字符),GraphRAG追求实体完整性——一个chunk里应该包含完整的实体描述和关系信息。
<span><span leaf="">from</span></span><span leaf=""> langchain_text_splitters </span><span><span leaf="">import</span></span><span leaf=""> RecursiveCharacterTextSplitter</span><span leaf=""><br></span><span leaf=""><br></span><span><span><span leaf="">def</span></span><span leaf=""> </span><span><span leaf="">graph_rag_chunking</span></span><span><span leaf="">(documents, chunk_size=</span><span><span leaf="">1200</span></span><span leaf="">, chunk_overlap=</span><span><span leaf="">100</span></span><span leaf="">)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""GraphRAG专用切分:chunk比向量RAG大,保证实体完整</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> 微软默认用1200 token(约1800字符),比向量RAG的500字符大很多。</span><span leaf=""><br></span><span leaf=""> 原因:小chunk会把一个实体描述截断,抽取出的关系不完整。</span><span leaf=""><br></span><span leaf=""> """</span></span><span leaf=""><br></span><span leaf=""> splitter = RecursiveCharacterTextSplitter(</span><span leaf=""><br></span><span leaf=""> chunk_size=chunk_size,</span><span leaf=""><br></span><span leaf=""> chunk_overlap=chunk_overlap,</span><span leaf=""><br></span><span leaf=""> separators=[</span><span><span leaf="">"\n\n"</span></span><span leaf="">, </span><span><span leaf="">"\n"</span></span><span leaf="">, </span><span><span leaf="">"。"</span></span><span leaf="">, </span><span><span leaf="">";"</span></span><span leaf="">, </span><span><span leaf="">","</span></span><span leaf="">, </span><span><span leaf="">" "</span></span><span leaf="">],</span><span leaf=""><br></span><span leaf=""> )</span><span leaf=""><br></span><span leaf=""> chunks = splitter.split_documents(documents)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> print(</span><span><span leaf="">f"切分完成:</span><span><span leaf="">{len(chunks)}</span></span><span leaf="">个chunk,平均长度</span><span><span leaf="">{sum(len(c.page_content) </span><span><span leaf="">for</span></span><span leaf=""> c </span><span><span leaf="">in</span></span><span leaf=""> chunks)//len(chunks)}</span></span><span leaf="">字符"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">return</span></span><span leaf=""> chunks</span><span leaf=""><br></span>
坑1:chunk太大导致抽取遗漏。我一开始用2400字符的chunk,结果LLM抽取实体时只抽了前半部分的关系,后半部分的实体被忽略了。原因是LLM的注意力在长文本里会衰减。微软默认1200 token是经验值,别随便改大。
阶段二:实体和关系抽取
这是GraphRAG最核心也是成本最高的一步。用LLM从每个chunk里抽取实体(人名、公司名、产品名、技术名)和关系(A是B的CEO、A依赖B、A包含B)。
<span><span leaf="">from</span></span><span leaf=""> langchain_core.prompts </span><span><span leaf="">import</span></span><span leaf=""> ChatPromptTemplate</span><span leaf=""><br></span><span><span leaf="">from</span></span><span leaf=""> langchain_core.output_parsers </span><span><span leaf="">import</span></span><span leaf=""> PydanticOutputParser</span><span leaf=""><br></span><span><span leaf="">from</span></span><span leaf=""> pydantic </span><span><span leaf="">import</span></span><span leaf=""> BaseModel, Field</span><span leaf=""><br></span><span><span leaf="">from</span></span><span leaf=""> typing </span><span><span leaf="">import</span></span><span leaf=""> Optional</span><span leaf=""><br></span><span leaf=""><br></span><span leaf=""><br></span><span><span><span leaf="">class</span></span><span leaf=""> </span><span><span leaf="">Entity</span></span><span><span leaf="">(BaseModel)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""实体定义"""</span></span><span leaf=""><br></span><span leaf=""> name: str = Field(description=</span><span><span leaf="">"实体名称"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> type: str = Field(description=</span><span><span leaf="">"实体类型:PERSON/ORGANIZATION/PRODUCT/TECHNOLOGY/CONCEPT/LOCATION"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> description: str = Field(description=</span><span><span leaf="">"实体描述,一句话说明这个实体是什么"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""><br></span><span leaf=""><br></span><span><span><span leaf="">class</span></span><span leaf=""> </span><span><span leaf="">Relationship</span></span><span><span leaf="">(BaseModel)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""关系定义"""</span></span><span leaf=""><br></span><span leaf=""> source: str = Field(description=</span><span><span leaf="">"源实体名称"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> target: str = Field(description=</span><span><span leaf="">"目标实体名称"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> description: str = Field(description=</span><span><span leaf="">"关系描述,如'是CEO'、'依赖'、'包含'"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> strength: float = Field(description=</span><span><span leaf="">"关系强度0-1,1表示强关系"</span></span><span leaf="">, default=</span><span><span leaf="">0.5</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""><br></span><span leaf=""><br></span><span><span><span leaf="">class</span></span><span leaf=""> </span><span><span leaf="">ExtractionResult</span></span><span><span leaf="">(BaseModel)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""单个chunk的抽取结果"""</span></span><span leaf=""><br></span><span leaf=""> entities: list[Entity]</span><span leaf=""><br></span><span leaf=""> relationships: list[Relationship]</span><span leaf=""><br></span><span leaf=""><br></span><span leaf=""><br></span><span><span><span leaf="">def</span></span><span leaf=""> </span><span><span leaf="">extract_entities_relations</span></span><span><span leaf="">(chunks, llm)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""从每个chunk抽取实体和关系</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> 这一步是GraphRAG成本的大头:每个chunk都要调用一次LLM。</span><span leaf=""><br></span><span leaf=""> 500个chunk = 500次LLM调用,GPT-4o约$30-50。</span><span leaf=""><br></span><span leaf=""> """</span></span><span leaf=""><br></span><span leaf=""> parser = PydanticOutputParser(pydantic_object=ExtractionResult)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> extract_prompt = ChatPromptTemplate.from_template(</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""你是一个信息抽取专家。请从以下文本中抽取实体和关系。</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">抽取规则:</span><span leaf=""><br></span><span leaf="">1. 实体类型:PERSON(人物)、ORGANIZATION(组织)、PRODUCT(产品)、TECHNOLOGY(技术)、CONCEPT(概念)、LOCATION(地点)</span><span leaf=""><br></span><span leaf="">2. 关系:实体之间的关联,如"任职于"、"开发"、"依赖"、"收购"、"合作"等</span><span leaf=""><br></span><span leaf="">3. 每个实体必须有简短描述</span><span leaf=""><br></span><span leaf="">4. 关系强度:明确提到=0.9,间接推断=0.5,弱关联=0.3</span><span leaf=""><br></span><span leaf="">5. 只抽取文本中明确出现的信息,不要编造</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">文本:</span><span leaf=""><br></span><span leaf="">{text}</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">{format_instructions}"""</span></span><span leaf=""><br></span><span leaf=""> )</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> all_entities = []</span><span leaf=""><br></span><span leaf=""> all_relationships = []</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> i, chunk </span><span><span leaf="">in</span></span><span leaf=""> enumerate(chunks):</span><span leaf=""><br></span><span leaf=""> chain = extract_prompt | llm | parser</span><span leaf=""><br></span><span leaf=""> result = chain.invoke({</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"text"</span></span><span leaf="">: chunk.page_content,</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"format_instructions"</span></span><span leaf="">: parser.get_format_instructions(),</span><span leaf=""><br></span><span leaf=""> })</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 记录每个实体来自哪个chunk,方便后续追溯</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> entity </span><span><span leaf="">in</span></span><span leaf=""> result.entities:</span><span leaf=""><br></span><span leaf=""> entity.description += </span><span><span leaf="">f" (来源: chunk_</span><span><span leaf="">{i}</span></span><span leaf="">)"</span></span><span leaf=""><br></span><span leaf=""> all_entities.append(entity)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> rel </span><span><span leaf="">in</span></span><span leaf=""> result.relationships:</span><span leaf=""><br></span><span leaf=""> all_relationships.append(rel)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">if</span></span><span leaf=""> (i + </span><span><span leaf="">1</span></span><span leaf="">) % </span><span><span leaf="">50</span></span><span leaf=""> == </span><span><span leaf="">0</span></span><span leaf="">:</span><span leaf=""><br></span><span leaf=""> print(</span><span><span leaf="">f"已处理 </span><span><span leaf="">{i+</span><span><span leaf="">1</span></span><span leaf="">}</span></span><span leaf="">/</span><span><span leaf="">{len(chunks)}</span></span><span leaf=""> 个chunk,累计抽取</span><span><span leaf="">{len(all_entities)}</span></span><span leaf="">个实体,</span><span><span leaf="">{len(all_relationships)}</span></span><span leaf="">条关系"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">return</span></span><span leaf=""> all_entities, all_relationships</span><span leaf=""><br></span>
坑2:实体名称不统一。同一个"张三"可能被抽成"张三”、“张三(CEO)"、“Zhang San"三种。后面构建图谱时这会被当成三个不同节点。解决:加一层实体合并/消歧。
<span><span><span leaf="">def</span></span><span leaf=""> </span><span><span leaf="">merge_entities</span></span><span><span leaf="">(entities)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""合并同名实体(简单版:名称归一化后合并描述)"""</span></span><span leaf=""><br></span><span leaf=""> entity_map = {} </span><span><span leaf=""># 归一化名称 → 合并后的实体</span></span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> entity </span><span><span leaf="">in</span></span><span leaf=""> entities:</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 归一化:去空格、去括号注释、统一大小写</span></span><span leaf=""><br></span><span leaf=""> normalized_name = entity.name.strip()</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 去掉括号里的注释,如 "张三(CEO)" → "张三"</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">if</span></span><span leaf=""> </span><span><span leaf="">"("</span></span><span leaf=""> </span><span><span leaf="">in</span></span><span leaf=""> normalized_name:</span><span leaf=""><br></span><span leaf=""> normalized_name = normalized_name.split(</span><span><span leaf="">"("</span></span><span leaf="">)[</span><span><span leaf="">0</span></span><span leaf="">]</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">if</span></span><span leaf=""> </span><span><span leaf="">"("</span></span><span leaf=""> </span><span><span leaf="">in</span></span><span leaf=""> normalized_name:</span><span leaf=""><br></span><span leaf=""> normalized_name = normalized_name.split(</span><span><span leaf="">"("</span></span><span leaf="">)[</span><span><span leaf="">0</span></span><span leaf="">]</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">if</span></span><span leaf=""> normalized_name </span><span><span leaf="">in</span></span><span leaf=""> entity_map:</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 已存在:合并描述</span></span><span leaf=""><br></span><span leaf=""> entity_map[normalized_name].description += </span><span><span leaf="">" | "</span></span><span leaf=""> + entity.description</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">else</span></span><span leaf="">:</span><span leaf=""><br></span><span leaf=""> entity.name = normalized_name</span><span leaf=""><br></span><span leaf=""> entity_map[normalized_name] = entity</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">return</span></span><span leaf=""> list(entity_map.values())</span><span leaf=""><br></span>
Java类比:实体合并就像你做用户数据清洗——同一个用户在三个系统里注册了三个账号,你得按手机号或邮箱归一化成一个用户。GraphRAG的实体合并是同样的逻辑。不合并的话,图里全是孤立节点,跟没有图一样。
阶段三:社区检测
抽取完实体和关系后,你有一个图:节点是实体,边是关系。但这个图可能很大很乱——500个文档可能抽出上万个实体,几万条关系。直接用这个图检索效率很低。
社区检测(Community Detection)把图分成若干"社区”——每个社区是一组紧密关联的实体。比如"产品线A相关的所有人和技术"可能构成一个社区,“产品线B"构成另一个社区。
GraphRAG用的是Leiden算法(比Louvain更精确的社区检测算法):
<span><span leaf="">import</span></span><span leaf=""> networkx </span><span><span leaf="">as</span></span><span leaf=""> nx</span><span leaf=""><br></span><span leaf=""><br></span><span><span><span leaf="">def</span></span><span leaf=""> </span><span><span leaf="">build_graph_and_detect_communities</span></span><span><span leaf="">(entities, relationships)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""构建图 + 社区检测"""</span></span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 1. 构建NetworkX图</span></span><span leaf=""><br></span><span leaf=""> graph = nx.Graph()</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 添加节点</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> entity </span><span><span leaf="">in</span></span><span leaf=""> entities:</span><span leaf=""><br></span><span leaf=""> graph.add_node(entity.name, </span><span leaf=""><br></span><span leaf=""> type=entity.type, </span><span leaf=""><br></span><span leaf=""> description=entity.description)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 添加边(关系强度作为边权重)</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> rel </span><span><span leaf="">in</span></span><span leaf=""> relationships:</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">if</span></span><span leaf=""> graph.has_edge(rel.source, rel.target):</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 已有边:取最大强度</span></span><span leaf=""><br></span><span leaf=""> graph[rel.source][rel.target][</span><span><span leaf="">'weight'</span></span><span leaf="">] = max(</span><span leaf=""><br></span><span leaf=""> graph[rel.source][rel.target][</span><span><span leaf="">'weight'</span></span><span leaf="">],</span><span leaf=""><br></span><span leaf=""> rel.strength</span><span leaf=""><br></span><span leaf=""> )</span><span leaf=""><br></span><span leaf=""> graph[rel.source][rel.target][</span><span><span leaf="">'description'</span></span><span leaf="">] += </span><span><span leaf="">" | "</span></span><span leaf=""> + rel.description</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">else</span></span><span leaf="">:</span><span leaf=""><br></span><span leaf=""> graph.add_edge(rel.source, rel.target, </span><span leaf=""><br></span><span leaf=""> weight=rel.strength, </span><span leaf=""><br></span><span leaf=""> description=rel.description)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> print(</span><span><span leaf="">f"图构建完成:</span><span><span leaf="">{graph.number_of_nodes()}</span></span><span leaf="">个节点,</span><span><span leaf="">{graph.number_of_edges()}</span></span><span leaf="">条边"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 2. 社区检测:使用贪婪模块度算法(NetworkX内置,效果接近Leiden)</span></span><span leaf=""><br></span><span leaf=""> communities = nx.community.greedy_modularity_communities(graph, weight=</span><span><span leaf="">'weight'</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> print(</span><span><span leaf="">f"社区检测完成:</span><span><span leaf="">{len(communities)}</span></span><span leaf="">个社区"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> i, community </span><span><span leaf="">in</span></span><span leaf=""> enumerate(communities):</span><span leaf=""><br></span><span leaf=""> print(</span><span><span leaf="">f" 社区</span><span><span leaf="">{i}</span></span><span leaf="">: </span><span><span leaf="">{len(community)}</span></span><span leaf="">个实体 - </span><span><span leaf="">{list(community)[:</span><span><span leaf="">5</span></span><span leaf="">]}</span></span><span leaf="">..."</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">return</span></span><span leaf=""> graph, communities</span><span leaf=""><br></span>
坑3:社区粒度不好控制。社区太大(一个社区几百个实体),摘要会丢失细节;社区太小(一个社区2-3个实体),摘要没什么信息量。微软GraphRAG用分层社区——先检测大社区,再在大社区里检测子社区,构建多层级结构。查询时可以根据问题粒度选择不同层级的社区。
<span><span><span leaf="">def</span></span><span leaf=""> </span><span><span leaf="">build_hierarchical_communities</span></span><span><span leaf="">(graph, max_levels=</span><span><span leaf="">3</span></span><span leaf="">)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""分层社区检测:每层在前一层的社区内继续细分"""</span></span><span leaf=""><br></span><span leaf=""> all_levels = []</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 第0层:全图社区检测</span></span><span leaf=""><br></span><span leaf=""> communities = list(nx.community.greedy_modularity_communities(graph, weight=</span><span><span leaf="">'weight'</span></span><span leaf="">))</span><span leaf=""><br></span><span leaf=""> all_levels.append(communities)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 递归:在每个社区内继续检测子社区</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> level </span><span><span leaf="">in</span></span><span leaf=""> range(</span><span><span leaf="">1</span></span><span leaf="">, max_levels):</span><span leaf=""><br></span><span leaf=""> sub_communities = []</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> community </span><span><span leaf="">in</span></span><span leaf=""> all_levels[</span><span><span leaf="">-1</span></span><span leaf="">]:</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">if</span></span><span leaf=""> len(community) < </span><span><span leaf="">5</span></span><span leaf="">: </span><span><span leaf=""># 太小的社区不再细分</span></span><span leaf=""><br></span><span leaf=""> sub_communities.append(community)</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">continue</span></span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> subgraph = graph.subgraph(community)</span><span leaf=""><br></span><span leaf=""> sub_comms = list(nx.community.greedy_modularity_communities(subgraph, weight=</span><span><span leaf="">'weight'</span></span><span leaf="">))</span><span leaf=""><br></span><span leaf=""> sub_communities.extend(sub_comms)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">if</span></span><span leaf=""> len(sub_communities) == len(all_levels[</span><span><span leaf="">-1</span></span><span leaf="">]):</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">break</span></span><span leaf=""> </span><span><span leaf=""># 没有进一步细分,停止</span></span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> all_levels.append(sub_communities)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">return</span></span><span leaf=""> all_levels</span><span leaf=""><br></span>
阶段四:社区摘要
每个社区生成一段LLM摘要,描述这个社区包含哪些实体、它们之间的关系、整体在讲什么。全局检索时就用这些摘要。
<span><span><span leaf="">def</span></span><span leaf=""> </span><span><span leaf="">generate_community_summaries</span></span><span><span leaf="">(communities, graph, llm)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""为每个社区生成摘要"""</span></span><span leaf=""><br></span><span leaf=""> summary_prompt = ChatPromptTemplate.from_template(</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""请总结以下实体及其关系,形成一段连贯的概述。</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">实体和关系:</span><span leaf=""><br></span><span leaf="">{graph_info}</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">要求:</span><span leaf=""><br></span><span leaf="">1. 提炼核心主题和关键关系</span><span leaf=""><br></span><span leaf="">2. 不要简单罗列实体,要总结出"这个群体在做什么"</span><span leaf=""><br></span><span leaf="">3. 200字以内</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">概述:"""</span></span><span leaf=""><br></span><span leaf=""> )</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> summaries = []</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> i, community </span><span><span leaf="">in</span></span><span leaf=""> enumerate(communities):</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 收集社区内所有实体和关系</span></span><span leaf=""><br></span><span leaf=""> graph_info = []</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> node </span><span><span leaf="">in</span></span><span leaf=""> community:</span><span leaf=""><br></span><span leaf=""> desc = graph.nodes[node].get(</span><span><span leaf="">'description'</span></span><span leaf="">, </span><span><span leaf="">''</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> graph_info.append(</span><span><span leaf="">f"[</span><span><span leaf="">{node}</span></span><span leaf="">] </span><span><span leaf="">{desc}</span></span><span leaf="">"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 收集社区内部的关系(不含跨社区关系)</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> edge </span><span><span leaf="">in</span></span><span leaf=""> graph.subgraph(community).edges(data=</span><span><span leaf="">True</span></span><span leaf="">):</span><span leaf=""><br></span><span leaf=""> source, target, data = edge</span><span leaf=""><br></span><span leaf=""> graph_info.append(</span><span><span leaf="">f"</span><span><span leaf="">{source}</span></span><span leaf=""> --</span><span><span leaf="">{data.get(</span><span><span leaf="">'description'</span></span><span leaf="">, </span><span><span leaf="">'关联'</span></span><span leaf="">)}</span></span><span leaf="">--> </span><span><span leaf="">{target}</span></span><span leaf="">"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 生成摘要</span></span><span leaf=""><br></span><span leaf=""> chain = summary_prompt | llm</span><span leaf=""><br></span><span leaf=""> summary = chain.invoke({</span><span><span leaf="">"graph_info"</span></span><span leaf="">: </span><span><span leaf="">"\n"</span></span><span leaf="">.join(graph_info)}).content</span><span leaf=""><br></span><span leaf=""> summaries.append(summary)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">if</span></span><span leaf=""> (i + </span><span><span leaf="">1</span></span><span leaf="">) % </span><span><span leaf="">10</span></span><span leaf=""> == </span><span><span leaf="">0</span></span><span leaf="">:</span><span leaf=""><br></span><span leaf=""> print(</span><span><span leaf="">f"已生成 </span><span><span leaf="">{i+</span><span><span leaf="">1</span></span><span leaf="">}</span></span><span leaf="">/</span><span><span leaf="">{len(communities)}</span></span><span leaf=""> 个社区摘要"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">return</span></span><span leaf=""> summaries</span><span leaf=""><br></span>
坑4:社区摘要成本高。100个社区 = 100次LLM调用。如果你有1000个社区,光摘要就要$100+。解决:小社区(<5个实体)可以用规则摘要(直接拼接实体描述),不用LLM。只有大社区才用LLM生成摘要。
查询阶段:局部检索 vs 全局检索
GraphRAG有两种查询模式,对应不同类型的问题。
局部检索:查实体关系
用户问"张三跟A公司什么关系”——这是实体级问题,用局部检索。
<span><span><span leaf="">def</span></span><span leaf=""> </span><span><span leaf="">local_search</span></span><span><span leaf="">(query, graph, entities, vectorstore, llm, top_k=</span><span><span leaf="">5</span></span><span leaf="">)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""局部检索:找到query相关的实体,遍历其邻居关系</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> 局部检索结合了向量检索(找相关实体)和图遍历(找关系)。</span><span leaf=""><br></span><span leaf=""> """</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 1. 用向量检索找到query最相关的实体</span></span><span leaf=""><br></span><span leaf=""> relevant_entities = vectorstore.similarity_search(query, k=top_k)</span><span leaf=""><br></span><span leaf=""> entity_names = [doc.metadata[</span><span><span leaf="">'entity_name'</span></span><span leaf="">] </span><span><span leaf="">for</span></span><span leaf=""> doc </span><span><span leaf="">in</span></span><span leaf=""> relevant_entities]</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 2. 图遍历:找到这些实体的邻居(1跳关系)</span></span><span leaf=""><br></span><span leaf=""> context = []</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> entity_name </span><span><span leaf="">in</span></span><span leaf=""> entity_names:</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">if</span></span><span leaf=""> entity_name </span><span><span leaf="">not</span></span><span leaf=""> </span><span><span leaf="">in</span></span><span leaf=""> graph:</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">continue</span></span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 实体本身的描述</span></span><span leaf=""><br></span><span leaf=""> node_data = graph.nodes[entity_name]</span><span leaf=""><br></span><span leaf=""> context.append(</span><span><span leaf="">f"实体: </span><span><span leaf="">{entity_name}</span></span><span leaf=""> (</span><span><span leaf="">{node_data.get(</span><span><span leaf="">'type'</span></span><span leaf="">, </span><span><span leaf="">''</span></span><span leaf="">)}</span></span><span leaf="">)"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> context.append(</span><span><span leaf="">f" 描述: </span><span><span leaf="">{node_data.get(</span><span><span leaf="">'description'</span></span><span leaf="">, </span><span><span leaf="">''</span></span><span leaf="">)}</span></span><span leaf="">"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 邻居关系</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> neighbor </span><span><span leaf="">in</span></span><span leaf=""> graph.neighbors(entity_name):</span><span leaf=""><br></span><span leaf=""> edge_data = graph[entity_name][neighbor]</span><span leaf=""><br></span><span leaf=""> neighbor_type = graph.nodes[neighbor].get(</span><span><span leaf="">'type'</span></span><span leaf="">, </span><span><span leaf="">''</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> context.append(</span><span><span leaf="">f" 关系: </span><span><span leaf="">{entity_name}</span></span><span leaf=""> --</span><span><span leaf="">{edge_data.get(</span><span><span leaf="">'description'</span></span><span leaf="">, </span><span><span leaf="">'关联'</span></span><span leaf="">)}</span></span><span leaf="">--> </span><span><span leaf="">{neighbor}</span></span><span leaf=""> (</span><span><span leaf="">{neighbor_type}</span></span><span leaf="">)"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 3. 用LLM基于图信息生成回答</span></span><span leaf=""><br></span><span leaf=""> graph_context = </span><span><span leaf="">"\n"</span></span><span leaf="">.join(context)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> answer_prompt = ChatPromptTemplate.from_template(</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""基于以下知识图谱信息回答问题。</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">知识图谱信息:</span><span leaf=""><br></span><span leaf="">{graph_context}</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">问题:{question}</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">回答:"""</span></span><span leaf=""><br></span><span leaf=""> )</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> chain = answer_prompt | llm</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">return</span></span><span leaf=""> chain.invoke({</span><span><span leaf="">"graph_context"</span></span><span leaf="">: graph_context, </span><span><span leaf="">"question"</span></span><span leaf="">: query}).content</span><span leaf=""><br></span>
全局检索:查社区摘要
用户问"公司所有产品线的整体技术架构是什么”——这是全局问题,用社区摘要。
<span><span><span leaf="">def</span></span><span leaf=""> </span><span><span leaf="">global_search</span></span><span><span leaf="">(query, community_summaries, llm, max_communities=</span><span><span leaf="">10</span></span><span leaf="">)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""全局检索:把所有社区摘要分批喂给LLM做Map-Reduce</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> Map阶段:每个社区摘要单独生成一个中间回答</span><span leaf=""><br></span><span leaf=""> Reduce阶段:汇总所有中间答案生成最终回答</span><span leaf=""><br></span><span leaf=""> """</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># Map阶段:每个社区摘要生成中间回答</span></span><span leaf=""><br></span><span leaf=""> map_prompt = ChatPromptTemplate.from_template(</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""基于以下社区摘要,回答问题。如果该社区的信息与问题无关,回复"无关"。</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">社区摘要:{summary}</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">问题:{question}</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">中间回答:"""</span></span><span leaf=""><br></span><span leaf=""> )</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> intermediate_answers = []</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> summary </span><span><span leaf="">in</span></span><span leaf=""> community_summaries[:max_communities]:</span><span leaf=""><br></span><span leaf=""> chain = map_prompt | llm</span><span leaf=""><br></span><span leaf=""> answer = chain.invoke({</span><span><span leaf="">"summary"</span></span><span leaf="">: summary, </span><span><span leaf="">"question"</span></span><span leaf="">: query}).content</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">if</span></span><span leaf=""> </span><span><span leaf="">"无关"</span></span><span leaf=""> </span><span><span leaf="">not</span></span><span leaf=""> </span><span><span leaf="">in</span></span><span leaf=""> answer:</span><span leaf=""><br></span><span leaf=""> intermediate_answers.append(answer)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># Reduce阶段:汇总中间答案</span></span><span leaf=""><br></span><span leaf=""> reduce_prompt = ChatPromptTemplate.from_template(</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""以下是多个信息来源对同一问题的回答,请综合成一个完整的答案。</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">问题:{question}</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">各来源回答:</span><span leaf=""><br></span><span leaf="">{answers}</span><span leaf=""><br></span><span leaf=""><br></span><span leaf="">综合答案:"""</span></span><span leaf=""><br></span><span leaf=""> )</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> chain = reduce_prompt | llm</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">return</span></span><span leaf=""> chain.invoke({</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"question"</span></span><span leaf="">: query,</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"answers"</span></span><span leaf="">: </span><span><span leaf="">"\n---\n"</span></span><span leaf="">.join(intermediate_answers),</span><span leaf=""><br></span><span leaf=""> }).content</span><span leaf=""><br></span>
坑5:全局检索延迟高。Map阶段要调用10次LLM(10个社区摘要),Reduce阶段再调1次。总延迟5-10秒。解决:可以并行Map阶段(用asyncio),或减少社区数量(只选跟query最相关的社区,用向量检索筛选)。
完整Pipeline:从文档到查询
<span><span><span leaf="">class</span></span><span leaf=""> </span><span><span leaf="">GraphRAGPipeline</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""GraphRAG完整Pipeline"""</span></span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span><span leaf="">def</span></span><span leaf=""> </span><span><span leaf="">__init__</span></span><span><span leaf="">(self, llm, embedding_model)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> self.llm = llm</span><span leaf=""><br></span><span leaf=""> self.embedding = embedding_model</span><span leaf=""><br></span><span leaf=""> self.graph = </span><span><span leaf="">None</span></span><span leaf=""><br></span><span leaf=""> self.communities = </span><span><span leaf="">None</span></span><span leaf=""><br></span><span leaf=""> self.community_summaries = </span><span><span leaf="">None</span></span><span leaf=""><br></span><span leaf=""> self.entity_vectorstore = </span><span><span leaf="">None</span></span><span leaf=""> </span><span><span leaf=""># 实体描述的向量索引</span></span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span><span leaf="">def</span></span><span leaf=""> </span><span><span leaf="">index</span></span><span><span leaf="">(self, documents)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""建索引:切分→抽取→构图→社区检测→摘要"""</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 1. 切分</span></span><span leaf=""><br></span><span leaf=""> chunks = graph_rag_chunking(documents)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 2. 抽取实体和关系</span></span><span leaf=""><br></span><span leaf=""> entities, relationships = extract_entities_relations(chunks, self.llm)</span><span leaf=""><br></span><span leaf=""> entities = merge_entities(entities) </span><span><span leaf=""># 合并同名实体</span></span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 3. 构图 + 社区检测</span></span><span leaf=""><br></span><span leaf=""> self.graph, self.communities = build_graph_and_detect_communities(entities, relationships)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 4. 社区摘要</span></span><span leaf=""><br></span><span leaf=""> self.community_summaries = generate_community_summaries(self.communities, self.graph, self.llm)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># 5. 实体描述建向量索引(用于局部检索)</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">from</span></span><span leaf=""> langchain_community.vectorstores </span><span><span leaf="">import</span></span><span leaf=""> Chroma</span><span leaf=""><br></span><span leaf=""> entity_docs = []</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">for</span></span><span leaf=""> entity </span><span><span leaf="">in</span></span><span leaf=""> entities:</span><span leaf=""><br></span><span leaf=""> entity_docs.append(Document(</span><span leaf=""><br></span><span leaf=""> page_content=entity.description,</span><span leaf=""><br></span><span leaf=""> metadata={</span><span><span leaf="">"entity_name"</span></span><span leaf="">: entity.name, </span><span><span leaf="">"entity_type"</span></span><span leaf="">: entity.type}</span><span leaf=""><br></span><span leaf=""> ))</span><span leaf=""><br></span><span leaf=""> self.entity_vectorstore = Chroma.from_documents(entity_docs, self.embedding)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> print(</span><span><span leaf="">f"GraphRAG索引完成:</span><span><span leaf="">{len(entities)}</span></span><span leaf="">实体, </span><span><span leaf="">{self.graph.number_of_edges()}</span></span><span leaf="">关系, </span><span><span leaf="">{len(self.communities)}</span></span><span leaf="">社区"</span></span><span leaf="">)</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> </span><span><span><span leaf="">def</span></span><span leaf=""> </span><span><span leaf="">query</span></span><span><span leaf="">(self, question, mode=</span><span><span leaf="">"auto"</span></span><span leaf="">)</span></span><span leaf="">:</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">"""查询:auto模式自动选择局部或全局检索"""</span></span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">if</span></span><span leaf=""> mode == </span><span><span leaf="">"local"</span></span><span leaf="">:</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">return</span></span><span leaf=""> local_search(question, self.graph, </span><span><span leaf="">None</span></span><span leaf="">, self.entity_vectorstore, self.llm)</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">elif</span></span><span leaf=""> mode == </span><span><span leaf="">"global"</span></span><span leaf="">:</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">return</span></span><span leaf=""> global_search(question, self.community_summaries, self.llm)</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">else</span></span><span leaf="">:</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf=""># auto:用LLM判断问题类型</span></span><span leaf=""><br></span><span leaf=""> route_prompt = </span><span><span leaf="">f"""判断以下问题的类型:</span><span leaf=""><br></span><span leaf=""> - 如果是具体实体关系问题("X和Y什么关系"、"X是谁")→ local</span><span leaf=""><br></span><span leaf=""> - 如果是全局分析问题("整体情况"、"所有X的"、"趋势")→ global</span><span leaf=""><br></span><span leaf=""> </span><span leaf=""><br></span><span leaf=""> 问题:</span><span><span leaf="">{question}</span></span><span leaf=""><br></span><span leaf=""> 类型(local/global):"""</span></span><span leaf=""><br></span><span leaf=""> route = self.llm.invoke(route_prompt).content.strip().lower()</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">if</span></span><span leaf=""> </span><span><span leaf="">"global"</span></span><span leaf=""> </span><span><span leaf="">in</span></span><span leaf=""> route:</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">return</span></span><span leaf=""> global_search(question, self.community_summaries, self.llm)</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">else</span></span><span leaf="">:</span><span leaf=""><br></span><span leaf=""> </span><span><span leaf="">return</span></span><span leaf=""> local_search(question, self.graph, </span><span><span leaf="">None</span></span><span leaf="">, self.entity_vectorstore, self.llm)</span><span leaf=""><br></span>
GraphRAG的成本优化
GraphRAG最大的问题是贵。500篇文档建索引,GPT-4o要$30-50。这里有几个省钱方案:
| 优化手段
|
节省比例
|
代价
实体抽取用小模型(GPT-4o-mini)
|
省70%
|
抽取质量下降约15%
| |
小社区用规则摘要不用LLM
|
省30%
|
小社区摘要质量略差
| |
增量索引(只处理新文档)
|
省80%+
|
需要维护索引版本
| |
实体抽取缓存(相同chunk不重复抽取)
|
省50%
|
需要hash去重
| |
批量API调用(OpenAI Batch API)
|
省50%
|
延迟增加24小时
|
我的建议:开发阶段用GPT-4o-mini做实体抽取,效果验证后再用GPT-4o重新建索引。或者用本地模型(Qwen-14B)做抽取,零API成本。
什么时候该上GraphRAG
<span leaf="">你的RAG遇到什么问题?</span><span leaf=""><br></span><span leaf="">│</span><span leaf=""><br></span><span leaf="">├── 用户问</span><span><span leaf="">"A和B什么关系"</span></span><span leaf="">→ 向量检索找不到跨文档关联</span><span leaf=""><br></span><span leaf="">│ └── 该上GraphRAG</span><span leaf=""><br></span><span leaf="">│</span><span leaf=""><br></span><span leaf="">├── 用户问</span><span><span leaf="">"整体趋势/所有产品的依赖关系"</span></span><span leaf=""><br></span><span leaf="">│ └── 该上GraphRAG(全局检索)</span><span leaf=""><br></span><span leaf="">│</span><span leaf=""><br></span><span leaf="">├── 用户问</span><span><span leaf="">"X影响了哪些东西"</span></span><span leaf=""><br></span><span leaf="">│ └── 该上GraphRAG(图遍历)</span><span leaf=""><br></span><span leaf="">│</span><span leaf=""><br></span><span leaf="">├── 用户问</span><span><span leaf="">"退货流程是什么"</span></span><span leaf="">→ 单文档能回答</span><span leaf=""><br></span><span leaf="">│ └── 不需要GraphRAG,向量RAG够了</span><span leaf=""><br></span><span leaf="">│</span><span leaf=""><br></span><span leaf="">└── 预算有限,文档量<100篇</span><span leaf=""><br></span><span leaf=""> └── 不建议上,LLM抽取成本太高</span><span leaf=""><br></span>
本篇要点
| 要点
|
说明
GraphRAG核心流程
|
切分→实体关系抽取→社区检测→社区摘要
| |
局部检索
|
向量检索找实体 + 图遍历找关系,适合实体级问题
| |
全局检索
|
Map-Reduce社区摘要,适合全局分析问题
| |
社区检测
|
Leiden算法把图分成紧密关联的子图,分层社区支持多粒度查询
| |
成本优化
|
小模型抽取 + 规则摘要 + 增量索引 + 缓存
| |
适用场景
|
实体关系密集 + 全局性问题 + 跨文档推理
|
踩坑清单
-
chunk太大抽取遗漏:超过1500字符LLM注意力衰减,后半部分实体被忽略。默认1200 token别改大。
-
实体名称不统一:同一实体被抽成多个名字,图里变成多个孤立节点。必须做实体合并/消歧。
-
社区粒度不可控:一个社区太大或太小都不好用。用分层社区检测,查询时选合适层级。
-
社区摘要成本高:100个社区100次LLM调用。小社区用规则摘要,大社区才用LLM。
-
全局检索延迟高:Map阶段串行调用10次LLM要5-10秒。用asyncio并行化,或减少参与Map的社区数。
-
增量更新困难:新增文档后不能只更新局部图,可能影响社区结构。微软GraphRAG的增量更新还在迭代中,目前最稳妥的方案是定期全量重建。
-
图太大查询慢:上万节点的图遍历会卡。给图建索引(如Neo4j的索引),或者用图数据库替代NetworkX。
踩坑清单补充
| 坑
|
症状
|
解决
chunk太大
|
LLM抽取后半部分实体被忽略
|
默认1200token别改大
| |
实体名称不统一
|
图里全是孤立节点
|
加实体合并层
| |
社区粒度不好
|
太大丢细节,太小没信息量
|
分层社区检测
| |
社区摘要成本高
|
100个社区100次LLM调用
|
小社区用规则摘要
| |
全局检索慢
|
Map阶段串行10次LLM
|
asyncio并行
| |
增量更新难
|
新文档可能影响社区结构
|
定期全量重建
|
下篇预告
下一篇讲多模态RAG——当你的文档里不只是文字,还有图片、表格、PDF中的图表,怎么检索?多模态Embedding、colpali视觉检索、表格抽取方案,一篇讲透。
你的项目有需要跨文档关联推理的场景吗?向量检索搞不定的那种?评论区说说。
觉得有用就点个在看,下一篇讲多模态RAG——图片表格怎么检索。
- 原文作者:知识铺
- 原文链接:https://index.zshipu.com/ai002/post/20260822/%E5%90%91%E9%87%8F%E6%A3%80%E7%B4%A2%E4%B8%8D%E5%A4%9F%E7%94%A8GraphRAG%E6%89%8D%E6%98%AF%E5%A4%9A%E8%B7%B3%E6%8E%A8%E7%90%86%E7%9A%84%E7%AD%94%E6%A1%88/
- 版权声明:本作品采用知识共享署名-非商业性使用-禁止演绎 4.0 国际许可协议进行许可,非商业转载请注明出处(作者,原文链接),商业转载请联系作者获得授权。
- 免责声明:本页面内容均来源于站内编辑发布,部分信息来源互联网,并不意味着本站赞同其观点或者证实其内容的真实性,如涉及版权等问题,请立即联系客服进行更改或删除,保证您的合法权益。转载请注明来源,欢迎对文章中的引用来源进行考证,欢迎指出任何有错误或不够清晰的表达。也可以邮件至 sblig@126.com