NVIDIA / NVIDIA/GenerativeAIExamples

Chain server fail to pull the result

未关闭
#269 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

主要语言
Jupyter Notebook
星标
4.2k
派生
1.1k
平均合并
10 小时 15 分钟
30 天内合并 PR
1

描述

2025/01/29 12:35:47.094 +00:00] [WARN] [retry/retry.go:104] ["retry func failed, reach max retry"] [traceID=5ce0d7f04403e4b765eefacedfbf5a5b] [attempt=2]
[2025/01/29 12:35:47.094 +00:00] [WARN] [proxy/task_search.go:581] ["search execute failed"] [traceID=5ce0d7f04403e4b765eefacedfbf5a5b] [nq=1] [error="failed to search/query delegator 2 for channel by-dev-rootcoord-dml_1_454401150931015494v0: fail to search on QueryNode 2: worker(2) query failed: Operator::GetOutput failed for [Operator:, plan node id: 11562] : => failed to search: config={{"ef":10,"k":40,"metric_type":"L2","span_id":"11dd9c06b163fd13","trace_flags":0,"trace_id":"5ce0d7f04403e4b765eefacedfbf5a5b"}} out of range in json: ef(10) should be larger than k(40) at /workspace/source/internal/core/src/index/VectorMemIndex.cpp:411"] [errorVerbose="failed to search/query delegator 2 for channel by-dev-rootcoord-dml_1_454401150931015494v0: fail to search on QueryNode 2: worker(2) query failed: Operator::GetOutput failed for [Operator:, plan node id: 11562] : => failed to search: config={{"ef":10,"k":40,"metric_type":"L2","span_id":"11dd9c06b163fd13","trace_flags":0,"trace_id":"5ce0d7f04403e4b765eefacedfbf5a5b"}} out of range in json: ef(10) should be larger than k(40) at /workspace/source/internal/core/src/index/VectorMemIndex.cpp:411\n(1) attached stack trace\n -- stack trace:\n | github.com/milvus-io/milvus/internal/proxy.(*LBPolicyImpl).ExecuteWithRetry.func1\n | \t/workspace/source/internal/proxy/lb_policy.go:218\n | [...repeated from below...]\nWraps: (2) failed to search/query delegator 2 for channel by-dev-rootcoord-dml_1_454401150931015494v0\nWraps: (3) attached stack trace\n -- stack trace:\n | github.com/milvus-io/milvus/internal/proxy.(*searchTask).searchShard\n | \t/workspace/source/internal/proxy/task_search.go:791\n | github.com/milvus-io/milvus/internal/proxy.(*LBPolicyImpl).ExecuteWithRetry.func1\n | \t/workspace/source/internal/proxy/lb_policy.go:210\n | github.com/milvus-io/milvus/pkg/util/retry.Do\n | \t/workspace/source/pkg/util/retry/retry.go:44\n | github.com/milvus-io/milvus/internal/proxy.(*LBPolicyImpl).ExecuteWithRetry\n | \t/workspace/source/internal/proxy/lb_policy.go:179\n | github.com/milvus-io/milvus/internal/proxy.(*LBPolicyImpl).Execute.func1\n | \t/workspace/source/internal/proxy/lb_policy.go:246\n | golang.org/x/sync/errgroup.(*Group).Go.func1\n | \t/go/pkg/mod/golang.org/x/sync@v0.7.0/errgroup/errgroup.go:78\n | runtime.goexit\n | \t/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.linux-amd64/src/runtime/asm_amd64.s:1695\nWraps: (4) fail to search on QueryNode 2\nWraps: (5) worker(2) query failed: Operator::GetOutput failed for [Operator:, plan node id: 11562] : => failed to search: config={{"ef":10,"k":40,"metric_type":"L2","span_id":"11dd9c06b163fd13","trace_flags":0,"trace_id":"5ce0d7f04403e4b765eefacedfbf5a5b"}} out of range in json: ef(10) should be larger than k(40) at /workspace/source/internal/core/src/index/VectorMemIndex.cpp:411\nError types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) merr.milvusError"]

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

首先从 internal/proxy/task_search.go:581 开始,沿着 internal/proxy/lb_policy.go 和 pkg/util/retry/retry.go 追踪报告的故障,然后检查 VectorMemIndex.cpp:411 中显示 ef(10) 低于 k(40) 的错误。在 chain-server 工作流中复现该搜索并确定预期结果;当查询在没有报告的 retry 失败的情况下返回结果时,即表示完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
go
领域
backend
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
需要澄清
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。