googleapis / googleapis/python-aiplatform

tests.system.vertexai.test_tokenization.TestTokenization: many tests failed

未关闭
#6,672 6 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
api: vertex-ai flakybot: issue priority: p1 type: bug
主要语言
Python
星标
905
派生
465
平均合并
1 天 13 小时
30 天内合并 PR
44

描述

Many tests failed at the same time in this package.

* I will close this issue when there are no more failures in this package _and_
there is at least one pass.
* No new issues will be filed for this package until this issue is closed.
* If there are already issues for individual test cases, I will close them when
the corresponding test passes. You can close them earlier, if you prefer, and
I won't reopen them while this issue is still open.

Here are the tests that failed:
* test_count_tokens_local[_get_tokenizer_for_model_preview-gemini-1.5-pro-002-udhr-udhr-PROD_ENDPOINT]
* test_count_tokens_local[get_tokenizer_for_model-gemini-1.5-pro-002-udhr-udhr-PROD_ENDPOINT]
* test_compute_tokens[_get_tokenizer_for_model_preview-gemini-1.5-pro-002-udhr-udhr-PROD_ENDPOINT]
* test_compute_tokens[get_tokenizer_for_model-gemini-1.5-pro-002-udhr-udhr-PROD_ENDPOINT]
* test_count_tokens_system_instruction[gemini-1.5-pro-002-PROD_ENDPOINT]
* test_count_tokens_system_instruction_is_function_call[gemini-1.5-pro-002-PROD_ENDPOINT]
* test_count_tokens_system_instruction_is_function_response[gemini-1.5-pro-002-PROD_ENDPOINT]
* test_count_tokens_tool_is_function_declaration[gemini-1.5-pro-002-PROD_ENDPOINT]
* test_count_tokens_content_is_function_call[gemini-1.5-pro-002-PROD_ENDPOINT]
* test_count_tokens_content_is_function_response[gemini-1.5-pro-002-PROD_ENDPOINT]

-----
commit: ca6b45e3fa09bfa53c2f2c1b1d44f9a3c7aa79d7
buildURL: [Build Status](https://source.cloud.google.com/results/invocations/f6fde6a0-bd74-4ea9-b7b7-c2c539a82eab), [Sponge](http://sponge2/f6fde6a0-bd74-4ea9-b7b7-c2c539a82eab)
status: failed

贡献指南

打开贡献指南

调研方向

首先运行 tests.system.vertexai.test_tokenization.TestTokenization,并检查 gemini-1.5-pro-002 针对 PROD_ENDPOINT 列出的失败项。跟踪本地计数、计算出的 token、系统指令、工具以及函数调用或响应的 tokenization 行为。完成的标准是 package 不再有任何失败,并且至少有一个测试通过。

由索引模型根据 Issue 内容生成。

评估

技术栈
google-cloud, python
领域
ai, testing-qa
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
冷清
描述清晰度
需要澄清
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。