googleapis / googleapis/python-aiplatform

Adding examples for correct prompt structure for custom endpoints on Vertex AI

未关闭
#4,264 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
api: vertex-ai
主要语言
Python
星标
905
派生
465
平均合并
1 天 13 小时
30 天内合并 PR
44

描述

Thanks for stopping by to let us know something could be better!

**PLEASE READ**: If you have a support contract with Google, please create an issue in the [support console](https://cloud.google.com/support/) instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

- Search the issues already opened: https://github.com/googleapis/python-aiplatform/issues
- Search StackOverflow: https://stackoverflow.com/questions/tagged/google-cloud-platform+python

If you are still having issues, please be sure to include as much information as possible:

#### Environment details

- OS type and version: Jupyter Notebook 6.5.5 on Colab enterprise
- Python version: `python --version` Python 3.10.12
- pip version: `pip --version`
- `google-cloud-aiplatform` version: `pip show google-cloud-aiplatform`

#### Steps to reproduce

1. Add Python def with the correct promot structure to pass on to open source models. I.e Llama 3.1
2. Add example REST API payload to prompt deployed endpoint on Vertex AI

#### Code example for Llama 3.1 managed infra endpoints

```python
prompt_payload = " What is the capital of Paris?" # @param {type:"string"}
prompt ="<|start_header_id|>user<|end_header_id|>"+prompt_payload+"<|eot_id|><|start_header_id|>assistant<|end_header_id|>"
max_tokens = 8000 # @param {type:"integer"}
temperature = 1.0 # @param {type:"number"}
top_p = 1.0 # @param {type:"number"}
top_k = 1.0 # @param {type:"number"}
raw_response = True # @param {type:"boolean"}

# Overides parameters for inferences.
# If you encounter the issue like `ServiceUnavailable: 503 Took too long to respond when processing`,
# you can reduce the max length, such as set max_tokens as 20.
instances = [
{
"prompt": prompt,
"max_tokens": max_tokens,
"temperature": temperature,
"top_p": top_p,
"top_k": top_k,
"raw_response": raw_response
}
]
predict_custom_trained_model_sample(
project="yourprojectid",
endpoint_id="yourprojectendpoint",
instances=instances
)
```

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

贡献指南

打开贡献指南

调研方向

首先定位 predict_custom_trained_model_sample 入口点和现有的自定义端点示例。查看所请求的 Llama 3.1 提示词结构和 REST API 负载,然后添加示例,展示 Vertex AI 管理的基础设施端点的这两种形式。完成的标准是用户可以按照示例构建提示词和负载。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
api, machine-learning
Issue 类型
文档
难度
3/5
预计耗时
1-2 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。