googleapis / googleapis/python-aiplatform

Adding examples for correct prompt structure for custom endpoints on Vertex AI

オープン
#4,264 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
api: vertex-ai
主要言語
Python
スター
905
フォーク
465
平均マージ
1日 13時間
マージ済み PR(30日)
44

説明

Thanks for stopping by to let us know something could be better!

**PLEASE READ**: If you have a support contract with Google, please create an issue in the [support console](https://cloud.google.com/support/) instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

- Search the issues already opened: https://github.com/googleapis/python-aiplatform/issues
- Search StackOverflow: https://stackoverflow.com/questions/tagged/google-cloud-platform+python

If you are still having issues, please be sure to include as much information as possible:

#### Environment details

- OS type and version: Jupyter Notebook 6.5.5 on Colab enterprise
- Python version: `python --version` Python 3.10.12
- pip version: `pip --version`
- `google-cloud-aiplatform` version: `pip show google-cloud-aiplatform`

#### Steps to reproduce

1. Add Python def with the correct promot structure to pass on to open source models. I.e Llama 3.1
2. Add example REST API payload to prompt deployed endpoint on Vertex AI

#### Code example for Llama 3.1 managed infra endpoints

```python
prompt_payload = " What is the capital of Paris?" # @param {type:"string"}
prompt ="<|start_header_id|>user<|end_header_id|>"+prompt_payload+"<|eot_id|><|start_header_id|>assistant<|end_header_id|>"
max_tokens = 8000 # @param {type:"integer"}
temperature = 1.0 # @param {type:"number"}
top_p = 1.0 # @param {type:"number"}
top_k = 1.0 # @param {type:"number"}
raw_response = True # @param {type:"boolean"}

# Overides parameters for inferences.
# If you encounter the issue like `ServiceUnavailable: 503 Took too long to respond when processing`,
# you can reduce the max length, such as set max_tokens as 20.
instances = [
{
"prompt": prompt,
"max_tokens": max_tokens,
"temperature": temperature,
"top_p": top_p,
"top_k": top_k,
"raw_response": raw_response
}
]
predict_custom_trained_model_sample(
project="yourprojectid",
endpoint_id="yourprojectendpoint",
instances=instances
)
```

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。