Azure-Samples / Azure-Samples/azureai-samples

Cloud Run does not work with AI Red Teaming Agent jupytner notebook sample

Open
#261 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Jupyter Notebook
Stars
548
Forks
318
Avg merge
3d 15h
Merged PRs (30d)
1

Description

### Operating System

Windows

### Version Information

Python Version: 3.11.5
azure.ai.evaluation package version: 1.8.0

### Steps to reproduce

Red teaming agent cloud run does not work with below error when run jupyter notebook code here: https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/develop/run-ai-red-teaming-cloud?tabs=python#create-an-ai-red-teaming-run

---------------------------------------------------------------------------
HttpResponseError Traceback (most recent call last)
Cell In[24], [line 50](vscode-notebook-cell:?execution_count=24&line=50)
[42](vscode-notebook-cell:?execution_count=24&line=42) red_team_agent = RedTeam(
[43](vscode-notebook-cell:?execution_count=24&line=43) attack_strategies=[AttackStrategy.BASE64],
[44](vscode-notebook-cell:?execution_count=24&line=44) risk_categories=[RiskCategory.VIOLENCE],
[45](vscode-notebook-cell:?execution_count=24&line=45) display_name="red-team-cloud-run",
[46](vscode-notebook-cell:?execution_count=24&line=46) target=target_config,
[47](vscode-notebook-cell:?execution_count=24&line=47) )
[49](vscode-notebook-cell:?execution_count=24&line=49) # Create and run the red teaming scan
---> [50](vscode-notebook-cell:?execution_count=24&line=50) red_team_response = project_client.red_teams.create(
[51](vscode-notebook-cell:?execution_count=24&line=51) red_team=red_team_agent,
[52](vscode-notebook-cell:?execution_count=24&line=52) headers={
[53](vscode-notebook-cell:?execution_count=24&line=53) "model-endpoint": model_endpoint,
[54](vscode-notebook-cell:?execution_count=24&line=54) "api-key": model_api_key,
[55](vscode-notebook-cell:?execution_count=24&line=55) },
[56](vscode-notebook-cell:?execution_count=24&line=56) )

File c:\git\Learn.AutoEval\.venv\Lib\site-packages\azure\core\tracing\decorator.py:119, in distributed_trace..decorator..wrapper_use_tracer(*args, **kwargs)
[117](file:///C:/git/Learn.AutoEval/.venv/Lib/site-packages/azure/core/tracing/decorator.py:117) # If tracing is disabled globally and user didn't explicitly enable it, don't trace.
[118](file:///C:/git/Learn.AutoEval/.venv/Lib/site-packages/azure/core/tracing/decorator.py:118) if user_enabled is False or (not tracing_enabled and user_enabled is None):
--> [119](file:///C:/git/Learn.AutoEval/.venv/Lib/site-packages/azure/core/tracing/decorator.py:119) return func(*args, **kwargs)
[121](file:///C:/git/Learn.AutoEval/.venv/Lib/site-packages/azure/core/tracing/decorator.py:121) # Merge span is parameter is set, but only if no explicit parent are passed
[122](file:///C:/git/Learn.AutoEval/.venv/Lib/site-packages/azure/core/tracing/decorator.py:122) if merge_span and not passed_in_parent:
...
[2857](file:///C:/git/Learn.AutoEval/.venv/Lib/site-packages/azure/ai/projects/operations/_operations.py:2857) deserialized = response.iter_bytes()

HttpResponseError: (UserError) RedTeam target is invalid
Code: UserError
Message: RedTeam target is invalid

### Expected behavior

Submit to red teaming agent to cloud run successfully

### Actual behavior

Meet exception:
HttpResponseError Traceback (most recent call last)
Cell In[24], [line 50](vscode-notebook-cell:?execution_count=24&line=50)
[42](vscode-notebook-cell:?execution_count=24&line=42) red_team_agent = RedTeam(
[43](vscode-notebook-cell:?execution_count=24&line=43) attack_strategies=[AttackStrategy.BASE64],
[44](vscode-notebook-cell:?execution_count=24&line=44) risk_categories=[RiskCategory.VIOLENCE],
[45](vscode-notebook-cell:?execution_count=24&line=45) display_name="red-team-cloud-run",
[46](vscode-notebook-cell:?execution_count=24&line=46) target=target_config,
[47](vscode-notebook-cell:?execution_count=24&line=47) )
[49](vscode-notebook-cell:?execution_count=24&line=49) # Create and run the red teaming scan
---> [50](vscode-notebook-cell:?execution_count=24&line=50) red_team_response = project_client.red_teams.create(
[51](vscode-notebook-cell:?execution_count=24&line=51) red_team=red_team_agent,
[52](vscode-notebook-cell:?execution_count=24&line=52) headers={
[53](vscode-notebook-cell:?execution_count=24&line=53) "model-endpoint": model_endpoint,
[54](vscode-notebook-cell:?execution_count=24&line=54) "api-key": model_api_key,
[55](vscode-notebook-cell:?execution_count=24&line=55) },
[56](vscode-notebook-cell:?execution_count=24&line=56) )

File c:\git\Learn.AutoEval\.venv\Lib\site-packages\azure\core\tracing\decorator.py:119, in distributed_trace..decorator..wrapper_use_tracer(*args, **kwargs)
[117](file:///C:/git/Learn.AutoEval/.venv/Lib/site-packages/azure/core/tracing/decorator.py:117) # If tracing is disabled globally and user didn't explicitly enable it, don't trace.
[118](file:///C:/git/Learn.AutoEval/.venv/Lib/site-packages/azure/core/tracing/decorator.py:118) if user_enabled is False or (not tracing_enabled and user_enabled is None):
--> [119](file:///C:/git/Learn.AutoEval/.venv/Lib/site-packages/azure/core/tracing/decorator.py:119) return func(*args, **kwargs)
[121](file:///C:/git/Learn.AutoEval/.venv/Lib/site-packages/azure/core/tracing/decorator.py:121) # Merge span is parameter is set, but only if no explicit parent are passed
[122](file:///C:/git/Learn.AutoEval/.venv/Lib/site-packages/azure/core/tracing/decorator.py:122) if merge_span and not passed_in_parent:
...
[2857](file:///C:/git/Learn.AutoEval/.venv/Lib/site-packages/azure/ai/projects/operations/_operations.py:2857) deserialized = response.iter_bytes()

HttpResponseError: (UserError) RedTeam target is invalid
Code: UserError
Message: RedTeam target is invalid

### Addition information

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.