microsoft / microsoft/typespec

Python emitter: generated docstrings can contain invalid Python escape sequences (e.g. \W) causing SyntaxWarning on 3.12+

Open
#10,784 1 comment 0 reactions 1 assignee Claimed by @msyyc View on GitHub
bug emitter:client:python
Dominant language
Java
Stars
5.9k
Forks
394
Avg merge
1d 23h
Merged PRs (30d)
104

Description

## Bug
The TypeSpec Python emitter (`@azure-tools/typespec-python` / `@typespec/http-client-python`) writes model/property doc strings verbatim into the generated Python source as the body of a docstring. When the original `@doc(...)` contains a backslash followed by a character that is not a recognized Python string escape (e.g. `\W`, `\d`, `\s`, etc.), the resulting Python source triggers:

```
SyntaxWarning: invalid escape sequence ''\W''
```

on Python 3.12+ (this will become a `SyntaxError` in a future Python version).

## Repro

TypeSpec source (excerpt) — `specification/compute/resource-manager/Microsoft.Compute/Compute/Compute/models.tsp` in `Azure/azure-rest-api-specs`:

```tsp
model OSProfile {
@doc("Specifies the password ... Has a special character (Regex match [\\W_]) ...")
@secret
adminPassword?: string;
}
```

The TypeSpec string value here is the literal `... (Regex match [\W_]) ...` (one backslash + `W`), which is correct for the swagger/REST description.

The generated Python (`azure-mgmt-compute==38.0.0`, `azure/mgmt/compute/models/_models.py`) contains:

```python
class OSProfile(_Model):
"""...
:ivar admin_password: ... Has a special character (Regex match [\W_]) ...
...
"""
```

Importing the module on Python 3.12 produces (issue [Azure/azure-sdk-for-python#47011](https://github.com/Azure/azure-sdk-for-python/issues/47011)):

```
SyntaxWarning: invalid escape sequence ''\W''

Has upper characters
Has a digit
Has a special character (Regex match [\W_])
```

## Expected
When emitting a Python docstring, the Python emitter should ensure the resulting Python string literal is valid — either by:

1. Writing the docstring as a raw string (`r"""..."""`), or
2. Escaping every backslash in the doc text (`\` → `\\`) before writing it into the Python source.

This should be done unconditionally for all generated docstrings (class docstrings, attribute `:ivar`/`:vartype`/`:keyword` lines, operation parameter docs, etc.), because users cannot reasonably be expected to pre-escape every `@doc` in TypeSpec just for the Python emitter — and doing so would corrupt the REST API description / other-language SDKs.

## Current workaround
Use `@@clientDoc(..., DocumentationMode.replace, "python")` in `client.tsp` to override the Python doc text with manually double-escaped backslashes. Example PR: https://github.com/Azure/azure-rest-api-specs/pull/43481

This is fragile — it requires every service team to discover the warning, then duplicate and hand-escape long doc strings just for Python.

## Environment
- Affected package: `azure-mgmt-compute==38.0.0` (generated)
- Python: 3.12+
- Original issue: https://github.com/Azure/azure-sdk-for-python/issues/47011

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.