microsoft / microsoft/typespec
[Bug]: Python emitter does not escape triple quotes in enum member documentation
- Dominant language
- Java
- Stars
- 5.9k
- Forks
- 394
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 104
Description
## Describe the bug
The Python emitter does not correctly escape embedded `"""` sequences in enum member `@doc` text when generating member docstrings in `models/_enums.py`.
An embedded triple-quote sequence terminates the generated docstring early. Subsequent documentation text is then interpreted as Python source rather than documentation. Depending on that text, generation either fails during formatting or succeeds with unintended class-body statements in the generated enum. Those statements execute when the generated module is imported.
This is being reported as a product correctness defect involving trusted TypeSpec input, not as a security vulnerability. Documentation should remain documentation in generated Python, including when it contains Python string delimiters.
### Affected versions reported
- `@azure-tools/typespec-python` 0.63.5
- `@typespec/http-client-python` 0.36.0, the underlying emitter engine
- `@typespec/compiler` 1.15.0
The reported reproduction uses default emitter options and completes generation with exit code 0 when the resulting Python is syntactically valid. These are the versions measured in the supplied report, not a claim about the latest releases.
## Reproduction
The following inline example describes the triggering input. It substitutes a harmless print statement for the shell command used in the supplied report; this adapted example has not been independently rerun.
1. Install the reported emitter and compiler versions in a TypeSpec service project.
```sh
npm install @azure-tools/typespec-python@0.63.5 @typespec/compiler@1.15.0
```
2. Include this enum in the service specification and reference it from an operation request or response. The API-surface reference matters because unused enums can be pruned from generated output.
```typespec
enum WidgetMode {
@doc("A\"\"\" ; print(\"This should remain documentation\") ; \"\"\"B")
Fast: "fast",
Slow: "slow",
}
```
3. Configure the emitter without custom options.
```yaml
emit:
- "@azure-tools/typespec-python"
options: {}
```
4. Compile the service specification with `npx tsp compile .` and inspect the generated `models/_enums.py` beneath the emitted package.
The defect places the text following the embedded delimiter outside the docstring. For the benign example above, the corresponding output would have this structure inside the enum class:
```python
FAST = "fast"
"""A"""
print("This should remain documentation")
"""B."""
```
The first embedded `"""` closes the member docstring after `A`. The print statement is now a class-body statement, and the second embedded delimiter starts a separate string containing the remainder of the documentation. Importing a module with this output prints the message instead of retaining it as documentation.
### Actual behavior reported
- When the text outside the prematurely closed docstring forms valid Python, generation succeeds. The bundled Black formatter accepts and reformats those statements rather than detecting that they originated in documentation.
- The supplied report observed the unintended class-body statements executing on import in three out of three runs. A control using documentation without triple quotes did not produce those effects.
- When the resulting Python is invalid, formatting fails and generation aborts instead.
- Plain newlines and U+2028, U+2029, and U+0085 were also tested in the supplied report and did not reproduce this delimiter problem. The issue here is specifically the embedded triple-quote sequence.
### Expected behavior
Embedded triple quotes are escaped or otherwise safely represented so that the complete documentation remains inside the generated docstring. Documentation content must not become Python statements or cause generated Python syntax errors.
## Relevant implementation and suggested fix
The enum member rendering site is in [`enum.py.jinja2`](https://github.com/microsoft/typespec/blob/main/packages/http-client-python/generator/pygen/codegen/templates/enum.py.jinja2). It places the wrapped member description between triple-double-quote delimiters:
```jinja2
"""{{ op_tools.wrap_string(value.description(is_operation_file=False), "\n ") }}"""
```
Correct the escaping at this documentation-rendering boundary, preferably with a shared Python docstring helper for other templates that use the same quoting pattern. Preserve intended documentation content, including quotes, backslashes, and line breaks.
This is separate from enum member value serialization. Fixing the writer for an enum's string value alone would not address its `@doc` text.
Suggested regression coverage:
- Enum member documentation containing embedded `"""`.
- Quotes combined with backslashes and multiline documentation.
- Documentation that would become valid Python statements if incorrectly emitted outside the docstring. Formatter success alone is not sufficient to establish correct output.
- A control without embedded delimiters, preserving existing documentation behavior.
Contributor guide
Research direction
Start with packages/http-client-python/generator/pygen/codegen/templates/enum.py.jinja2 and inspect how member descriptions become docstrings in generated models/_enums.py. Run the supplied TypeSpec reproduction with npx tsp compile ., then add regression coverage for embedded triple quotes, quotes, backslashes, multiline text, and a control case. Done means the complete documentation remains a docstring and importing generated Python does not execute its contents.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100