googleapis / googleapis/google-api-python-client

Option to skip per-method docstring generation in build() (memory / high-concurrency)

Đang mở
#2,779 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
8.9k
Fork
2.6k
Merge trung bình
2 ngày 28 phút
Pull request đã merge (30 ngày)
17

Mô tả

## Feature request: an option to skip per-method docstring generation in `build()`

### Problem

`build()` (and lazy sub-resource construction) generates a fully-expanded, recursively
pretty-printed prototype of each method's **response schema** and attaches it as
`method.__doc__` (`discovery.createMethod` → `schema.Schemas.prettyPrintSchema` /
`prettyPrintByName`). For APIs with large, deeply-nested schemas this is very expensive, and it
is paid **every time a service/resource is constructed**.

Concrete numbers from profiling Sheets v4 (`google-api-python-client==2.198.0`, Python 3.13),
measured with RSS (no tracemalloc, to avoid its overhead):

- Building the service, then touching one sub-resource (`service.spreadsheets()`, **no API
call**): **~66 MB**.
- Of that, ~99.9% is the docstring schema expansion — no-oping `prettyPrintSchema`/
`prettyPrintByName` drops it to **~1 MB**. The `.spreadsheets()` methods themselves are ~24 KB.
- The docstrings are only useful for interactive `help()`; in a server they are never read.

### Impact

In a concurrent server (a fresh service built per request, common with per-user credentials),
these allocations are **not shared** across in-flight requests. 8 concurrent Sheets requests
each build ~66 MB of docstrings simultaneously ≈ **530 MB peak**, which OOM-kills a
memory-limited container. This is the concurrent-peak sibling of the long-standing
reference-cycle memory issue in #535 (whose recommended fix — build/reuse a single service — is
not always feasible when credentials differ per request).

### Request

A supported way to skip docstring generation at build time, e.g.:

```python
build("sheets", "v4", credentials=creds, generate_docstrings=False)
# or a module/env toggle
```

Today the only options are to monkeypatch `Schemas.prettyPrintSchema`/`prettyPrintByName`
(fragile across versions) or fork. A first-class flag would let memory-constrained / high-
concurrency deployments opt out of documentation strings they never use.

### Environment

- `google-api-python-client==2.198.0`, Python 3.13
- Reproly: build any large-schema API (Sheets v4), touch a sub-resource, measure RSS; repeat
concurrently to see the multiplier.

Related: #535 (memory from repeated `build()` / reference cycles).

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Start at build() and trace discovery.createMethod, schema.Schemas.prettyPrintSchema, and prettyPrintByName to see where response-schema docstrings are generated during service and lazy sub-resource construction. Reproduce the Sheets v4 memory profile described in the issue; done means a supported build-time option skips that generation without affecting normal construction when enabled.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
python
Lĩnh vực
api, backend
Loại issue
Tính năng
Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức độ hoạt động
Ít trao đổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
52/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.