NVIDIA / NVIDIA/cudf

[FEA] Implement cudf.pandas Profiler using sys.monitoring

Open
#17,519 1 comment 1 reaction 0 assignees View on GitHub
cudf.pandas feature request Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
Currently the `cudf.pandas.Profiler` uses `sys.settrace` to profile lines or cudf.pandas fast-slow calls. Currently the downsides of this approach is:

1. The tracing function is called for every frame while we're only interested in certain events (line and function calls) providing unnecessary overhead otherwise.
2. Maintaining some complexity in a monolithic tracing function.

**Describe the solution you'd like**
The `sys.monitoring` namespace, a new Python 3.12 feature, should be able to address both downsides: https://docs.python.org/3/library/sys.monitoring.html

1. We can register a callback for specific events only (`sys.monitoring.events.LINE`, `sys.monitoring.events.CALL`)
2. We can isolate profiling logic for each event

```python
@contextmanager
def profile(function_profile, line_profile, fn):
if line_profile:
sys.monitoring.register_callback(sys.monitoring.PROFILER_ID, sys.monitoring.events.LINE, line_profiler)
elif function_profile:
sys.monitoring.register_callback(sys.monitoring.PROFILER_ID, sys.monitoring.events.CALL, function_profiler)

```
Additionally we may find opportunities to improve profiling performance by disabling the profiling by having the callback return `sys.monitoring.DISABLE`.

**Describe alternatives you've considered**
Status quo

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.