apache / apache/lucene

Add -Ptests.asyncprofile option. [LUCENE-10099]

Open
#11,137 4 comments 0 reactions 1 assignee Claimed by @markrmiller View on GitHub
legacy-jira-priority:Minor module:test-framework type:test
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Allow async-profiler in addition to the JFR profiler.

async profiler description:
```
This project is a low overhead sampling profiler for Java that does not suffer from Safepoint bias problem.
It features HotSpot-specific APIs to collect stack traces and to track memory allocations.
The profiler works with OpenJDK, Oracle JDK and other Java runtimes based on the HotSpot JVM.
async-profiler can trace the following kinds of events:

* CPU cycles
* Hardware and Software performance counters like cache misses, branch misses, page faults, context switches etc.
* Allocations in Java Heap
* Contented lock attempts, including both Java object monitors and ReentrantLocks
```
 

The "tests.asyncprofile" mode switch turns on the async-profiler () during test execution,
and prints a simple summary at the end.

For example, top-10 histogram of methods (cpu samples):
```
gradlew -p lucene/core test -Ptests.asyncprofile=true
```
Alternatively, you can profile heap allocations instead: TBD

**Requirements**
- You need to download async-profiler 2.x ()
- You need the .so lib in a standard shared folder or one of the directories from LD_LIBRARY_PATH (Linux, though not Arch by
default), DYLD_LIBRARY_PATH (Mac OS), or -Djava.library.path. I think ${HOME}/libs also works
on Linux. We could also allow specifying the location directly as an option.
- Debug version of Java is important for best results and is required for heap allocation profiling. Available as package on Ubuntu, can be built with a small modification on Arch.

The following permissions need to be set to allow for the proper permissions for profiling:
```
# sysctl kernel.perf_event_paranoid=1
# sysctl kernel.kptr_restrict=0
```

---
Migrated from [LUCENE-10099](https://issues.apache.org/jira/browse/LUCENE-10099) by Mark Robert Miller (@markrmiller), updated Sep 22 2021
Linked issues:
- #11154

Pull requests: https://github.com/apache/lucene/pull/295

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.