caikit / caikit/caikit

libgomp: Thread creation failed: Resource temporarily unavailable

Open
#622 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
114
Forks
70
PR merge metrics
No merged PRs in 30d

Description

## Describe the bug

Under high concurrency testing, latency got slower, timeouts started to happen, and we get intermittent " libgomp: Thread creation failed: Resource temporarily unavailable".

Not sure how best to characterize the "concurrency", but on CPU I'd say we test w/ 100 concurrent requests and see latency degrade and at 500 we start getting errors.

This is a regression that showed up in 0.25.6.

> **NOTE! It looks like configuring `server_thread_pool_size: 5` fixes it. In 0.25.6 that default was changed from 5 to 100 (related to HTTP issues). We are only using gRPC.**

## Platform

Please provide details about the environment you are using, including the following:

- Interpreter version:
- Library version: Comparing 0.25.x versions

## Sample Code

Please include a minimal sample of the code that will (if possible) reproduce the bug in isolation

## Expected behavior

In my case, it seems like the default should be the better performing 5.

Ideally limits would prevent the resource error from ever happening (but don't know how possible that is to fix).

## Observed behavior

What you see happening (error messages, stack traces, etc...)

## Additional context

Add any other context about the problem here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.