exadel-inc / exadel-inc/CompreFace

Error during synchronization between servers

Open
#1,087 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
8.3k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**

/api/v1/recognition/recognize response is sometimes 500:

{
"message" : "Error during synchronization between servers: [500 INTERNAL SERVER ERROR] during [POST] to [http://compreface-core:3000/find_faces_base64?limit=1&det_prob_threshold=0.5&face_plugins=calculator] [FacesFeignClient#findFacesBase64(FindFacesRequest,Integer,Double,String)]: [{\"message\":\"MXNetError: MXNetError: could not execute a primitive\"}\n]",
"code" : 41
}

{
"message" : "Error during synchronization between servers: [500 INTERNAL SERVER ERROR] during [POST] to [http://compreface-core:3000/find_faces_base64?limit=1&det_prob_threshold=0.5&face_plugins=calculator] [FacesFeignClient#findFacesBase64(FindFacesRequest,Integer,Double,String)]: [{\"message\":\"MXNetError: Traceback (most recent call last):\\n File \\\"../src/ndarray/ndarray.cc\\\", line 707\\nMXNetError: The size of NDArray doesn't match the requested MKLDNN memory desc. MKLDNN memory requests for 1140480 bytes, but got 737280 bytes from NDArray\"}\n]",
"code" : 41
}

**Expected behavior**

/api/v1/recognition/recognize to respond with 200

**Screenshots**

None

**Desktop (please complete the following information):**

- OS: Windows 10, Docker Desktop + WSL
- Browser N/A
- Version: SubCenter-ArcFace-r100 1.1.0

**Logs**

Logs for each service in separate files inside attached zip. compreface-core log is last 12 hours, since full log is 41GB

[SubCenter-ArcFace-r100-logs.zip](https://github.com/exadel-inc/CompreFace/files/11737084/SubCenter-ArcFace-r100-logs.zip)

**Additional context**

Have done around 40k requests to a recognition service and around 170k requests to a detection service with no issues, then this started to happen.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.