parse-community / parse-community/parse-server
Huge log files created and crashes the app after API scriptlog call
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 21.4k
- Forks
- 4.8k
- Avg merge
- 7h 45m
- Merged PRs (30d)
- 11
Description
Issue Description
Hi
My log file grow to 50MB and more for a single file.
A. how to prevent this from happening? (lets assume i want to continue verbose logs and thats the amount of data that i have)
B. Once going to the log section on the dashboard (or doing the API call, thats why its not a dashboard issue), my server CPU jumped high, node memory went over 2GB (which crashed my AWS t3.small server)
Steps to reproduce
Not an easy thing to do, unfortunately i can't share my file with you because of security details inside the log.
My data structure has something like a 1 MB data in each document (row) of a specific collection.
Calling the Class Get all many times will start to fill the log file.
Total log library size was 150MB (100+ files) with 2 very large files:
60MB of info.log and 40MB of error.log of 1 specific day.
Once you have that files call the API XXX/parse/scriptlog?level=info&n=100
eventually the call either return 504 or crash the server
Expected Results
Max log file size should be 5MB(configurable?)
Query logs should not crash or consume all memory.
Actual Outcome
60MB log file and crash the system
Environment Setup
-
Server
- parse-server version (Be specific! Don't say 'latest'.) : 3.10.0
- Operating System: AWS linux and Mac Os Catalina
- Hardware: t3.small which is 2CPU 2GB RAM
- Localhost or remote server? (AWS, Heroku, Azure, Digital Ocean, etc): AWS (happens on localhost as well)
-
Database
- MongoDB version: 3.4.0
- Storage engine: [FILL THIS OUT]
- Hardware: 4 cpu 8GB RAM
- Localhost or remote server? (AWS, mLab, ObjectRocket, Digital Ocean, etc): AWS EC2
Logs/Trace
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by examining the /parse/scriptlog?level=info&n=100 entry point and reproduce the request with the large info.log and error.log files described in the issue. Done means verbose logging can continue without unbounded log files, and querying the logs no longer causes excessive memory use, a 504, or a server crash.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- api, backend, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100