500 - Internal Server Error

The web server was running fine for days, then I added a zip of some source code (curl) to test the source code static analysis plugin. I selected all plugin checkmarks.
When I came back a few hours later, the webpage show the error:

```
Internal Server Error
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
```
The docker logs showed many errors like this:
```
[2021-10-30 16:16:19][docker][WARNING]: [source_code_analysis]: encountered process error while processing
[2021-10-30 16:16:20][docker][WARNING]: [source_code_analysis]: encountered process error while processing
ERROR: Could not connect to clamd on LocalSocket /var/run/clamav/clamd.ctl: No such file or directory
ERROR: Could not connect to clamd on LocalSocket /var/run/clamav/clamd.ctl: No such file or directory
ERROR: Could not connect to clamd on LocalSocket /var/run/clamav/clamd.ctl: No such file or directory
ERROR: Could not connect to clamd on LocalSocket /var/run/clamav/clamd.ctl: No such file or directory
[2021-10-30 16:16:21][docker][WARNING]: [source_code_analysis]: encountered process error while processing
```
for a long time leading up to a more fatal error like this:

```
[2021-10-30 16:16:22][Analysis][INFO]: Analysis Completed:
UID: 914684dae82a1ef21e8343a7a5edd90850916439c1a250b9d164bd16258ec37e_3250
Processed analysis: ['unpacker', 'file_type', 'crypto_hints', 'binwalk', 'users_and_passwords', 'printable_strings', 'malware_scanner', 'software_components', 'ip_and_uri_finder', 'crypto_material', 'qemu_exec', 'file_system_metadata', 'input_vectors', 'interesting_uris', 'string_evaluator', 'source_code_analysis', 'exploit_mitigations', 'cpu_architecture', 'elf_analysis', 'file_hashes', 'cve_lookup', 'init_systems', 'kernel_config', 'hardware_analysis', 'cwe_checker', 'known_vulnerabilities', 'tlsh']
Files included: set()
[2021-10-30 16:16:41][db_interface_backend][ERROR]: Update of analysis failed badly (connection closed)
Process ExceptionSafeProcess-54:
[2021-10-30 16:16:41][db_interface_statistic][ERROR]: Could not store statistic backend (localhost:27018: [Errno 104] Connection reset by peer)
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/pymongo/pool.py", line 710, in command
return command(self, dbname, spec, secondary_ok,
File "/usr/local/lib/python3.8/dist-packages/pymongo/network.py", line 152, in command
reply = receive_message(sock_info, request_id)
File "/usr/local/lib/python3.8/dist-packages/pymongo/network.py", line 200, in receive_message
_receive_data_on_socket(sock_info, 16, deadline))
File "/usr/local/lib/python3.8/dist-packages/pymongo/network.py", line 291, in _receive_data_on_socket
chunk_length = sock_info.sock.recv_into(mv[bytes_read:])
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/FACT_core/src/storage/db_interface_statistic.py", line 37, in update_statistic
self.statistic.insert_one(content_dict)
File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 705, in insert_one
self._insert(document,
File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 620, in _insert
return self._insert_one(
File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 609, in _insert_one
self.__database.client._retryable_write(
File "/usr/local/lib/python3.8/dist-packages/pymongo/mongo_client.py", line 1552, in _retryable_write
return self._retry_with_session(retryable, func, s, None)
File "/usr/local/lib/python3.8/dist-packages/pymongo/mongo_client.py", line 1438, in _retry_with_session
return self._retry_internal(retryable, func, session, bulk)
File "/usr/local/lib/python3.8/dist-packages/pymongo/mongo_client.py", line 1470, in _retry_internal
return func(session, sock_info, retryable)
File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 597, in _insert_command
result = sock_info.command(
File "/usr/local/lib/python3.8/dist-packages/pymongo/pool.py", line 726, in command
self._raise_connection_failure(error)
File "/usr/local/lib/python3.8/dist-packages/pymongo/pool.py", line 929, in _raise_connection_failure
_raise_connection_failure(self.address, error)
File "/usr/local/lib/python3.8/dist-packages/pymongo/pool.py", line 247, in _raise_connection_failure
raise AutoReconnect(msg)
pymongo.errors.AutoReconnect: localhost:27018: [Errno 104] Connection reset by peer
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/opt/FACT_core/src/helperFunctions/process.py", line 57, in run
raise exception
File "/opt/FACT_core/src/helperFunctions/process.py", line 52, in run
Process.run(self)
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/opt/FACT_core/src/scheduler/Analysis.py", line 391, in result_collector
self.post_analysis(fw)
File "/opt/FACT_core/src/storage/db_interface_backend.py", line 154, in add_analysis
self._update_analysis(file_object, analysis_system, processed_analysis[analysis_system])
File "/opt/FACT_core/src/storage/db_interface_backend.py", line 170, in _update_analysis
raise exception
File "/opt/FACT_core/src/storage/db_interface_backend.py", line 162, in _update_analysis
collection.update_one(
File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 1028, in update_one
self._update_retryable(
File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 877, in _update_retryable
return self.__database.client._retryable_write(
File "/usr/local/lib/python3.8/dist-packages/pymongo/mongo_client.py", line 1552, in _retryable_write
return self._retry_with_session(retryable, func, s, None)
File "/usr/local/lib/python3.8/dist-packages/pymongo/mongo_client.py", line 1438, in _retry_with_session
return self._retry_internal(retryable, func, session, bulk)
File "/usr/local/lib/python3.8/dist-packages/pymongo/mongo_client.py", line 1470, in _retry_internal
return func(session, sock_info, retryable)
File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 869, in _update
return self._update(
File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 838, in _update
result = sock_info.command(
File "/usr/local/lib/python3.8/dist-packages/pymongo/pool.py", line 726, in command
self._raise_connection_failure(error)
File "/usr/local/lib/python3.8/dist-packages/pymongo/pool.py", line 710, in command
return command(self, dbname, spec, secondary_ok,
File "/usr/local/lib/python3.8/dist-packages/pymongo/network.py", line 152, in command
reply = receive_message(sock_info, request_id)
File "/usr/local/lib/python3.8/dist-packages/pymongo/network.py", line 200, in receive_message
_receive_data_on_socket(sock_info, 16, deadline))
File "/usr/local/lib/python3.8/dist-packages/pymongo/network.py", line 297, in _receive_data_on_socket
raise AutoReconnect("connection closed")
pymongo.errors.AutoReconnect: connection closed
[2021-10-30 16:16:42][process][ERROR]: Exception in Unpacking process:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/pymongo/pool.py", line 710, in command
return command(self, dbname, spec, secondary_ok,
File "/usr/local/lib/python3.8/dist-packages/pymongo/network.py", line 152, in command
reply = receive_message(sock_info, request_id)
File "/usr/local/lib/python3.8/dist-packages/pymongo/network.py", line 200, in receive_message
_receive_data_on_socket(sock_info, 16, deadline))
File "/usr/local/lib/python3.8/dist-packages/pymongo/network.py", line 291, in _receive_data_on_socket
chunk_length = sock_info.sock.recv_into(mv[bytes_read:])
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/FACT_core/src/helperFunctions/process.py", line 52, in run
Process.run(self)
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/opt/FACT_core/src/scheduler/Unpacking.py", line 69, in unpack_worker
extracted_objects = unpacker.unpack(fo)
File "/opt/FACT_core/src/unpacker/unpack.py", line 42, in unpack
extracted_file_objects = self.generate_and_store_file_objects(extracted_files, Path(tmp_dir.name) / 'files', current_fo)
File "/opt/FACT_core/src/unpacker/unpack.py", line 83, in generate_and_store_file_objects
self.db_interface.set_unpacking_lock(current_file.uid)
File "/opt/FACT_core/src/storage/db_interface_common.py", line 314, in set_unpacking_lock
self.locks.insert_one({'uid': uid})
File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 705, in insert_one
self._insert(document,
File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 620, in _insert
return self._insert_one(
File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 609, in _insert_one
self.__database.client._retryable_write(
File "/usr/local/lib/python3.8/dist-packages/pymongo/mongo_client.py", line 1552, in _retryable_write
return self._retry_with_session(retryable, func, s, None)
File "/usr/local/lib/python3.8/dist-packages/pymongo/mongo_client.py", line 1438, in _retry_with_session
return self._retry_internal(retryable, func, session, bulk)
File "/usr/local/lib/python3.8/dist-packages/pymongo/mongo_client.py", line 1470, in _retry_internal
return func(session, sock_info, retryable)
File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 597, in _insert_command
result = sock_info.command(
File "/usr/local/lib/python3.8/dist-packages/pymongo/pool.py", line 726, in command
self._raise_connection_failure(error)
File "/usr/local/lib/python3.8/dist-packages/pymongo/pool.py", line 929, in _raise_connection_failure
_raise_connection_failure(self.address, error)
File "/usr/local/lib/python3.8/dist-packages/pymongo/pool.py", line 247, in _raise_connection_failure
raise AutoReconnect(msg)
pymongo.errors.AutoReconnect: localhost:27018: [Errno 104] Connection reset by peer

[2021-10-30 16:16:42][db_interface_statistic][ERROR]: Could not store statistic database (connection closed)
Traceback (most recent call last):
```

The error seems to be related to mongodb. It looks like it stopped working at some point because I don't see port 27018 in the `netstat` output anymore.

I also see messages like this in the logs

```
[2021-10-31 00:47:06][docker][WARNING]: [source_code_analysis]: encountered process error while processing
[2021-10-31 00:47:06][docker][WARNING]: [source_code_analysis]: encountered process error while processing
```

Start by tracing the failures from scheduler/Analysis.py, storage/db_interface_backend.py, storage/db_interface_statistic.py, and unpacker/unpack.py, using the reported MongoDB connection resets and source_code_analysis errors. Reproduce the zip upload with all plugins enabled and inspect the Docker logs. Done means the analysis completes without the web server returning a 500 or losing its MongoDB connection.

Written by the indexing model from the issue text.