Error 500 on heavy load.

I have experienced a strange behaviour over the REST interface using node-fetch module.

While this is not the most efficient way of doing things, it has thrown up the following error 500.

I read in around 11 ttl files so that total about 1,6m triples. I now want to pivot them and create a file per subject. I process each of the 1,6m triples as set out below. Check whether the resource exists. If it does not, create it using PUT, else PATCH in the new triple.

For logging I write out a 'pulse' every 5,000 records.

The code below is an extract of the process.

   let getResult = await fetch(newUrl, {method: "HEAD"})
//                console.log(getResult.status)
                if (getResult.status === 404) {
                    options = {
                        method: "PUT",
                        headers: {
                            "content-type" : "text/turtle",
                            "Link": "<http://www.w3.org/ns/ldp#Resource>; rel='type'"
                        },
                        body: `${newSubject} <${newPredicate}> ${newObject} .`
                   }
                }
                else {
                    options = {
                        method: "PATCH",
                        headers: {
                            "content-type" : "application/sparql-update",
                            "Link": "<http://www.w3.org/ns/ldp#Resource>; rel='type'"
                        },
                        body: `INSERT DATA { ${newSubject} <${newPredicate}> ${newObject} }`
                    }
                };
                let res1 = await fetch(newUrl, options);
                if ((res1.status != 200) && (res1.status != 201)) console.log("PROBLEMS WITH THE SOLID-SERVER", res1.status, JSON.stringify(options));
                if ( i % 5000 === 0) console.log("output", i, newUrl, res1.status);
 

Just after the 80,000th log entry and before the 85,000th log entry, the server starts producing error 500 and never recovers. As can be seen in the output below:

...
output 75000 https://127.0.0.1:8443/eim/App_Inst/714101 200
output 80000 https://127.0.0.1:8443/eim/App_Inst/876505 200
PROBLEMS WITH THE SOLID-SERVER 500 {"method":"PUT","headers":{"content-type":"text/turtle","Link":"<http://www.w3.org/ns/ldp#Resource>; rel='type'"},"body":"<> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://127.0.0.1:8443/eim/App_Instance_Review> ."}
PROBLEMS WITH THE SOLID-SERVER 500 {"method":"PUT","headers":{"content-type":"text/turtle","Link":"<http://www.w3.org/ns/ldp#Resource>; rel='type'"},"body":"<> <https://127.0.0.1:8443/eim/App_Inst_ID> \"\" ."}
...

Beginne bei der REST-Schnittstelle und reproduziere die node-fetch-Arbeitslast, die etwa 1,6 Millionen Tripel verarbeitet, unter Verwendung der gemeldeten PUT- und PATCH-Anfragen. Untersuche, warum der Server nach ungefähr 80.000 Datensätzen HTTP-500-Antworten zurückgibt und ob er sich wieder erholt; als abgeschlossen gilt die Aufgabe, wenn die Arbeitslast keine anhaltenden 500-Fehler mehr erzeugt und der Fehler durch einen geeigneten Test oder eine Reproduktion abgedeckt ist.