clearlydefined / clearlydefined/crawler

The ClearlyDefined crawler has a lot of exceptions from logs

Open
#426 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
60
Forks
38
Avg merge
1h 49m
Merged PRs (30d)
1

Description

There are a lot of exceptions in the ClearlyDefined crawler. This may slow down crawlers to consume the coordinates queue message. We could start to clean the exceptions from the top numbers.

```
exceptions
| where timestamp >= ago(90d)
| summarize count() by bin(timestamp, 1d)
| render timechart

exceptions
| where timestamp >= ago(80d)
| summarize count() by problemId
| order by count_ desc
```

![image](https://user-images.githubusercontent.com/20408623/114226765-e08cd680-9928-11eb-8b9d-7db5766da140.png)

![image](https://user-images.githubusercontent.com/20408623/114226792-eaaed500-9928-11eb-9d5b-a08bf604f801.png)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.