axetroy / axetroy/crawler

案例运行错误,RangeError: Use `yyyy` instead of `YYYY`(date-fns)

Open
#86 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
41
Forks
8
PR merge metrics
No merged PRs in 30d

Description

```ts
import { Crawler, Provider, Response, Options } from "@axetroy/crawler";

class MyProvider implements Provider {
name = "scrapinghub";
urls = ["https://blog.scrapinghub.com"];
async parse($: Response) {
const $nextPage = $("a.next-posts-link").eq(0);

if ($nextPage) {
$.follow($nextPage.prop("href"));
}

return $(".post-header>h2")
.map((_, el) => $(el).text())
.get();
}
}

const config: Options = {
timeout: 1000 * 5,
retry: 3,
};

new Crawler(MyProvider, config)
.on("data", (articles: string[]) => {
for (const article of articles) {
process.stdout.write(article + "\n");
}
})
.on("error", (err, task) => {
console.log(`request fail on ${task.url}: ${err.message}`);
})
.start();
```
error
```
(node:47382) UnhandledPromiseRejectionWarning: RangeError: Use `yyyy` instead of `YYYY` for formatting years; see: https://git.io/fxCyr
at throwProtectedError (/Users/guanwei/code/demo/spider/wb_xhs_media/node_modules/date-fns/_lib/protectedTokens/index.js:22:11)
at /Users/guanwei/code/demo/spider/wb_xhs_media/node_modules/date-fns/format/index.js:429:41
at Array.map ()
at Object.format (/Users/guanwei/code/demo/spider/wb_xhs_media/node_modules/date-fns/format/index.js:413:45)
at Printf.template (/Users/guanwei/code/demo/spider/wb_xhs_media/node_modules/@axetroy/crawler/src/logger.ts:16:18)
at Printf.transform (/Users/guanwei/code/demo/spider/wb_xhs_media/node_modules/logform/printf.js:11:26)
at Format.transform (/Users/guanwei/code/demo/spider/wb_xhs_media/node_modules/logform/combine.js:20:24)
at DerivedLogger._transform (/Users/guanwei/code/demo/spider/wb_xhs_media/node_modules/winston/lib/winston/logger.js:305:29)
at DerivedLogger.Transform._read (/Users/guanwei/code/demo/spider/wb_xhs_media/node_modules/readable-stream/lib/_stream_transform.js:177:10)
at DerivedLogger.Transform._write (/Users/guanwei/code/demo/spider/wb_xhs_media/node_modules/readable-stream/lib/_stream_transform.js:164:83)
(node:47382) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:47382) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at src/logger.ts, where date-fns formatting appears in the stack trace, and reproduce the example with the shown crawler configuration. Check the format token used there against date-fns's protected-token error; done means the example runs without this RangeError and the logger still emits timestamps.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.