embulk / embulk/embulk-input-jdbc
Reduce log option
- Dominant language
- Java
- Stars
- 106
- Forks
- 76
- PR merge metrics
- No merged PRs in 30d
Description
Do you have some idea to reduce log volume of ingestion as a option?
Once I have ingestion for near the billion of records, There are so huge outputs generated.
I would like to have some option to reduce log volume. But `--log-level warn` or `--log-level error` is too much reduced.
```
2020-09-14 04:02:42.879 +0000 [INFO] (0017:task-0000): SQL: FETCH FORWARD 10000 FROM cur
2020-09-14 04:02:42.890 +0000 [INFO] (0017:task-0000): > 0.01 seconds
2020-09-14 04:02:42.891 +0000 [INFO] (0017:task-0000): Fetched 500 rows.
2020-09-14 04:02:42.893 +0000 [INFO] (0017:task-0000): Fetched 1,000 rows.
2020-09-14 04:02:42.896 +0000 [INFO] (0017:task-0000): Fetched 2,000 rows.
2020-09-14 04:02:42.903 +0000 [INFO] (0017:task-0000): Fetched 4,000 rows.
2020-09-14 04:02:42.916 +0000 [INFO] (0017:task-0000): Fetched 8,000 rows.
2020-09-14 04:02:42.921 +0000 [INFO] (0017:task-0000): SQL: FETCH FORWARD 10000 FROM cur
2020-09-14 04:02:42.931 +0000 [INFO] (0017:task-0000): > 0.01 seconds
2020-09-14 04:02:42.933 +0000 [INFO] (0017:task-0000): Fetched 500 rows.
2020-09-14 04:02:42.935 +0000 [INFO] (0017:task-0000): Fetched 1,000 rows.
2020-09-14 04:02:42.938 +0000 [INFO] (0017:task-0000): Fetched 2,000 rows.
2020-09-14 04:02:42.944 +0000 [INFO] (0017:task-0000): Fetched 4,000 rows.
2020-09-14 04:02:42.957 +0000 [INFO] (0017:task-0000): Fetched 8,000 rows.
2020-09-14 04:02:42.962 +0000 [INFO] (0017:task-0000): SQL: FETCH FORWARD 10000 FROM cur
2020-09-14 04:02:42.971 +0000 [INFO] (0017:task-0000): > 0.01 seconds
2020-09-14 04:02:42.974 +0000 [INFO] (0017:task-0000): Fetched 500 rows.
2020-09-14 04:02:42.976 +0000 [INFO] (0017:task-0000): Fetched 1,000 rows.
2020-09-14 04:02:42.979 +0000 [INFO] (0017:task-0000): Fetched 2,000 rows.
2020-09-14 04:02:42.985 +0000 [INFO] (0017:task-0000): Fetched 4,000 rows.
2020-09-14 04:02:43.000 +0000 [INFO] (0017:task-0000): Fetched 8,000 rows.
2020-09-14 04:02:43.006 +0000 [INFO] (0017:task-0000): SQL: FETCH FORWARD 10000 FROM cur
2020-09-14 04:02:43.113 +0000 [INFO] (0017:task-0000): > 0.11 seconds
2020-09-14 04:02:43.116 +0000 [INFO] (0017:task-0000): Fetched 500 rows.
2020-09-14 04:02:43.118 +0000 [INFO] (0017:task-0000): Fetched 1,000 rows.
2020-09-14 04:02:43.121 +0000 [INFO] (0017:task-0000): Fetched 2,000 rows.
2020-09-14 04:02:43.127 +0000 [INFO] (0017:task-0000): Fetched 4,000 rows.
2020-09-14 04:02:43.140 +0000 [INFO] (0017:task-0000): Fetched 8,000 rows.
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.