apache / apache/hudi

Trim existing unit tests to finish in much shorter amount of time

Open
#14,737 0 comments 0 reactions 1 assignee Claimed by @yihua View on GitHub
area:tests from-jira priority:critical type:epic
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

spark-client-tests

278.165 s - in org.apache.hudi.table.TestHoodieMergeOnReadTable
201.628 s - in org.apache.hudi.metadata.TestHoodieBackedMetadata
185.716 s - in org.apache.hudi.client.TestHoodieClientOnCopyOnWriteStorage
158.361 s - in org.apache.hudi.index.TestHoodieIndex
156.196 s - in org.apache.hudi.table.TestCleaner
132.369 s - in org.apache.hudi.table.action.commit.TestCopyOnWriteActionExecutor
93.307 s - in org.apache.hudi.table.action.compact.TestAsyncCompaction
67.301 s - in org.apache.hudi.table.upgrade.TestUpgradeDowngrade
45.794 s - in org.apache.hudi.client.TestHoodieReadClient
38.615 s - in org.apache.hudi.index.bloom.TestHoodieBloomIndex
31.181 s - in org.apache.hudi.client.TestTableSchemaEvolution
20.072 s - in org.apache.hudi.table.action.compact.TestInlineCompaction

grep " Time elapsed" hudi-client/hudi-spark-client/target/surefire-reports/* | awk -F',' ' { print $5 } ' | awk -F':' ' { print $2 } ' | sort -nr | less

hudi-utilities

209.936 s - in org.apache.hudi.utilities.functional.TestHoodieDeltaStreamer
204.653 s - in org.apache.hudi.utilities.functional.TestHoodieMultiTableDeltaStreamer
34.116 s - in org.apache.hudi.utilities.sources.TestKafkaSource
29.865 s - in org.apache.hudi.utilities.sources.TestParquetDFSSource
26.189 s - in org.apache.hudi.utilities.sources.helpers.TestDatePartitionPathSelector

Other Tests

42.595 s - in org.apache.hudi.common.functional.TestHoodieLogFormat
38.918 s - in org.apache.hudi.common.bootstrap.TestBootstrapIndex
22.046 s - in org.apache.hudi.common.functional.TestHoodieLogFormatAppendFailure

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-1574
- Type: Epic
- Affects version(s):
- 0.9.0
- Fix version(s):
- 0.16.0
- 1.1.0

---

## Comments

02/Feb/21 22:05;vinoth;these are the ones that take up most time. if we can reduce these by speeding up setup etc, it would go a long way with the travis time crunch;;;

---

29/Mar/21 04:21;shivnarayan;we could do one simple fix.  TestHoodieMultiTableDeltaStreamer extends from  TestHoodieDeltaStreamer and hence I guess it runs all tests from TestHoodieDeltaStreamer again. ;;;

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.