apache / apache/linkis

[Feature][Linkis]Optimize project build time - reduce compilation duration from 1 hour

Open
#5,321 2 comments 0 reactions 1 assignee Claimed by @kazutoiris View on GitHub
feature
Dominant language
Java
Stars
3.4k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

## Description / 问题描述

**English:**
The current Linkis project build time is relatively long, a full compilation may take a significant amount of time depending on the environment. This impacts developer productivity and CI/CD pipeline efficiency. We need to optimize the build process to improve compilation performance.

**中文:**
当前 Linkis 项目的编译时间较长,完整编译根据环境不同可能需要较长时间。这影响了开发者的工作效率和 CI/CD 流水线的执行效率。我们需要优化构建过程以提升编译性能。

## Current Behavior / 当前行为

**English:**
- Full project compilation takes a long time (varies by environment, typically 30-60+ minutes)
- Incremental builds are not fully optimized
- Maven dependency resolution can be slow on first build
- All tests run during default build, adding overhead

**中文:**
- 完整项目编译耗时较长(因环境而异,通常 30-60+ 分钟)
- 增量构建未完全优化
- 首次构建时 Maven 依赖解析可能较慢
- 默认构建时运行所有测试,增加了额外开销

## Expected Behavior / 期望行为

**English:**
- Significant reduction in full build time (target: 30-50% improvement)
- Faster incremental builds for better development iteration
- Optimized dependency resolution and caching
- Flexible build profiles for different scenarios (quick build, full build, CI build)

**中文:**
- 显著缩短完整构建时间(目标:提升 30-50%)
- 加快增量构建速度,改善开发迭代体验
- 优化依赖解析和缓存机制
- 提供灵活的构建配置文件以适应不同场景(快速构建、完整构建、CI 构建)

## Proposed Solutions / 建议解决方案

### 1. Maven Build Optimization / Maven 构建优化

**English:**
- Enable parallel module builds with `-T` option (e.g., `-T 1C` for 1 thread per CPU core)
- Consider Maven Daemon (mvnd) for faster JVM startup and build caching
- Optimize `pom.xml` plugin configurations to skip unnecessary phases
- Use local repository mirror or caching proxy for faster dependency resolution

**中文:**
- 使用 `-T` 选项启用并行模块构建(如 `-T 1C` 每 CPU 核心 1 线程)
- 考虑使用 Maven Daemon (mvnd) 加速 JVM 启动和构建缓存
- 优化 `pom.xml` 插件配置,跳过不必要的阶段
- 使用本地仓库镜像或缓存代理加速依赖解析

### 2. Build Profile Optimization / 构建配置优化

**English:**
- Create a `quick` profile that skips tests, javadoc, and source generation
- Create a `ci` profile optimized for CI/CD environments
- Document recommended build commands for different use cases

**中文:**
- 创建 `quick` 配置文件,跳过测试、javadoc 和源码生成
- 创建针对 CI/CD 环境优化的 `ci` 配置文件
- 为不同使用场景记录推荐的构建命令

### 3. Module Dependency Optimization / 模块依赖优化

**English:**
- Analyze and optimize inter-module dependencies
- Reduce unnecessary transitive dependencies
- Consider lazy module loading where applicable

**中文:**
- 分析并优化模块间依赖关系
- 减少不必要的传递依赖
- 在适用的情况下考虑延迟模块加载

### 4. Test Execution Optimization / 测试执行优化

**English:**
- Enable parallel test execution with `maven-surefire-plugin`
- Separate unit tests from integration tests
- Provide option to run tests selectively by module

**中文:**
- 使用 `maven-surefire-plugin` 启用并行测试执行
- 分离单元测试和集成测试
- 提供按模块选择性运行测试的选项

## Acceptance Criteria / 验收标准

**English:**
- [ ] Measurable improvement in build time (benchmark before and after)
- [ ] Add build profiles for quick/full/CI builds
- [ ] Document optimized build commands in developer guide
- [ ] No regression in build output or test coverage

**中文:**
- [ ] 构建时间有可衡量的提升(优化前后基准测试对比)
- [ ] 添加快速/完整/CI 构建的配置文件
- [ ] 在开发者指南中记录优化后的构建命令
- [ ] 构建产物和测试覆盖率无回归

## Environment / 环境信息

- Linkis Version: 1.x
- Java Version: JDK 8/11
- Build Tool: Maven 3.x
- OS: Linux/MacOS/Windows

> Note: Build time varies significantly based on hardware (CPU cores, memory, disk speed), network conditions, and local Maven repository state.
>
> 注意:构建时间因硬件配置(CPU 核心数、内存、磁盘速度)、网络条件和本地 Maven 仓库状态而有显著差异。

## References / 参考资料

- [Maven Parallel Builds](https://maven.apache.org/guides/mini/guide-multiple-modules.html)
- [Maven Daemon (mvnd)](https://github.com/apache/maven-mvnd)
- [Maven Surefire Parallel Execution](https://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html)
- [Linkis Build Guide](https://linkis.apache.org/docs/latest/development/build)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.