apache / apache/incubator-seata

Bug: distribution/LICENSE files are manually maintained without unified standards, unverifiable and hard to maintain

Open
#8,193 0 comments 0 reactions 1 assignee Claimed by @xuxiaowei-com-cn View on GitHub
Dominant language
Java
Stars
26k
Forks
8.8k
Avg merge
1d 8h
Merged PRs (30d)
4

Description

### Check Ahead

- [x] I have searched the [issues](https://github.com/seata/seata/issues) of this repository and believe that this is not a duplicate.

- [x] I am willing to try to fix this bug myself.

### Ⅰ. Issue Description

1. In the 2.6.0 branch, the namingserver module uses org.apache.httpcomponents related dependencies.
2. In the 2.x branch (2.8.0-SNAPSHOT), the namingserver module does not use org.apache.httpcomponents related dependencies, but distribution/LICENSE-namingserver still contains org.apache.httpcomponents entries. The same applies to 2.7.0.

```shell
xuxiaowei@xiaomi  ~/IdeaProjects/github.com/xuxiaowei-com-cn/incubator-seata-3   2.6.0  mvn dependency:tree -pl namingserver | grep org.apache.httpcomponents
[INFO] +- org.apache.httpcomponents.client5:httpclient5:jar:5.4.3:compile
[INFO] | +- org.apache.httpcomponents.core5:httpcore5:jar:5.3.4:compile
[INFO] | \- org.apache.httpcomponents.core5:httpcore5-h2:jar:5.3.4:compile
[INFO] +- org.apache.httpcomponents:httpcore:jar:4.4.16:compile
[INFO] +- org.apache.httpcomponents:httpasyncclient:jar:4.1.5:compile
[INFO] | +- org.apache.httpcomponents:httpcore-nio:jar:4.4.16:compile
[INFO] | \- org.apache.httpcomponents:httpclient:jar:4.5.14:compile
xuxiaowei@xiaomi  ~/IdeaProjects/github.com/xuxiaowei-com-cn/incubator-seata-3   2.6.0  cat distribution/LICENSE-namingserver | grep org.apache.httpcomponents
org.apache.httpcomponents:httpasyncclient 4.1.5 Apache-2.0
org.apache.httpcomponents:httpclient 4.5.14 Apache-2.0
org.apache.httpcomponents:httpcore 4.4.16 Apache-2.0
org.apache.httpcomponents:httpcore-nio 4.4.16 Apache-2.0
org.apache.httpcomponents.core5:httpcore5 5.3.4 Apache-2.0
org.apache.httpcomponents.core5:httpcore5-h2 5.3.4 Apache-2.0
org.apache.httpcomponents.client5:httpclient5 5.4.3 Apache-2.0
xuxiaowei@xiaomi  ~/IdeaProjects/github.com/xuxiaowei-com-cn/incubator-seata-3   2.6.0 
xuxiaowei@xiaomi  ~/IdeaProjects/github.com/xuxiaowei-com-cn/incubator-seata-3   2.x  mvn dependency:tree -pl namingserver -am | grep org.apache.httpcomponents
✘ xuxiaowei@xiaomi  ~/IdeaProjects/github.com/xuxiaowei-com-cn/incubator-seata-3   2.x  cat distribution/LICENSE-namingserver | grep org.apache.httpcomponents
org.apache.httpcomponents:httpasyncclient 4.1.5 Apache-2.0
org.apache.httpcomponents:httpclient 4.5.14 Apache-2.0
org.apache.httpcomponents:httpcore 4.4.16 Apache-2.0
org.apache.httpcomponents:httpcore-nio 4.4.16 Apache-2.0
org.apache.httpcomponents.core5:httpcore5 5.3.4 Apache-2.0
org.apache.httpcomponents.core5:httpcore5-h2 5.3.4 Apache-2.0
org.apache.httpcomponents.client5:httpclient5 5.4.3 Apache-2.0
xuxiaowei@xiaomi  ~/IdeaProjects/github.com/xuxiaowei-com-cn/incubator-seata-3   2.x 
```

The `distribution/` directory contains three LICENSE files:

- `distribution/LICENSE`
- `distribution/LICENSE-namingserver`
- `distribution/LICENSE-server`

These files are currently **manually maintained** without a unified generation standard, leading to the following problems:

1. **Cannot be verified/reproduced** — Different people editing these files by hand results in inconsistent formats. There is no automated way to verify that the listed dependencies match the actual build dependencies, or that all transitive dependencies are included. When dependencies are upgraded or added, the LICENSE files may not be updated consistently.

2. **Not sorted by rules (e.g., by name)** — License sections and dependency entries within each section are not alphabetically sorted, making it difficult to:
- Find a specific dependency
- Review changes in diffs
- Maintain the files without introducing duplicates or missing entries

3. **Inconsistent across the three files** — For example:
- `LICENSE` — dependencies are not sorted by name, making the file disorganized
- `LICENSE-namingserver` — same as above
- `LICENSE-server` — same as above

### Ⅱ. Describe what happened

When a developer adds or upgrades a dependency, they must manually update one or more of these LICENSE files. Because:

- There's no documented procedure for which dependencies go in which LICENSE file
- There's no tool to validate that all dependencies are covered
- There's no enforced sorting convention
- The relationship between build dependencies and LICENSE entries is not traceable

...the resulting files become inconsistent over time, and PR reviews cannot easily verify the correctness of LICENSE changes.

### Ⅲ. Describe what you expected to happen

All three LICENSE files should be **auto-generated** by a standard tool, ensuring:

1. Dependencies are automatically extracted from the build configuration (e.g., `pom.xml`, `package.json`)
2. License information is resolved from SPDX or project metadata
3. Entries are sorted consistently (e.g., alphabetically by artifact name within each license category)
4. The generation is reproducible — running the same tool on the same source produces the same output
5. CI can verify that the committed LICENSE files match the auto-generated ones (e.g., via a CI check)

### Ⅳ. How to reproduce it (as minimally and precisely as possible)

1. Check out the current codebase on the `2.x` branch
2. Look at `distribution/LICENSE`, `distribution/LICENSE-namingserver`, `distribution/LICENSE-server`
3. Observe that:
- License sections are in different orders across the three files
- Dependencies within each section are not alphabetically sorted
- There is no tool/script to regenerate them

Example of unsorted entries in `distribution/LICENSE-server` MIT section:

```
org.checkerframework:checker-qual 3.37.0 MIT see:licenses/checker-qual-MIT
redis.clients:jedis 3.8.0 MIT see:licenses/jedis-MIT
org.slf4j:jul-to-slf4j 1.7.36 MIT see:licenses/jul-to-slf4j-MIT
org.slf4j:slf4j-api 1.7.36 MIT see:licenses/slf4j-api-MIT
com.github.andrewoma.dexx:dexx-collections 0.2 MIT see:licenses/dexx-collections-MIT
```

### Ⅴ. Anything else we need to know?

Use **[Apache SkyWalking Eyes (license-eye)](https://github.com/apache/skywalking-eyes/)** to auto-generate all LICENSE files.

**Key benefits:**

| Aspect | Current (Manual) | Proposed (license-eye) |
|-----------------|-------------------|-----------------------------------|
| Generation | Manual editing | `license-eye` — one command |
| Verification | No way to verify | `license-eye` — CI-friendly |
| Sorting | Ad-hoc | Consistent alphabetical ordering |
| Reproducibility | Not reproducible | Fully reproducible |
| Maintenance | Error-prone | Automated with dependency changes |

**Suggested steps:**

1. Add per-module license-eye configuration files (`.licenserc-distribution.yaml`, `.licenserc-namingserver.yaml`, `.licenserc-server.yaml`), each defining:
- Which dependencies to include/exclude for that module
- Output path for the corresponding LICENSE file
- Sorting and formatting rules
2. Create a generation script (e.g., `script/license/generate-license.py`) that wraps `license-eye dependency resolve` and formats the output into ASF-required LICENSE structures (Apache License header + subcomponent sections grouped by license type)
3. Regenerate all three LICENSE files using the generation script, with a Makefile target (e.g., `make generate-license-all`) for convenience
4. Add a CI check using `license-eye dependency check` to ensure future changes stay consistent
5. Document the process in the contributor guide

**Implementation details (from PR #8194):**

The implementation provides:

- **`script/license/generate-license.py`** — Core Python script that:
- Runs `license-eye dependency resolve` against per-module config files
- Normalizes license names to SPDX identifiers (20+ URL/text variants mapped)
- Supports manual license overrides for 17 dependencies that license-eye cannot auto-resolve
- Handles multi-licensing (dual-license entries → takes first per ASF convention)
- Groups dependencies by license type with consistent alphabetical ordering
- Handles bundled assets (fonts) that license-eye cannot resolve via `DISTRIBUTION_EXTRA_ENTRIES`
- Determines `see:licenses/xxx` references automatically

- **`script/license/generate-license.sh`** — Shell wrapper that ensures correct working directory and checks prerequisites

- **CI workflow** (`.github/workflows/license-checker.yaml`) split into two jobs:
- `check-license-header` — existing license header + dependency license checks
- `check-license-files` — **NEW**: builds the project, generates LICENSE files, verifies they are up to date, and uploads a `license-diff.diff` artifact on failure for easy patching

- **Makefile targets**: `generate-license-all`, `generate-license-namingserver`, `generate-license-server`, `generate-license-distribution`

### Ⅵ. Environment

- Seata version: 2.x branch
- Branch: `2.x`
- OS: macOS / Linux (any)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.