`just build-assets` fails on Linux when `npx` is missing and creates root-owned assets
- Dominant language
- Rust
- Stars
- 72
- Forks
- 13
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 5
Description
### Problem Description
On Linux hosts, running `just build-assets` can fail or cause subsequent permission issues due to three distinct bugs in the build chain:
1. **Missing `npx` dependency**: The builder backend uses `cdxgen` to generate the CycloneDX OBOM. It defaults to using `npx` to run it, which fails if Node/npm is installed without `npx` (common on some Debian/Ubuntu setups), even though `pnpm` (which is a project prerequisite) is available.
2. **Root-owned assets**: When building the EROFS rootfs, `mkfs.erofs` runs inside a Docker container. Because the output directory is mounted, the generated `rootfs.erofs` file is owned by `root:root` on Linux hosts. This prevents non-root host processes (like `just smoke` / `create_hash_assets.py`) from creating hardlinks to it, resulting in `PermissionError: Operation not permitted`.
3. **ReadOnly Binary Overwrite**: Native agent cross-compilation fails with `Permission Denied` when trying to overwrite existing read-only (555) binaries in `target/linux-agent/`.
### Steps to Reproduce
1. Ensure `npx` is not installed, but `pnpm` is.
2. Run `just build-assets code x86_64`. It will fail with `FileNotFoundError: [Errno 2] No such file or directory: 'npx'`.
3. (If `npx` is present or bypassed) Run `just build-assets code x86_64` as non-root on Linux.
4. Run `just smoke`. It will fail with `PermissionError: [Errno 1] Operation not permitted` when trying to link `rootfs.erofs`.
### Proposed Solution / Fixes Applied
We have implemented the following fixes in `src/capsem/builder/docker.py`:
1. **cdxgen Fallback**: Modified `_cdxgen_command` to check for `npx` availability and fallback to `pnpm dlx @cyclonedx/cdxgen@latest` if `npx` is missing.
2. **Restore Ownership**: Added a `chown` step inside the EROFS builder container to change the ownership of the output file back to the host user's UID:GID when running on Linux.
3. **Unlink before copy**: Updated `cross_compile_agent` to unlink the destination binary before copying to avoid permission errors when overwriting read-only files.
```patch
diff --git a/src/capsem/builder/docker.py b/src/capsem/builder/docker.py
index ea948bb7..abf418d7 100644
--- a/src/capsem/builder/docker.py
+++ b/src/capsem/builder/docker.py
@@ -458,6 +458,10 @@ def create_erofs(
level_flag = f",level={compression_level}" if compression_level else ""
mkdir_output = "" if out_dir == "." else f"mkdir -p /assets/{out_dir} && "
+ chown_cmd = ""
+ if sys.platform != "darwin" and hasattr(os, "getuid") and hasattr(os, "getgid"):
+ chown_cmd = f" && chown {os.getuid()}:{os.getgid()} /assets/{out_rel}"
+
run_cmd([
runtime, "run", "--rm",
"-v", f"{common_dir}:/assets",
@@ -467,10 +471,11 @@ def create_erofs(
f"DEBIAN_FRONTEND=noninteractive apt-get install -y erofs-utils && "
f"mkdir /rootfs && {mkdir_output}tar xf /assets/{tar_rel} -C /rootfs && "
f"mkfs.erofs -Enosbcrc -z{compression}{level_flag}{cluster_flag} "
- f"/assets/{out_rel} /rootfs",
+ f"/assets/{out_rel} /rootfs{chown_cmd}",
])
+
def erofs_utils_image_for(compression: str) -> str:
"""Return the container image used to create an EROFS image."""
if compression == "zstd":
@@ -633,7 +638,10 @@ def cross_compile_agent(
if not src.exists():
raise RuntimeError(f"Expected binary not found: {src}")
dst = output_dir / binary
+ if dst.exists():
+ dst.unlink()
shutil.copy2(str(src), str(dst))
+
if dst.stat().st_size == 0:
raise RuntimeError(f"Binary is empty: {dst}")
copied.append(dst)
@@ -717,11 +725,22 @@ def _cdxgen_command() -> list[str]:
default uses npm's package runner so the rootfs build does not depend on a
globally installed binary.
"""
- configured = os.environ.get("CAPSEM_CDXGEN_CMD", "npx --yes @cyclonedx/cdxgen@latest")
- command = shlex.split(configured)
- if not command:
- raise RuntimeError("CAPSEM_CDXGEN_CMD must not be empty")
- return command
+ configured = os.environ.get("CAPSEM_CDXGEN_CMD")
+ if configured:
+ command = shlex.split(configured)
+ if not command:
+ raise RuntimeError("CAPSEM_CDXGEN_CMD must not be empty")
+ return command
+
+ if shutil.which("npx"):
+ return ["npx", "--yes", "@cyclonedx/cdxgen@latest"]
+ elif shutil.which("pnpm"):
+ return ["pnpm", "dlx", "@cyclonedx/cdxgen@latest"]
+ else:
+ raise RuntimeError(
+ "Neither 'npx' nor 'pnpm' found in PATH. Cannot run cdxgen to generate OBOM."
+ )
+
def _validate_cyclonedx_obom(path: Path) -> None:
```
Contributor guide
Research direction
Start in src/capsem/builder/docker.py at create_erofs, cross_compile_agent, and _cdxgen_command; compare their behavior with the Linux reproduction steps. Run just build-assets code x86_64 followed by just smoke, including an environment without npx but with pnpm. Done means the build completes, generated assets are usable by non-root host processes, and read-only binaries can be rebuilt.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, linux, python
- Domain
- build-system, devops, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100