apache / apache/iggy

Native Windows build of `server` fails: hwloc/pkg-config caveat + Unix-only APIs in `journal`/`message_bus`

Open
#4,037 7 comments 0 reactions 1 assignee Claimed by @TalaatHarb View on GitHub
bug rust server
Dominant language
Rust
Stars
4.9k
Forks
432
Avg merge
2d 10h
Merged PRs (30d)
173

Description

### Description

## Motivation

Apache Iggy currently has no documented, working path to compile `iggy-server` natively on Windows (MSVC toolchain). Docker is the only reliable option today. A native Windows binary would remove that friction for Windows-based contributors and evaluators and would likely help adoption — Windows developers could `cargo build`/`cargo run` the server directly, without Docker Desktop/WSL2, the same way `iggy-cli` and the SDK already work natively. This issue documents every blocker found while investigating (AI disclosure: GitHub Copilot helped in the investigation and in drafting the issue), in the hope it's useful to whoever picks this up. No code changes are proposed here, only findings (might not be comprehensive)+ suggested directions.

Tested with the repo-pinned toolchain (`rust-toolchain.toml` = 1.98.0) on `x86_64-pc-windows-msvc`.

## Part 1 — `hwlocality-sys` / pkg-config (already resolvable today)

`cargo check -p server --bin iggy-server` fails out of the box with:

```
thread 'main' panicked at hwlocality-sys-.../build.rs:...:
Could not find a suitable version of hwloc: ... pkg-config could not be found.
```

This comes from the optional NUMA-awareness dependency `hwlocality`/`hwlocality-sys` (see `core/server/Cargo.toml`), whose build script (`hwlocality-sys-0.7.1/build.rs`) calls `pkg_config::Config::new().probe("hwloc")`. Neither `pkg-config` nor `hwloc` ship with a default Windows/MSVC setup.

**This part is fully solvable** with [vcpkg](https://github.com/microsoft/vcpkg):

```powershell
git clone https://github.com/microsoft/vcpkg D:\tools\vcpkg
D:\tools\vcpkg\bootstrap-vcpkg.bat -disableMetrics

# hwloc's port builds via MSYS2/autotools even on Windows, which needs real
# scratch space; if TEMP/TMP point at a low-space drive you can hit
# "No space left on device" from aclocal/m4. Redirect if needed:
$env:TEMP = "D:\some\writable\path"
$env:TMP = "D:\some\writable\path"

D:\tools\vcpkg\vcpkg.exe install hwloc:x64-windows pkgconf:x64-windows
```

That produces:
- `D:\tools\vcpkg\installed\x64-windows\lib\pkgconfig\hwloc.pc`
- `D:\tools\vcpkg\installed\x64-windows\lib\hwloc.lib` / `bin\hwloc-15.dll`
- `D:\tools\vcpkg\installed\x64-windows\tools\pkgconf\pkgconf.exe`

Wiring these into the environment before invoking cargo lets the build script succeed:

```powershell
$env:PKG_CONFIG_PATH = "D:\tools\vcpkg\installed\x64-windows\lib\pkgconfig"
$env:PKG_CONFIG = "D:\tools\vcpkg\installed\x64-windows\tools\pkgconf\pkgconf.exe"
$env:PATH = "D:\tools\vcpkg\installed\x64-windows\bin;$env:PATH"
```

After this, `cargo check -p server --bin iggy-server` produces **zero** hwloc/pkg-config errors — confirmed by grepping the full build output for `hwloc|pkg-config|pkgconf`.

Suggested follow-up for this part alone: document this recipe in `README.md`/`CONTRIBUTING.md`, and/or add a `vendored` (CMake-based) opt-in for Windows similar to the existing musl `vendored` feature in `core/server/Cargo.toml`, so contributors don't need vcpkg at all. (The `vendored` feature was tried in this investigation too; it needs `cmake` on PATH, and — like the default path — still hits Part 2 below.)

## Part 2 — Unix-only APIs block compilation regardless of hwloc

With hwloc/pkg-config solved, `cargo check -p server --bin iggy-server` still fails to compile, on two crates that use Unix-only APIs with no Windows branch:

### `core/journal/src/file_storage.rs`

```
error[E0432]: unresolved import `std::os::fd`
--> core\journal\src\file_storage.rs:23:14
|
23 | use std::os::fd::AsFd;
| ^^ `std::os::fd` is Unix-only (Windows uses std::os::windows::io)

error[E0599]: no method named `as_fd` found for reference `&compio::compio_fs::File`
--> core\journal\src\file_storage.rs:82:40
|
82 | let file = fs::File::from(file.as_fd().try_clone_to_owned()?);
```

This is the `truncate()` helper: it needs a duplicate, synchronous `std::fs::File` handle to call `set_len`/`sync_all` on (compio's async `set_len` needs `IORING_OP_FTRUNCATE`, unsupported when falling back, per the comment in that function).

**Suggested direction:** `compio-driver` (a transitive dependency already in the tree, pulled in via `compio`) ships its *own* cross-platform `AsFd`/`AsRawFd`/`OwnedFd` abstraction that is implemented for both Unix fds and Windows `HANDLE`/`SOCKET` values (see `compio-driver-0.12.4/src/sys/pal/windows/fd.rs` vs `.../unix/mod.rs`). Swapping the Unix-only `std::os::fd::AsFd` import for compio's own re-exported trait (whatever compio publicly re-exports it as, e.g. `compio::driver::AsFd`/`AsRawFd`) should let this function become cross-platform, since `std::fs::File` already implements `From` on Windows the same way it implements `From` on Unix. Would need verifying compio actually re-exports this trait publicly from `compio::fs`/`compio::driver` for use outside the crate — if not, an issue/PR against `compio` itself might be a prerequisite.

### `core/message_bus/` — `fd_transfer.rs`, `socket_opts.rs`, `transports/tcp.rs`

```
error[E0432]: unresolved import (implicit via) std::os::unix::io::{AsRawFd, FromRawFd, RawFd}
error[E0425]: cannot find function `fcntl` / value `F_DUPFD_CLOEXEC` in crate `libc` (fd_transfer.rs)
error[E0425]: cannot find value `SOMAXCONN` in crate `libc` (socket_opts.rs)
error[E0425]: cannot find function `shutdown` / value `SHUT_RD` in crate `libc` (transports/tcp.rs)
error[E0599]: no method named `as_raw_fd` found for `compio::compio_net::TcpStream` / `PollFd`
```

Three distinct issues here, roughly increasing in difficulty:

1. **`socket_opts.rs::bind_reusable_tcp_listener`** — uses `libc::SOMAXCONN` (POSIX-only constant name; `libc` doesn't expose it for the `windows` target). Windows Winsock has its own `SOMAXCONN` (in `windows-sys::Win32::Networking::WinSock`), so this is a straightforward `#[cfg(unix)]`/`#[cfg(windows)]` constant swap, or just hardcoding a large backlog value (e.g. `1024`) cross-platform since `socket2::Socket::listen` takes a plain `i32` either way.

2. **`transports/tcp.rs::spawn_shutdown_watchdog`** — uses `libc::shutdown(fd, libc::SHUT_RD)` directly on a raw fd obtained via `PollFd::as_raw_fd()` (imported from `std::os::unix::io`, hence unavailable on Windows). `socket2::Socket` (already a dependency here) exposes a portable `.shutdown(std::net::Shutdown::Read)` that internally calls the correct platform syscall (`shutdown()`/Winsock `shutdown()` with `SD_RECEIVE`) — using that instead of the raw `libc::shutdown` call would likely make this portable without needing raw fds at all, provided a `socket2::Socket` (not just the raw fd) can be obtained/kept from the `PollFd` at this call site.

3. **`fd_transfer.rs`** — the hardest one. This module intentionally passes a raw fd between shards (`dup_fd`/`wrap_duped_fd`) using `fcntl(F_DUPFD_CLOEXEC)` for atomic duplicate-with-close-on-exec, plus `std::os::unix::io::{AsRawFd, FromRawFd, RawFd}` and `compio::net::TcpStream::{as_raw_fd, from_raw_fd}`. On Windows:
- `compio::net::TcpStream` does implement Windows support (`compio-net-0.12.5/src/incoming/windows.rs` exists), and `compio-driver`'s Windows `RawFd` type alias is `HANDLE`/`SOCKET`-based (see Part 2's journal note) — so switching the imports from `std::os::unix::io::*` to compio's own cross-platform re-exports may resolve the `as_raw_fd`/`from_raw_fd` compile errors directly.
- The `fcntl(F_DUPFD_CLOEXEC)` atomic-dup-with-cloexec trick has no 1:1 Windows equivalent, but the underlying need (duplicate a socket handle, non-inheritable, to hand to another shard/thread) maps to Win32 `DuplicateHandle` with `bInheritHandle = FALSE` — semantically simpler on Windows since handles aren't inherited by child processes by default unless explicitly requested. `socket2::Socket::try_clone()` (already a workspace dependency) is cross-platform and uses `DuplicateHandle` internally on Windows, so it may be usable directly in place of the raw `fcntl` call if the surrounding fd-transfer plumbing can be adapted to work through `socket2::Socket`/`OwnedSocket` instead of a bare `RawFd`. This would need real design work (and testing) rather than a one-line fix — flagging it as the most involved sub-part of this issue.

## Suggested acceptance criteria

- `cargo check -p server --bin iggy-server --target x86_64-pc-windows-msvc` succeeds with no `#[cfg(windows)]`-breaking errors, either natively or with a documented one-time toolchain setup (vcpkg or a `vendored`-hwloc opt-in).
- CI ideally gains a Windows check job for at least `cargo check -p server` (even if `cargo test`/full integration remains Linux-only initially, given `io_uring`/compio's Linux-specific fast paths).
- README/CONTRIBUTING gets a "Building on Windows" section documenting whatever ends up working (hwloc/pkg-config setup at minimum; native full build if Part 2 gets fixed).

## Environment for reproduction

- Rust: pinned via `rust-toolchain.toml` (1.98.0), target `x86_64-pc-windows-msvc`
- `hwlocality-sys` 0.7.1, `compio` 0.12.x / `compio-driver` 0.12.4 / `compio-net` 0.12.5 (as resolved in `Cargo.lock` at time of testing)

### Affected area / component

_No response_

### Proposed solution

_No response_

### Alternatives considered

_No response_

### Contribution

- [x] I'm willing to submit a pull request to implement this feature

### Good first issue

- [ ] I think this could be a good first issue for a new contributor

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.