pip install fail to link

I'm doing a Python extended library based on CMake and C++, and the dependencies include OpenCV and PCL, which are installed in the system.

The Python environment is managed by conda.

While the CMake build and system-wide pipx build succeeded, pip install -e . fails on linking: a bunch of error like /home/vscode/miniforge3/envs/lidar_od/compiler_compat/ld: /usr/lib/x86_64-linux-gnu/libopencv_imgcodecs.so.4.6.0: undefined reference to jpeg_calc_output_dimensions@LIBJPEG_8.0'`

I think there is something missing in my configurations. But I could not find any tutorials or examples.

structure is as follows:

.
├── apps
│   ├── io
│   ├── logs
│   ├── test
│   └── tools
├── build
│   ├── apps
│   └── CMakeFiles
├── cmake
├── dist
├── images
├── include
│   └── lidar_odometry
├── python_bind
│   └── example_data
├── scripts
├── src
│   ├── cam_model
│   ├── cloud_process
│   ├── factors
│   ├── feature
│   ├── lidar_map
│   ├── lins
│   ├── misc
│   ├── registration
│   └── viewer
└── thirdparty
    └── x86_64

CMakeLists.txt

# pyproject.toml
[build-system]
requires = ["scikit-build-core>=0.10", "pybind11", "cmake", "ninja"]
build-backend = "scikit_build_core.build"


[project]
name = "lidar_odometry"
version = "0.0.1"
description = "A minimal example package (with pybind11)"
readme = "README.md"
authors = [
    { name = "xb", email = "me@email.com" },
    { name = "lzn", email = "lzn@email.com" },
]
requires-python = ">=3.9, <3.11"
classifiers = [
    "Development Status :: 4 - Beta",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3 :: Only",
    "Programming Language :: Python :: 3.9",
    "Programming Language :: Python :: 3.10",
]


[tool.scikit-build]
wheel.expand-macos-universal-tags = true
minimum-version = "build-system.requires"
cmake = { build-type = "Release", args = ['-GNinja'] }
build-dir = "build/{wheel_tag}"

Start with pyproject.toml and the attached CMakeLists.txt; reproduce pip install -e . inside the conda environment and compare it with the successful CMake and pipx builds. Done means the editable install completes without the reported OpenCV/libjpeg linker error.