Azure / Azure/azure-sdk-for-cpp

Package Acquisition using Source / CMake Fetch Content

Open
#6,309 1 comment 0 reactions 1 assignee Claimed by @ronniegeraghty View on GitHub
Docs pillar-acquisition
Dominant language
C++
Stars
205
Forks
172
Avg merge
1d 3h
Merged PRs (30d)
37

Description

Currently, directly including the source for 3rd party dependencies is the most common why C++ developers integrate dependencies into their projects. The Azure SDK for C++ is supposed to support this through CMake Fetch Content, allowing it to take care of downloading the source code at a specific version and linking the build process for the specific Azure SDK library. But it seems our support for CMake Fetch Content has lapsed.

## Current CMake Fetch Content Issue:

### Project Set Up

- **main.cpp** file:

```cpp
#include
#include
#include

using namespace Azure::Security::KeyVault::Secrets;

int main()
{
try
{
// Set Key Vault URL string
auto const keyVaultUrl = std::getenv("AZURE_KEYVAULT_URL");

// Create Default Azure Credential to Authenticate.
// It will pick up on our AzureCLI login
auto credential = std::make_shared();

// Create Key Vault Secret Client
SecretClient secretClient(keyVaultUrl, credential);

// Create a Secret
std::string secretName("MySampleSecret");
std::string secretValue("My super secret value");
secretClient.SetSecret(secretName, secretValue);

// Get the Secret
KeyVaultSecret secret = secretClient.GetSecret(secretName).Value;
std::string valueString = secret.Value.HasValue()
? secret.Value.Value()
: "NONE RETURNED";
std::cout << "Secret is returned with name " << secret.Name
<< " and value " << valueString << std::endl;
}
catch (Azure::Core::Credentials::AuthenticationException const &e)
{
std::cout << "Authentication Exception happened:" << std::endl
<< e.what() << std::endl;
return 1;
}
catch (Azure::Core::RequestFailedException const &e)
{
std::cout << "Key Vault Secret Client Exception happened:" << std::endl
<< e.Message << std::endl;
return 1;
}

return 0;
}
```

- **CMakeLists.txt** file

```cmake
# Specify the minimum version of CMake required to build this project
cmake_minimum_required(VERSION 3.30.0)

# Define the project name, version, and the languages used
project(azure_sample VERSION 0.1.0 LANGUAGES C CXX)

# Fetch content
include(FetchContent)
FetchContent_Declare(azuresdkforcpp
# Set the SDK URL path and releasse tag
GIT_REPOSITORY https://github.com/Azure/azure-sdk-for-cpp.git
GIT_TAG azure-security-keyvault-secrets_4.2.1
)
FetchContent_GetProperties(azuresdkforcpp)
if(NOT azuresdkforcpp_POPULATED)
FetchContent_MakeAvailable(azuresdkforcpp)
add_subdirectory(${azuresdkforcpp_SOURCE_DIR} ${azuresdkforcpp_BINARY_DIR} EXCLUDE_FROM_ALL)
endif()

add_executable(azure_sample main.cpp)

target_compile_definitions(azure_sample PRIVATE _azrue_BUILD_SAMPLES)

target_link_libraries(azure_sample Azure::azure-security-keyvault-secrets)
```

### Windows Result

On Windows, I hit an issue running the CMake config command: `cmake -B ./build`

```output
PS C:\Users\user\CMakeFetchContentSample> cmake -B ./build
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.26100.0 to target Windows 10.0.22631.
-- The C compiler identification is MSVC 19.42.34436.0
-- The CXX compiler identification is MSVC 19.42.34436.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/IntPreview/VC/Tools/MSVC/14.42.34433/bin/HostX64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/IntPreview/VC/Tools/MSVC/14.42.34433/bin/HostX64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Cloning into 'azuresdkforcpp-src'...
HEAD is now at 1d22b9bc5 Fix broken link (#5273)
No transport adapter was selected, using WinHTTP as the default option for Windows.
Vcpkg integrate step.
CMAKE_TOOLCHAIN_FILE is not defined. Define it for the user.
AZURE_SDK_DISABLE_AUTO_VCPKG is not defined. Fetch a local copy of vcpkg.
Vcpkg commit string used: 43cf47eccfbe27006cf9534a5db809798f8c37fe
CMake Warning (dev) at C:/Program Files/CMake/share/cmake-3.31/Modules/FetchContent.cmake:1953 (message):
Calling FetchContent_Populate(vcpkg) is deprecated, call
FetchContent_MakeAvailable(vcpkg) instead. Policy CMP0169 can be set to
OLD to allow FetchContent_Populate(vcpkg) to be called directly for now,
but the ability to call it with declared details will be removed completely
in a future version.
Call Stack (most recent call first):
build/_deps/azuresdkforcpp-src/cmake-modules/AzureVcpkg.cmake:36 (FetchContent_Populate)
build/_deps/azuresdkforcpp-src/CMakeLists.txt:25 (az_vcpkg_integrate)
This warning is for project developers. Use -Wno-dev to suppress it.

Vcpkg integrate step - DONE.
-- CMAKE_SYSTEM_NAME=Windows;
-- VCPKG_TARGET_TRIPLET=;
Vcpkg integrate step.
Vcpkg integrate step - DONE.
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
CMake Error at build/_deps/azuresdkforcpp-src/sdk/core/azure-core/CMakeLists.txt:37 (find_package):
Could not find a package configuration file provided by "wil" with any of
the following names:

wilConfig.cmake
wil-config.cmake

Add the installation prefix of "wil" to CMAKE_PREFIX_PATH or set "wil_DIR"
to a directory containing one of the above files. If "wil" provides a
separate development package or SDK, be sure it has been installed.

-- Configuring incomplete, errors occurred!
```

Our current build process using CMake Fetch Content doesn't seem to either be properly pulling the `wil` dependency or is not properly able to locate the dependency.

### Linux Result

On Linux (WSL/Ubuntu) I also hit an issue running the CMake config command: `cmake -B ./build`:

```output
ronnieg@---:~/code/cmakefetchcontentsample$ cmake -B ./build
-- The C compiler identification is GNU 13.3.0
-- The CXX compiler identification is GNU 13.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Cloning into 'azuresdkforcpp-src'...
HEAD is now at 1d22b9bc5 Fix broken link (#5273)
No transport adapter was selected, using libcurl as the default option for POSIX.
Vcpkg integrate step.
CMAKE_TOOLCHAIN_FILE is not defined. Define it for the user.
AZURE_SDK_DISABLE_AUTO_VCPKG is not defined. Fetch a local copy of vcpkg.
Vcpkg commit string used: 43cf47eccfbe27006cf9534a5db809798f8c37fe
CMake Warning (dev) at /home/ronnieg/cmake-3.31.2-linux-x86_64/share/cmake-3.31/Modules/FetchContent.cmake:1953 (message):
Calling FetchContent_Populate(vcpkg) is deprecated, call
FetchContent_MakeAvailable(vcpkg) instead. Policy CMP0169 can be set to
OLD to allow FetchContent_Populate(vcpkg) to be called directly for now,
but the ability to call it with declared details will be removed completely
in a future version.
Call Stack (most recent call first):
build/_deps/azuresdkforcpp-src/cmake-modules/AzureVcpkg.cmake:36 (FetchContent_Populate)
build/_deps/azuresdkforcpp-src/CMakeLists.txt:25 (az_vcpkg_integrate)
This warning is for project developers. Use -Wno-dev to suppress it.

Vcpkg integrate step - DONE.
-- CMAKE_SYSTEM_NAME=Linux;
-- VCPKG_TARGET_TRIPLET=;
Vcpkg integrate step.
Vcpkg integrate step - DONE.
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
CMake Error at /home/ronnieg/cmake-3.31.2-linux-x86_64/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:233 (message):
Could NOT find CURL (missing: CURL_LIBRARY CURL_INCLUDE_DIR)
Call Stack (most recent call first):
/home/ronnieg/cmake-3.31.2-linux-x86_64/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:603 (_FPHSA_FAILURE_MESSAGE)
/home/ronnieg/cmake-3.31.2-linux-x86_64/share/cmake-3.31/Modules/FindCURL.cmake:203 (find_package_handle_standard_args)
build/_deps/azuresdkforcpp-src/sdk/core/azure-core/CMakeLists.txt:31 (find_package)

-- Configuring incomplete, errors occurred!
```

Seems like this is just a case of missing curl. I installed curl and cleared the CMake cache and tried again. The next dependency I was told was missing was OpenSSL, so again I installed it and reran the config. Then I ran into the following issue:

```output
ronnieg@CPC-rgera-JR9LR:~/code/cmakefetchcontentsample$ cmake -B ./build
-- The C compiler identification is GNU 13.3.0
-- The CXX compiler identification is GNU 13.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Cloning into 'azuresdkforcpp-src'...
HEAD is now at 1d22b9bc5 Fix broken link (#5273)
No transport adapter was selected, using libcurl as the default option for POSIX.
Vcpkg integrate step.
CMAKE_TOOLCHAIN_FILE is not defined. Define it for the user.
AZURE_SDK_DISABLE_AUTO_VCPKG is not defined. Fetch a local copy of vcpkg.
Vcpkg commit string used: 43cf47eccfbe27006cf9534a5db809798f8c37fe
CMake Warning (dev) at /home/ronnieg/cmake-3.31.2-linux-x86_64/share/cmake-3.31/Modules/FetchContent.cmake:1953 (message):
Calling FetchContent_Populate(vcpkg) is deprecated, call
FetchContent_MakeAvailable(vcpkg) instead. Policy CMP0169 can be set to
OLD to allow FetchContent_Populate(vcpkg) to be called directly for now,
but the ability to call it with declared details will be removed completely
in a future version.
Call Stack (most recent call first):
build/_deps/azuresdkforcpp-src/cmake-modules/AzureVcpkg.cmake:36 (FetchContent_Populate)
build/_deps/azuresdkforcpp-src/CMakeLists.txt:25 (az_vcpkg_integrate)
This warning is for project developers. Use -Wno-dev to suppress it.

Vcpkg integrate step - DONE.
-- CMAKE_SYSTEM_NAME=Linux;
-- VCPKG_TARGET_TRIPLET=;
Vcpkg integrate step.
Vcpkg integrate step - DONE.
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found CURL: /usr/lib/x86_64-linux-gnu/libcurl.so (found version "8.5.0")
Libcurl version 8.5.0
-- Found OpenSSL: /usr/lib/x86_64-linux-gnu/libcrypto.so (found version "3.0.13")
-- Retrieving version from /home/ronnieg/code/cmakefetchcontentsample/build/_deps/azuresdkforcpp-src/sdk/core/azure-core/src/private/package_version.hpp
-- VERSION_MAJOR 1
-- VERSION_MINOR 12
-- VERSION_PATCH 0
-- VERSION_PRERELEASE beta.1
-- AZ_LIBRARY_VERSION
-- Including AMQP library
Vcpkg integrate step.
Vcpkg integrate step - DONE.
-- Using vendored uamqp
CMake Deprecation Warning at build/_deps/azuresdkforcpp-src/sdk/core/azure-core-amqp/vendor/azure-uamqp-c/CMakeLists.txt:4 (cmake_minimum_required):
Compatibility with CMake < 3.10 will be removed from a future version of
CMake.

Update the VERSION argument value. Or, use the ... syntax
to tell CMake that the project requires at least but has been updated
to work with policies introduced by or earlier.

CMake Error at build/_deps/azuresdkforcpp-src/sdk/core/azure-core-amqp/vendor/azure-uamqp-c/CMakeLists.txt:56 (find_package):
Could not find a package configuration file provided by
"azure_macro_utils_c" with any of the following names:

azure_macro_utils_cConfig.cmake
azure_macro_utils_c-config.cmake

Add the installation prefix of "azure_macro_utils_c" to CMAKE_PREFIX_PATH
or set "azure_macro_utils_c_DIR" to a directory containing one of the above
files. If "azure_macro_utils_c" provides a separate development package or
SDK, be sure it has been installed.

-- Configuring incomplete, errors occurred!
```

This new error appears to be caused by missing the **azure_macro_utils_c** dependency that is required by the SDK's vendored version of **azure-uamqp-c**. I understand that this could probably also by installing the dependency on my env and moving on, but I question why the build process is attempting to build any part of our AMQP stack when I'm only trying to use our Key Vault Secrets and Identity libraries.

## Solutions for building the Azure SDK from source

- Fix current CMake Fetch Content scenario that invokes vcpkg underneath
- Update CMake Fetch Content scenario to only pull the desired source (maybe included core) and have the user resolve all external dependencies
- Users pull the zips from our releases in GitHub and they include only the desired library (maybe a compatible version of core as well) then users can either build that source for their platform and take a dep on the build binary or include the source in their project and build in their project's build

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.