Blueprint Issue: Upgrade AWS SDK for C++ to Support `credential_process`

Blueprint Issue: Upgrade AWS SDK for C++ to Support credential_process

Title

Upgrade AWS SDK for C++ to Support credential_process for IAM Roles Anywhere and Custom Credential Helpers

Problem Statement

OSQuery currently uses AWS SDK for C++ version 1.7.230 (updated in November 2020), which does not support the credential_process capability. This feature is essential for modern AWS authentication patterns, particularly:

1. IAM Roles Anywhere: AWS's solution for workload identity outside of AWS infrastructure
2. Custom credential helpers: Enterprise-specific credential management systems
3. Short-lived credentials: Enhanced security through external credential rotation

The credential_process feature allows AWS SDKs to execute an external command to fetch credentials dynamically, as specified in the AWS config file (~/.aws/config). This is documented in the AWS CLI configuration documentation.

Current Behavior

When a user configures credential_process in their AWS profile:

[profile osquery]
credential_process = /usr/local/bin/aws-credential-helper
region = us-east-1

And runs osquery with --aws_profile_name=osquery --aws_kinesis_stream=my-stream, the credential process is silently ignored. OSQuery's ProfileConfigFileAWSCredentialsProvider (from SDK 1.7.230) only reads static credentials from the credentials file and does not recognize or execute the credential_process directive.

Impact

This limitation affects all AWS service integrations in osquery:

• Kinesis Streams (aws_kinesis logger)
• Kinesis Firehose (aws_firehose logger)
• S3 (file carving, log forwarding)
• STS (assume role operations)
• EC2 (instance metadata tables)

Organizations using IAM Roles Anywhere or custom credential systems cannot use osquery's native AWS integrations without workarounds like hardcoded credentials or wrapper scripts.

Proposed Solution

Upgrade the AWS SDK for C++ dependency from 1.7.230 to a version ≥ 1.9.0, which includes support for credential_process.

Recommended Target Version

AWS SDK for C++ 1.11.x (latest stable: 1.11.680 as of submission)

Rationale for 1.11.x:

• Mature and actively maintained branch
• Full credential_process support (added in 1.9.0)
• Extensive bug fixes and security updates since 1.7.230
• Better long-term maintenance story
• Backward compatible with existing credential provider chains

Alternative: Minimum viable version would be 1.9.379 (last patch of 1.9.x series) if concerns about the larger version jump exist.

Implementation Changes Required

1. Update Git Submodule

   • File: .gitmodules
   • Update submodule reference for libraries/cmake/source/aws-sdk-cpp/src/aws-sdk-cpp

2. Review CMake Build Configuration

   • File: libraries/cmake/source/aws-sdk-cpp/CMakeLists.txt
   • Verify source file lists are current with new SDK version
   • Check for new dependencies or build flags

3. No Changes to Credential Chain Expected

   • File: osquery/utils/aws/aws_util.cpp
   • The existing ProfileConfigFileAWSCredentialsProvider in SDK ≥1.9.0 automatically supports credential_process
   • The credential provider chain order remains unchanged
   • No osquery-specific code changes required - this is purely a dependency upgrade

4. Update Documentation

   • File: docs/wiki/deployment/aws-logging.md
   • Add example of credential_process configuration
   • Document IAM Roles Anywhere use case

Testing Requirements

• All existing credential methods must continue to work:

  • ✅ Command-line flags (--aws_access_key_id, --aws_secret_access_key)
  • ✅ Named AWS profiles with static credentials
  • ✅ Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
  • ✅ EC2 instance profile metadata
  • ✅ STS assume role
  • ✅ NEW: AWS profiles with credential_process

• Test across all supported platforms:

  • Linux x86_64, Linux ARM64
  • macOS x86_64, macOS ARM64
  • Windows x86_64

• Specific AWS service validation:

  • Kinesis logger plugin
  • Firehose logger plugin
  • S3 operations
  • EC2 metadata tables

Backward Compatibility

The AWS SDK for C++ maintains backward compatibility between minor versions. Upgrading from 1.7.x to 1.9.x or 1.11.x should not break existing functionality. The credential provider chain behavior is additive - existing methods continue to work, with credential_process support added when configured.

Security Considerations

✅ Enhances security by enabling:

• Short-lived credentials via external rotation
• Elimination of long-lived static credentials
• Integration with enterprise identity systems
• Support for AWS IAM Roles Anywhere (temporary credentials)

Use Cases

Use Case 1: IAM Roles Anywhere

Organizations with workloads outside AWS can use IAM Roles Anywhere to obtain temporary credentials:

[profile osquery]
credential_process = aws_signing_helper credential-process --certificate /path/to/cert.pem --private-key /path/to/key.pem --trust-anchor-arn arn:aws:rolesanywhere:... --profile-arn arn:aws:rolesanywhere:... --role-arn arn:aws:iam::...
region = us-east-1

Use Case 2: Enterprise Credential Management

Companies with custom credential vending systems can integrate seamlessly:

[profile osquery]
credential_process = /usr/local/bin/company-credential-helper --environment prod --service osquery
region = us-west-2

Use Case 3: Kubernetes Workload Identity

Workloads in Kubernetes can fetch credentials dynamically without storing secrets:

[profile osquery]
credential_process = /opt/k8s/aws-credential-provider
region = eu-west-1

Migration Path

For users currently working around this limitation:

Before (workaround):

eval $(credential-helper)  # Sets AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
osqueryd --aws_kinesis_stream=...

After (native):

# ~/.aws/config
[profile osquery]
credential_process = /usr/local/bin/credential-helper
region = us-east-1

osqueryd --aws_profile_name=osquery --aws_kinesis_stream=...

Implementation Timeline

1. Phase 1: Blueprint Review (this issue)

   • Community discussion
   • Core team approval
   • Finalize target SDK version

2. Phase 2: Development

   • Update submodule reference
   • Verify CMake builds across platforms
   • Run existing test suites

3. Phase 3: Testing

   • CI validation on all platforms
   • Community beta testing
   • Documentation updates

4. Phase 4: Release

   • Merge to master
   • Include in next minor release
   • Update CHANGELOG

Open Questions

1. Preferred SDK version: 1.9.x (conservative) vs 1.11.x (recommended)?
2. Release targeting: Next minor release or patch release?
3. Beta testing: Would the team want beta testers with IAM Roles Anywhere setups?
4. Build time impact: Should we measure build time differences with newer SDK?

Additional Context

• AWS SDK C++ credential_process was added in version 1.9.0 (released late 2021)
• AWS SDK C++ repository: https://github.com/aws/aws-sdk-cpp
• Current osquery SDK update commit: e2b3598ba (Nov 2020)
• Submodule location: libraries/cmake/source/aws-sdk-cpp/src/aws-sdk-cpp

References

• AWS credential_process documentation: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sourcing-external.html
• IAM Roles Anywhere: https://docs.aws.amazon.com/rolesanywhere/latest/userguide/introduction.html
• AWS SDK C++ changelog: https://github.com/aws/aws-sdk-cpp/blob/main/CHANGELOG.md
• osquery AWS utilities: osquery/utils/aws/aws_util.cpp

---

I am willing to:

• Contribute the implementation
• Test beta builds with IAM Roles Anywhere
• Update documentation
• Provide feedback during review

Start with the AWS SDK submodule at libraries/cmake/source/aws-sdk-cpp/src/aws-sdk-cpp and review libraries/cmake/source/aws-sdk-cpp/CMakeLists.txt for the proposed version and build implications. Check osquery/utils/aws/aws_util.cpp and the existing test suites, then update docs/wiki/deployment/aws-logging.md; done means the dependency builds across supported platforms, existing credential methods remain functional, and credential_process is validated.