Supportability: clear separation of concerns between wrapper SDK, native SDKs, and backends

Goal

Ensure that when something goes wrong, users and support engineers can immediately determine whether the issue originates in the MulticloudDB wrapper SDK, the underlying native SDK (azure-cosmos, aws-sdk-dynamodb, google-cloud-spanner), or the backend service itself — and know exactly what information to collect for each layer.

Problem Statement

A multi-layer SDK introduces diagnostic ambiguity. Today a caller receives a MulticloudDbException that wraps a native exception, but there is no systematic way to:

1. Distinguish a wrapper-layer bug (e.g., incorrect query translation) from a native SDK bug (e.g., connection pool exhaustion) from a backend error (e.g., throttling, partition split)
2. Collect the right diagnostic artifact for each layer in a single pass
3. Report the exact versions of the wrapper SDK and each native SDK dependency
4. Correlate a single user operation end-to-end across wrapper → native SDK → backend service

Current State (what already works)

Capability	Status	Notes
Exception cause chain preservation	✅	MulticloudDbException(error, cause) passes native exception as getCause()
Normalized error categories	✅	10 MulticloudDbErrorCategory values mapped per provider
Provider details on errors	✅	providerDetails() map carries status codes, request IDs, RU charge (Cosmos)
OperationDiagnostics on exceptions	✅	Provider, operation name, wall-clock duration, request ID
Separate logging namespaces	✅	com.multiclouddb.api.* vs com.multiclouddb.provider.cosmos.* etc.
Native client escape hatch	✅	nativeClient(Class<T>) with info-level log on access
Retryable flag on errors	✅	Per-provider status code → retryable mapping

Gaps to Address

1. SDK Version Identification

Problem: There is no programmatic way for a user or support engineer to determine the wrapper SDK version or the versions of native SDK dependencies at runtime.

What's needed:

• A MulticloudDb.version() or equivalent API that returns the wrapper SDK version (read from Maven-generated pom.properties or a manifest attribute)
• A MulticloudDbClient.providerVersions() or equivalent that reports the native SDK version(s) in use
• Version information included in diagnostic output and exception toString() representations

2. User-Agent / Telemetry Identification

Problem: Native SDKs embed user-agent strings for backend telemetry (e.g., Cosmos DB uses azsdk-java-cosmos/<version>). The wrapper SDK does not append its identity, making it invisible in backend telemetry and support traces.

What's needed:

• Append multiclouddb-java/<version> to the native SDK's user-agent or application identifier
• For Cosmos DB: set via CosmosClientBuilder.userAgentSuffix()
• For DynamoDB: set via ClientOverrideConfiguration.addApiCallAttemptListener() or equivalent
• For Spanner: set via channel configurator or SpannerOptions user-agent field

3. Native Diagnostics Passthrough

Problem: OperationDiagnostics captures wall-clock duration and request ID, but does not preserve the full native diagnostic payload that backend support teams require. For example:

• Cosmos DB's CosmosDiagnostics contains detailed timeline, replica endpoints, retry history, and transport-level metrics
• DynamoDB's consumed capacity and table throughput details
• Spanner's query plan statistics

What's needed:

• An OperationDiagnostics.nativeDiagnostics() method (or similar) that returns the provider's raw diagnostic object (typed or as String)
• On error paths: attach native diagnostics to MulticloudDbException so they survive the catch-rethrow boundary
• On success paths: make native diagnostics optionally available on response objects (e.g., QueryPage.diagnostics())

4. Error Attribution Guidance

Problem: When a user encounters an error, they need to know which team to contact and what information to provide. The current exception model preserves data but does not guide the user on attribution.

What's needed:

• Documentation (and optionally an API) that classifies each MulticloudDbErrorCategory by likely origin:

  Category	Likely Origin	First Responder
  INVALID_REQUEST	Caller or wrapper (bad query translation)	MulticloudDB SDK team
  AUTHENTICATION_FAILED	Configuration or backend IAM	Cloud provider / caller
  AUTHORIZATION_FAILED	Backend RBAC policy	Cloud provider / caller
  NOT_FOUND	Caller (wrong key) or backend	Caller first, then provider
  CONFLICT	Caller (duplicate create) or concurrency	Caller first
  THROTTLED	Backend capacity	Cloud provider
  TRANSIENT_FAILURE	Backend or network	Retry; then cloud provider
  PERMANENT_FAILURE	Backend or wrapper bug	MulticloudDB SDK team
  PROVIDER_ERROR	Native SDK or backend (unmapped)	MulticloudDB SDK team → cloud provider
  UNSUPPORTED_CAPABILITY	Wrapper (intentional)	MulticloudDB SDK docs

• A troubleshooting guide that walks through: check category → check providerDetails → check getCause() → collect diagnostics → file issue with appropriate team

5. Diagnostic Export for Support Tickets

Problem: There is no single method to produce a support-ready diagnostic bundle that a user can attach to a GitHub issue or support ticket.

What's needed:

• A MulticloudDbException.toDiagnosticString() (or similar) that produces a structured, paste-friendly block containing:

  MulticloudDB SDK version: 0.1.0
  Provider: COSMOS (azure-cosmos 4.78.0)
  Operation: create
  Category: CONFLICT
  Retryable: false
  Duration: 42ms
  Request ID: <activity-id>
  Provider Details: {statusCode=409, subStatusCode=0, requestCharge=1.0}
  Native Diagnostics: <CosmosDiagnostics summary or first N chars>
  Cause: com.azure.cosmos.CosmosException: ...
  

• Sensitive data (connection strings, keys) must never appear in this output
• Optionally: a static MulticloudDb.diagnosticReport() that captures environment info (JDK version, OS, SDK versions, provider configuration summary)

6. Correlation ID Propagation

Problem: The wrapper generates or extracts a requestId per operation, but there is no way for callers to inject their own correlation ID that flows through to the native SDK and backend.

What's needed:

• An optional correlationId parameter on operations (or a Context / RequestOptions object) that:
  • Gets attached to OperationDiagnostics
  • Gets propagated to the native SDK where supported (e.g., Cosmos DB's CosmosItemRequestOptions.setCorrelationActivityId())
• This enables correlation across distributed systems and log aggregation platforms

7. Structured Logging Convention

Problem: Current logging is functional but ad-hoc. Different providers log different fields in different formats, making log aggregation across providers difficult.

What's needed:

• A documented logging convention for the SDK:
  • Wrapper layer logs use prefix: [multiclouddb]
  • All operation logs include: provider, operation, database, collection, durationMs, requestId
  • Error logs additionally include: category, retryable, statusCode
• Structured logging support (MDC or key-value pairs) for integration with log aggregation tools (ELK, Application Insights, CloudWatch)

8. Health Check / Connectivity Validation

Problem: There is no built-in way to verify that the client can reach the backend, which is the first step in any support triage process.

What's needed:

• A MulticloudDbClient.healthCheck() method that:
  • Validates connectivity to the configured backend
  • Returns a structured result: reachable (boolean), latency, provider details, any warnings
  • Does not throw on failure (returns a result object with error details)
• This becomes the standard first step in any troubleshooting runbook

Approach

1. Version & telemetry (gaps 1, 2): Add version resource loading, expose version() API, configure user-agent suffix on each provider's native client builder
2. Diagnostics enrichment (gaps 3, 5): Extend OperationDiagnostics to carry native diagnostics; add toDiagnosticString() on exception
3. Attribution & guidance (gap 4): Create troubleshooting documentation, optionally add likelyOrigin() to error categories
4. Correlation (gap 6): Add optional correlation ID to operation signatures or introduce a RequestContext parameter
5. Logging (gap 7): Standardize log format across all providers, document convention
6. Health check (gap 8): Implement lightweight connectivity check per provider

Acceptance Criteria

• MulticloudDb.version() returns the correct SDK version at runtime
• Each provider's native client includes multiclouddb-java/<version> in its user-agent string
• OperationDiagnostics exposes native diagnostic payload (at minimum as a String) on both error and success paths
• MulticloudDbException.toDiagnosticString() produces a complete, support-ready diagnostic block with no sensitive data
• Troubleshooting guide documents error attribution by category with "who to contact" guidance
• Caller-supplied correlation ID flows through to OperationDiagnostics and (where supported) to the native SDK
• All provider implementations follow a documented structured logging convention
• healthCheck() returns a structured connectivity result for each provider
• No secrets, connection strings, or API keys appear in any diagnostic output