AWS Lambda extension support

# AWS Lambda support

This issue is intended to give some context and track a couple of PRs that are adding features independently of the goal: support for AWS lambda. This comes after stumbling about this PR https://github.com/aws-samples/aws-lambda-extensions/pull/42 attempting to add Fluentbit as an AWS Lambda extension. The goal of this possible implementation would be to allow Fluentbit to be embedded within lambda and processing logs received from the [lambda telemetry API](https://aws.amazon.com/blogs/compute/introducing-the-aws-lambda-telemetry-api/).

Lambda extensions run alongside the lambda workload and receive logs as they come before being frozen once the extension hands back the environment until the next lambda gets invoked unthawing the environment. This event-driven workflow requires Fluentbit to be event-driven as well for its flush mechanism to ensure logs are force flushed to the outputs in a deterministic manner ([doc](https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtime-environment.html)).

PRs to make the feature possible:

1. **Flush now**: modifies the HTTP server to add a `/api/v2/flush` route which when called, will flush the in-transit chunks, returning when acknowledged by the engine. [PR](https://github.com/fluent/fluent-bit/pull/12192)
2. **In-flight tasks metrics**: modifies the HTTP server's `/api/v2/metrics` route adding an optional `?recalculate=true` GET parameter to force recalculation of metrics which are currently delayed. The purpose is to inspect `fluentbit_storage_chunks` in real-time to ensure all chunks are flushed before handing back the environment to AWS lambda for freezing.

The lambda extension (an intermediary piece of software) starting Fluentbit would then normally flush logs during the lifetime of the lambda workload, and leverage the aforementioned features upon termination of the lambda to force flushing every chunk. Upon lambda terminating, the extension would call the `/api/v2/flush` route, and then the `/api/v2/metrics?recalculate=true` endpoint either until all chunks are flushed, or the extension times out waiting for chunks to be flushed. The extension then hands back the environment to AWS for freezing until the next lambda invocation.

Would the above approach (force flush + poll metrics) be the preferred way to solve this? I'm open to alternative, simpler solutions from experts more familiar with the codebase. In the meantime, I'll open a PR for the flush feature, and work on the second one if the above is deemed reasonable.

Start by reviewing the proposed flush feature in PR 12192 and the existing HTTP server handling for /api/v2/metrics. Determine whether the two endpoints can support the AWS Lambda extension workflow described; done means the approach is agreed and the required flush and metrics behavior is validated for deterministic log delivery.

Written by the indexing model from the issue text.