✨ Cost Optimization - Reduce high ingestion costs in staging Log Analytics workspace

Requested by: @chrisschultzssw via YakShaver.ai 🦬
cc: @Marxoz, @PothieuG, @Freego1783

Hi Team!

  🟥  Watch the video (1 min 0 sec)

  Url from screen share: https://portal.azure.com/#@ssw.com.microsoft.com/resource/subscriptions/bb18d4cd-83e3-472e-9886-00c2983629ae/resourceGroups/SSW_Rules_Staging/providers/Microsoft.OperationalInsights/workspaces/log-sswules-staging/usageAndCosts

Pain

The staging Log Analytics workspace for Rules V3 (TinaCMS) is incurring an unexpectedly high estimated monthly cost (~USD 56) compared to the production workspace (~USD 20). Investigation shows that App Dependencies and App Requests are responsible for most of the data ingestion. Review the telemetry configuration for the staging environment, identify the cause of excessive ingestion, and reduce costs while maintaining necessary observability.

Acceptance Criteria

1. Staging Log Analytics workspace ingestion costs are investigated and the main contributors are clearly identified.
2. Unnecessary or overly verbose telemetry (e.g. App Dependencies, App Requests) in staging is reduced, sampled, or disabled as appropriate.
3. Estimated monthly cost for the staging workspace is significantly reduced and closer to production levels, unless a justified difference is documented.
4. Any configuration changes are documented, including rationale and impact on monitoring.
5. Production Log Analytics workspace behavior and costs are not negatively impacted by the changes.

Screenshot

Figure: Staging Log Analytics workspace shows unusually high cost driven by app dependencies and requests

Start with the staging Log Analytics workspace usage and costs page linked in the issue, then inspect the staging telemetry configuration and compare its App Dependencies and App Requests ingestion with production. Done means the main cost contributors are identified, staging ingestion is reduced without harming required monitoring, the estimated cost is closer to production, and the configuration changes and monitoring impact are documented.