deprecation and removal process of JMX Gatherer

The goal of this issue is to define and discuss the steps of JMX Gatherer deprecation and removal.

current status (as of writing)

• JMX Gatherer is documented as deprecated
• JMX Scraper provides a replacement of JMX Gatherer.
• JMX Scraper provides a legacy metrics definitions set that is very close to JMX Gatherer (see otel.jmx.target.source in config)
• JMX Scraper provides a migration guide
• otel collector jmxreceiver can use JMX Gatherer (default) or JMX Scraper (opt-in)

desired end goal

• users of JMX Gatherer are aware the component is being deprecated and should be able to migrate to JMX Scraper
• users of JMX Gatherer through the collector jmxreceiver could have a multi-step migration process to minimize migration burden
• users should always be able to opt-out by sticking to an older version of JMX Gatherer and/or using otel.jmx.target.source=legacy with JMX Scraper.
• JMX Gatherer component is removed from code and is no longer published on release.

things to discuss

• for jmxreceiver should we do the migration in one step (big-bang) or two steps (implementation+legacy first, then aligned metrics) ?
• for direct usages of JMX Gatherer, should we use an obtrusive way to ensure users are aware the component is being removed ?

steps

• step 1 : in jmxreceiver switch default implementation to JMX Scraper with otel.jmx.target.source=legacy
  • users can opt-out by switching to JMX Gatherer implementation (jmxreceiver should provide a documented way to do that)
  • users can opt-in to aligned metrics by setting otel.jmx.target.source to instrumentation or auto to expose themselves to more recent (but moving) metrics definitions until https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/12158 is complete
• wait ~ 3 months for feedback on step 1.
• wait for completion of https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/12158, which implies otel.jmx.target.source set to auto or instrumentation would be equivalent.
• step 2 : in jmxreceiver set otel.jmx.target.source=auto
  • users can opt-out by switching to JMX Gatherer implementation
  • users can partially opt-out by switching to otel.jmx.target.source=legacy
  • communicate breaking change as all captured metrics will change
• wait ~ 3 months for feedback on step 2.
• step 3 : implement an obtrusive way to notify users of JMX Gatherer
  • we can add a gentle warning with a link to migration docs, but unlikely to get noticed
  • we can add an extra mandatory CLI argument making it fail otherwise
  • users of custom groovy metrics definitions need to be aware they will have to migrate
  • users can always opt-out by adding the extra argument or sticking to an older version of JMX Gatherer
• wait ~ 3 months for feedback on step 3.
• step 4 : remove JMX Gatherer from code and published artifacts
  • users can always opt-out by sticking to the latest published (or older) version of JMX Gatherer
  • we can close related issues and remove component:jmx-metrics label.