Decide on what/when to instrument code for OTEL and document this

Add some guidelines in the developer guide on what methods to instrument in mx-bluesky / Hyperion and dodal:

At a minimum, suggest
* Significant high level experiment plans / subplans
* Callback events (should check whether bluesky provides any OOTB instrumentation for this
* Interactions with external services (again, need to check how much we get for free)
* Agamemnon instruction fetch/decode
* Nexus file writing

Similarly for dodal:
* Non-trivial device implementations, e.g. arming the eiger
* Instrumentation of ophyd-async may be helpful and reduce the amount we need to instrument directly in dodal
*

## Acceptance Criteria
- Specific criteria that will be used to judge if the issue is fixed