elastic / elastic/observability-migration-platform

Validate and normalize endpoint URLs before starting a migration

Open
#344 0 comments 0 reactions 0 assignees View on GitHub
asset:dashboards bug phase:4 program:operator-ux source:grafana workstream:stability
Dominant language
Python
Stars
6
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
23

Description

## Problem

`obs-migrate migrate` accepts malformed or inconsistently formatted endpoint URLs and only fails later with low-level `requests` exceptions.

Two cases were encountered during a real Grafana-to-Kibana migration:

1. A Grafana URL ending in `/` generated a double-slash API path:

```text
http://localhost:3000//api/search?type=dash-db&limit=500
```

This resulted in:

```text
requests.exceptions.HTTPError: 404 Client Error: Not Found
```

2. Kibana and Elasticsearch URLs beginning with `hhttps://` were not rejected during argument parsing. The migration successfully extracted and translated the dashboard before failing during upload:

```text
requests.exceptions.InvalidSchema:
No connection adapters were found for
'hhttps://example.kb.elastic.cloud/api/data_views'
```

## Reproduction

```bash
obs-migrate migrate \
--source grafana \
--assets dashboards \
--input-mode api \
--grafana-url http://localhost:3000/ \
--grafana-user admin \
--grafana-pass REDACTED \
--kibana-url hhttps://example.kb.elastic.cloud \
--kibana-api-key REDACTED \
--es-url hhttps://example.es.elastic.cloud:443 \
--es-api-key REDACTED \
--upload
```

## Current behavior

- Endpoint URLs are consumed without early validation.
- Grafana URLs may generate paths containing `//api/...`.
- Malformed target URLs produce internal `requests` tracebacks.
- The pipeline may perform extraction and translation before discovering that the upload endpoint is invalid.
- Errors do not identify the invalid CLI argument or suggest a correction.

## Expected behavior

- Validate all configured endpoint URLs before starting the pipeline.
- Only accept supported `http://` and `https://` schemes.
- Normalize trailing slashes before constructing API paths.
- Report concise, actionable errors without an internal traceback for expected configuration failures.

For example:

```text
Error: invalid value for --kibana-url: 'hhttps://example.kb.elastic.cloud'

The URL must begin with http:// or https://.
Did you mean https://example.kb.elastic.cloud?
```

For an API response failure:

```text
Error: unable to query the Grafana dashboard API.

GET http://localhost:3000/api/search returned HTTP 404.
Verify --grafana-url and confirm that the Grafana API is reachable.
```

## Acceptance criteria

- [ ] Reject malformed Grafana, Kibana, Elasticsearch, Prometheus, and Loki URLs during initial argument/configuration validation.
- [ ] Accept endpoint URLs with or without a trailing slash.
- [ ] Never construct `//api/...` request paths.
- [ ] Identify the invalid CLI flag in the error message.
- [ ] Suggest corrections for recognizable scheme typos such as `hhttps://`.
- [ ] Convert expected connection, authentication, and HTTP failures into actionable CLI errors.
- [ ] Preserve the underlying exception through chaining or debug logging.
- [ ] Never print credentials or API keys in errors.
- [ ] Add tests for valid URLs, trailing slashes, malformed or missing schemes, HTTP 401/403/404, connection refusal, and DNS failure.
- [ ] Verify malformed Kibana/Elasticsearch URLs fail before source extraction begins.

## User impact

Early validation prevents confusing partial runs, avoids unnecessary translation work, and makes common first-run configuration mistakes immediately understandable.

Contributor guide

Open the contributing guide

Research direction

Start at the obs-migrate migrate argument/configuration validation entry point and trace how endpoint values become API paths and how requests failures are surfaced. Add tests covering URL validation, trailing-slash normalization, expected HTTP and connection failures, and credential-safe errors. Done means invalid endpoints fail before extraction and valid endpoints produce actionable CLI errors without internal tracebacks.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, grafana, prometheus, python
Domain
backend, cli, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.