AbsaOSS / AbsaOSS/enceladus

HDFS Browser calls backend too many times

Aperta
#1,686 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
bug priority: undecided
Lingua principale
Scala
Stelle
33
Fork
16
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

In situations where a non-HDFS path is defined in MappingTable/Dataset dialog (valid in "simple mode" when the HDFS Browser is off), switching to HDFS Browser path selection mode results in an excessive amount of backend service calls.

Some extra information and reasoning can be seen at https://github.com/AbsaOSS/enceladus/pull/1672#issuecomment-777450325.

## To Reproduce
Steps to reproduce the behavior
1. _Edit_ existing schema or try to _Add_ a new one
2. Have simple (non hdfs-browser) mode enabled, enter non-hdfs path, e.g. `s3://mybucket123/path/to/somewhere`
3. Open browser console and watch for erroneous ajax request (or inspect the Ajax call being made using different means)
4. See that there are more than 1 error request being made for each non-HDFS path (500 or 404 - based on the path you enter)

## Expected behavior
Each HDFS browser with an invalid path should result in only 1 failed ajax call.

## Additional context
Possibly, a discussion can be held on how/if at all the current from of the HDFS Brower should behave for s3-path. (Simple fail, limited listing support?).

Technically, it would seem that some of the suspicions of the root cause may be:
- `HDFSBrowserRenderer.render.js:29` calls `_treeNavigateTo(...)` that results in the REST AJAX call being made. Why this renderer get refreshed multiple times and even if this call should be part of the rendering process remains an open question.
- HDFSBrowser attempts to break path by `/`s and list the tokenized path segments. This only seems suitable for local HDFS paths (starting with `/`). Non-local HDFS paths (`hdfs://myservice.abcd:12345/example/remote/hdfs/path`) or s3-style paths (`s3://bucket1234/path/on/s3`) do not seem to be currently supported in the HDFS Browser

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.