MAGPIE_HOSTNAME_CMD not working with MAGPIE_NO_LOCAL_DIR
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 198
- Forks
- 51
- PR merge metrics
- No merged PRs in 30d
Description
Hello, we have a setup in our data center that requires both `MAGPIE_HOSTNAME_CMD` and `MAGPIE_NO_LOCAL_DIR` to be used. However, the `no-local-dir` patches use `` myhostname=`hostname` ``, so they do not respect `MAGPIE_HOSTNAME_CMD` or `MAGPIE_HOSTNAME_CMD_MAP`. As a result, there is a mismatch between the config file directories used by Magpie and the ones that e.g. Hadoop or Spark recognize.
I'm willing to contribute a PR to fix this, though I'm not 100% sure what the best approach is. For Hadoop, I think we could add a `magpie_hostname` to the `hadoop-user-functions.sh` and use that for setting `myhostname` in all the setup scripts. For other software (taking Spark as example), I think we can add `MAGPIE_HOSTNAME_CMD` and `MAGPIE_HOSTNAME_CMD_MAP` to `spark-env.sh`, and add a `magpie_hostname` function to each of the setup scripts.
Does that approach make sense to you, @chu11? And if so, would you like me to draft a PR implementing this?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin with the no-local-dir patches and the Hadoop hadoop-user-functions.sh and Spark spark-env.sh/setup scripts named in the issue. Trace how myhostname is set, then verify that MAGPIE_HOSTNAME_CMD and MAGPIE_HOSTNAME_CMD_MAP are honored consistently so Magpie, Hadoop, and Spark select matching configuration directories.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, shell, spark
- Domain
- data-engineering, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100