[Feature] Add jobs-service deployable to k8s recipes
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 399
- Forks
- 80
- Avg merge
- 3d 23h
- Merged PRs (30d)
- 15
Description
### Willingness to contribute
{"label"=>"No. I cannot contribute a bug fix at this time.", "value"=>"cannot_contribute"}
### Feature Request Proposal
OH runs three services: tables, hts, and jobs. All of them can be tried out in local docker deployable as documented in SETUP.md
For k8s deployable, we only have recipe to deploy tables and hts. Jobs need to be added to the deployable.
https://github.com/linkedin/openhouse/blob/main/DEPLOY.md
The issue should add the necessary k8s deployment spec for jobs service. And plug it into minikube and azure deployable.
### Motivation
Jobs service is a critical piece for OH control plane for managing 10s of thousands of jobs at scale.
### What component(s) does this feature affect?
- [ ] `Table Service`: This is the RESTful catalog service that stores table metadata. `:services:tables`
- [X] `Jobs Service`: This is the job orchestrator that submits data services for table maintenance. `:services:jobs`
- [ ] `Data Services`: This is the jobs that performs table maintenance. `apps:spark`
- [ ] `Iceberg internal catalog`: This is the internal Iceberg catalog for OpenHouse Catalog Service. `:iceberg:openhouse`
- [ ] `Spark Client Integration`: This is the Apache Spark integration for OpenHouse catalog. `:integration:spark`
- [ ] `Documentation`: This is the documentation for OpenHouse. `docs`
- [ ] `Local Docker`: This is the local Docker environment for OpenHouse. `infra/recipes/docker-compose`
- [ ] `Other`: Please specify the component.
### Details
_No response_
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with DEPLOY.md and compare the existing Kubernetes recipes for the tables and hts services. Identify the deployment specification needed for the jobs service, then trace how the minikube and Azure deployables include the existing recipes. Done means jobs can be deployed through both documented targets alongside tables and hts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, java, kubernetes
- Domain
- cloud, devops, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100