NatLabRockies / NatLabRockies/OpenStudio-server
Nomad executor for external batch: MVP suitability and approach
Nobody has claimed this yet.
- Dominant language
- Ruby
- Stars
- 58
- Forks
- 27
- PR merge metrics
- No merged PRs in 30d
Description
Planning content authored by @anchapin (17fa57cf on the #842 branch), moved out of external_batch/README.md so that file stays contract-only — context in https://github.com/NatLabRockies/OpenStudio-server/pull/842#issuecomment-4875622331. This issue tracks the Nomad-based MVP; when a Nomad executor lands, its docs should land with it (like aws/README.md did).
MVP Suitability for Nomad Deployment
The external batch execution system is well-suited for a Nomad-based MVP with the following characteristics:
✅ Fully Functional for MVP:
- Core simulation execution via OpenStudio CLI
- Result ingestion and status tracking
- LHS sampling with pure-Ruby backend (no R dependency)
- Standard OpenStudio Measure execution
- Baseline model simulations (SmallOffice, MediumOffice, etc.)
- EnergyPlus annual simulations with weather files
- Result extraction (EUI, end-use breakdowns, timeseries data)
✅ Nomad-Compatible Workflow:
- Shared filesystem approach eliminates transport/credential complexities
- Nomad jobs can run the local_executor.rb equivalent directly
- No Docker/containers required for basic execution
- Works with existing OpenStudio Server deployment patterns
📝 Known Limitations Acceptable for MVP:
All documented limitations represent either advanced features not required for basic functionality or visualization enhancements that don't affect result validity. The core simulation and result processing pipeline remains fully operational.
Recommended MVP Approach
- Deployment Target: Nomad cluster with shared storage (NFS, SMB, or similar)
- Executor Pattern: Adapt local_executor.rb for Nomad job execution
- Analysis Types: Standard OpenStudio Server analyses (avoid UrbanOpt/custom gemfile for now)
- Result Access: Standard MongoDB endpoints for EUI, timeseries, etc.
- Future Enhancements: Address limitations post-MVP based on user feedback
Limitation impact assessment (per v1 limitation)
- UrbanOpt analyses and per-analysis
gemfileare not supported (packager raises). These features require specialized bundle install and custom Gemfile handling that isn't available in the external batch executor environment. Advanced features primarily used in testing/specialized workflows; most standard analyses do not use them. - Datapoint initialize/finalize shell scripts run on POSIX executors only. Primarily used for testing/custom initialization; not required for basic simulation operation. Initial Nomad MVP will target Linux-based execution, so this doesn't bite.
- Preflight histogram images are not generated by the Ruby sampling backend (needs R/cairo). Visualization only — core simulation results (EUI, timeseries, etc.) remain fully usable via standard endpoints.
- Transport to remote executors is manual/scripted. NOT a limitation for shared-filesystem deployments: a Nomad job running the
local_executor.rbpattern against mounted storage accesses package/results directly, no credential handling needed beyond filesystem permissions.
Editor's note (brianlball): one correction from the original text — analyses don't need "urbanopt": false / "gemfile": false set explicitly in the analysis JSON; the packager only raises when those features are enabled, absent is fine.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with external_batch/README.md and the local_executor.rb pattern, then review the linked PR 842 context and aws/README.md for the existing executor documentation approach. Define the Nomad MVP around shared storage, standard analyses, and the documented limitations; done means a working Nomad executor with accompanying documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ruby
- Domain
- infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100