Improved CI/CD pipeline

Feature request

Description

Current Pipeline

Each repo has its own testing, mostly using github actions and a jenkins instance that does deployment for all the repos.

Problem

• Each repo can only tests code within itself, there are cases where changes in one repo breaks another repo and we wouldn't know about them until someone tries to pull the new changes and realize things are not working.
• Sometimes there are meta PRs that spans across multiple repos, the current pipeline is unable to test that the system is working if all the PRs are merged.
• The deployment pipeline only supports ros packages, most of the packages in rmf-web are not ros packages so can't be deployed. We also can't build a custom pipeline in rmf-web because some packages are dependent on other repos.

Implementation considerations

An ideal pipeline should be able to

• Perform integration and e2e tests across the whole rmf system.
• Perform testing on multi repo PRs, i.e. performing the tests with the correct branch of each repo.
• Secure, being a public project, the CI/CD server is prone to injection attacks, an example is a malicious actor that submits a PR that changes the build script to include malicious code that steals secrets and send them to the attacker.
  • The CI/CD pipelines needs to verify that a PR is trusted before running it.
  • It should also run all work in an isolated environment.
• Support multiple types of deployment
  • deploy ros packages to a deb repo
  • deploy ros packages to a rpm repo
  • publish npm packages to npmjs.com or other package repositories
  • publish docker images to dockerhub.com or other registries
  • publish python packages to pypi.org or other package repositories
  • deploying rmf to a testing and staging environment

Alternatives

Additional information

Begin by reviewing the existing GitHub Actions workflows and Jenkins deployment setup across the repositories, including how ROS, Python, npm, and Docker artifacts are currently handled. Define the scope and acceptance criteria for cross-repository testing, trusted isolated execution, and the requested deployment targets before proposing an implementation.