[BUG] CI does not block PR merge when CI test breaks

### Willingness to contribute

Yes. I can contribute a fix for this bug independently.

### Feathr version

v0.7.2

### System information

- **OS Platform and Distribution (e.g., Linux Ubuntu 20.0)**: ubuntu
- **Python version**: v0.7.2
- **Spark version, if reporting runtime issue**: v0.7.2

### Describe the problem

The ask is block PR merge if CI test not pass.

There might be some dependencies for doing this
- Fix known racing condition CI breaks
- Ensure dedicated cloud resources are reserved for CI test.

### Tracking information

_No response_

### Code to reproduce bug

_No response_

### What component(s) does this bug affect?

- [ ] `Python Client`: This is the client users use to interact with most of our API. Mostly written in Python.
- [ ] `Computation Engine`: The computation engine that execute the actual feature join and generation work. Mostly in Scala and Spark.
- [ ] `Feature Registry API`: The frontend API layer supports SQL, Purview(Atlas) as storage. The API layer is in Python(FAST API)
- [ ] `Feature Registry Web UI`: The Web UI for feature registry. Written in React