Clarify scope for integration tests without api keys

Search before asking

• I have searched the Inference issues and found no similar feature requests.

Question

Hi roboflow team. I'm interested in contributing to this project. I saw the CONTRIBUTING.md had a contribution idea of making integration tests not require api keys so I thought that would be a good place to start.

I've been digging into the integration test suite and my current understanding is that almost every integration test that requires a roboflow api key needs it to download a model the test needs from the roboflow api.

I only found one suite, test_workflow_with_google_vision_ocr.py , that has an exception to this, test_workflow_with_google_ocr_without_api_key_via_proxy, where the roboflow api key is used because the test itself is checking if the roboflow api can be used as a proxy in this workflow when no google api key is set. I assume that test would need to be addressed differently than the majority that need to download a model.

Before I start working on an implementation I was hoping you could clarify

1. Would making the integration test suite no longer require api keys still be useful?
2. I'm assuming that the request to make the tests work without an api keys, means the tests themselves don't exist to test that the inference api is able to download models from roboflow as they would need to bypass calling the roboflow api to not require a key. Is that assumption correct?
3. I noticed that most of the tests are asserting just that the response shape is correct and not really asserting that the content/result of the endpoint call has a specific result. For those tests did you have in mind that the api call is bypassed through some sort of mock of the model loading into the modelmanager that can be used for inferencing or whatever the endpoint being tested needs or were you thinking more of actually loading the true model artifact through a different means than the roboflow api?
4. Are there any other constraints I should account for?

Thanks for any guidance. I'm happy to take this on if it'd be useful, or if there's something that would be more helpful I'd appreciate it if you can point me that direction.

Additional

No response

Start with CONTRIBUTING.md and the integration test suite, especially test_workflow_with_google_vision_ocr.py and test_workflow_with_google_ocr_without_api_key_via_proxy. Read how tests download models through the Roboflow API and how the model manager is used for inference. Clarify the intended constraints and define which tests can run without API keys before deciding what done means.