invoke-ai / invoke-ai/InvokeAI

Improve InvokeAI import patterns

Open
#6,575 0 comments 0 reactions 1 assignee Claimed by @RyanJDick View on GitHub
enhancement
Dominant language
Python
Stars
28.2k
Forks
3k
Avg merge
6d 5h
Merged PRs (30d)
19

Description

## Problem

The InvokeAI import graph is a tangled web ripe with circular imports and import-order dependencies. We currently work around these issues with short-term hacks, but they are a hindrance to development. The good news is that these issues are generally easy to avoid by following some simple patterns.

Examples of workarounds for import issues:
- https://github.com/invoke-ai/InvokeAI/blob/bb6ff4cf37db3d0dac2c260e662c0009c8b27bdb/invokeai/app/services/config/config_default.py#L159
- https://github.com/invoke-ai/InvokeAI/blob/bb6ff4cf37db3d0dac2c260e662c0009c8b27bdb/invokeai/backend/ip_adapter/ip_adapter.py#L140-L141
- https://github.com/invoke-ai/InvokeAI/blob/bb6ff4cf37db3d0dac2c260e662c0009c8b27bdb/invokeai/backend/raw_model.py#L3-L7
- https://github.com/invoke-ai/invoke-training/blob/main/src/invoke_training/model_merge/scripts/merge_lora_into_model.py#L10-L14

## Proposal

Here is a description of the direction that I think we should be moving to solve these problems (discussed offline with several members of the Invoke team):
1. Cross-directory imports should be one-directional. For example, files in `invokeai/app/invocations` are expected to import from `invokeai.backend`, but files in `invokeai/backend` should not import from `invokeai.app.invocations`.
2. Avoid re-exporting symbols from `__init__.py` files unnecessarily. This pattern makes sense for building clean public APIs, but most of our codebase is non-public (with the exception of `invokeai.invocation_api`). Also, re-exporting from `__init__.py` tends to lead to eager importing of modules before they are needed, which exacerbates circular import issues if the import tree is poorly structured, and could be contributing to the slow app launch times.
3. We have a bad habit of doing *stuff* on import. During import, we should only be defining classes/functions/constants - not executing code. This anti-pattern has led to several import-order dependencies.
4. Use absolute imports rather than relative imports. This is primarily a style preference, but PEP 8 recommends absolute imports, and in my experience they work better with Python LSP services (e.g. for auto-import and refactoring features). We currently have a mix of absolute and relative imports (mostly absolute) - we should standardize this. Also, if we enforce absolute imports with a linter, this makes refactoring easier. With absolute imports, if I move `invokeai.app.service.some.module`, then I can confidently search and replace all imports of this module. With relative imports, it is harder to find all references to the moved module.

## Next Steps

This is intended to be a long-lived issue as we gradually work towards the proposed goal state over time. We won't try to solve all of these problem in a single pass.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.