Load Plugins at Runtime for the Python API

I have been thoroughly enjoying and using the Python API from this library for a research project related to language models and hardware design.

I understand the use case of the plugin system is designed so that plugins are developed and installed separately from the main package. However, it is nice to be able to define models and other components and load them at runtime without having to develop a separate package/project.

After reading the source code, I was able to come up with a simple working example to load a model plugin at runtime:

import importlib

def load_plugin_module(name, module_path):
    mod = importlib.import_module(module_path)
    llm.plugins.pm.register(mod, name)

if __name__ == "__main__":
    load_plugin_module("my_markov_plugin", "my_models.markov")
    model = llm.get_model("markov")

This is just an initial idea to motivate the inclusion of utility functions to load plugins at runtime. I want to support loading specific classes directly rather than whole modules, but I am having trouble understanding how the Pluggy system works and how to achieve this at runtime.

Would like to hear what others also think about this idea.

Thanks!

Start with the Python API plugin flow shown in the issue: importlib, llm.plugins.pm.register, and llm.get_model("markov"). Read how the Pluggy-based plugin manager currently registers modules, then define whether runtime loading should support whole modules, individual classes, or both. Done should be a decided and documented runtime-loading behavior with a clear resolution path for registered models.