Allow plugins to persist metadata so they don't need to be loaded to reveal features like their names

	@glyph reported
Trac ID	trac#3773
Type	enhancement
Created	2009-04-14 04:08:30Z

Right now our plugin system goes out of its way to avoid loading any Python code if its cache has been populated, but then in 90% of the cases we actually use plugins, we go ahead and load all of our plugins anyway to ask them what their names and descriptions are. For example, twistd subcommands, and reactor plugins, and reporter plugins.

This was originally my intention with the plugin system. If you look at CachedPlugin, you can see it has name and description attributes; you just can't access the description from anywhere. (And the 'name' is also its attribute name, so it doesn't serve this purpose.)

There should be a facility for plugins to provide metadata which can be stored without loading the module again, so that by the time we're actually loading the dropin file we know we're actually going to be using its code, and we don't need indirection helpers like ServiceMaker and _Reporter and Reactor.

The way I envision this working is that IPlugin (or its successor) would have a couple of methods or attributes. Let's say for the sake of argument that it has "name" and "description". (I am interested in hearing about how to make this flexible without saying something potentially damning, like "pickle whatever you want", but these serve as a good basis for discussion.)

In Twisted, we have a helper, Plugin, which looks like this:

#!py
class Plugin(object):
    "IPlugin implementation for real, loaded plugins"
    implements(IPlugin)
    def __init__(self, name, description):
        self.name = name
        self.description = description
    def willProvide(self, interface):
        return interface.providedBy(self)
    def load(self):
        return self

And something that's like CachedPlugin, now:

#!py
class PluginDescription(object):
    implements(IPlugin)
    def __init__(self, fqpn, realPluginProvides, name, description):
        self.fqpn = fqpn
        self.realPluginProvides = realPluginProvides
        self.name = name
        self.description = description
    def willProvide(self, interface):
        for provided in self.realPluginProvides:
            if provided.isOrExtends(interface):
               return True
        return False
    def load(self):
        return namedAny(self.fqpn)

In your plugin file, you'd do something like this:

#!py
# mymodule.py
class MyPlugin(Plugin):
    implements(IWorkDoer)
    def __init__(self, name, description, secret):
        Plugin.__init__(self, name, description)
        self.secret = secret # NOT persisted
    def doWork(self):
        print self.name, 'doing', self.secret, 'work'

Then later in your dropin, you do this:

#!py
# mydropin.py
from mymodule import MyPlugin, MySpecialPlugin
alpha = MyPlugin("Alpha", "The first plugin", "the alpha")
beta = MyPlugin("Beta", "Second plugin, better", "some other")
# ...

and finally, when you're loading your plugins, you do this:

#!py
def showPlugins():
    for plugin in inspectPlugins(mypackage.plugins):
        if plugin.willProvide(IWorkDoer):
            print "Plugin:", plugin.name
            print '---'
            print plugin.description
            print
def runPlugin(name):
    from twisted.plugin import inspectPlugins
    for plugin in inspectPlugins(mypackage.plugins):
        # "plugin" is documented as providing `IPlugin`,
        # but *not* your interface.
        if plugin.name == name and plugin.willProvide(IWorkDoer):
            realPlugin = plugin.load()
            realPlugin.doWork()

trac-id__3773 3773
type__enhancement enhancement
reporter__glyph glyph
priority__normal normal
milestone__ 
branch__ 
branch_author__ 
status__new new
resolution__None None
component__core core
keywords__ 
time__1239682110000000 1239682110000000
changetime__1302052410000000 1302052410000000
version__None None
owner__ 

Start by reading the existing CachedPlugin, Plugin, IPlugin, and inspectPlugins entry points described in the issue. Trace how plugin metadata is cached and how dropins are inspected without loading Python code. Done means plugin names and descriptions can be read from persisted metadata, while real plugin code loads only when needed.