An explicit `trainer`'s method to trigger `callback`'s hook.

🚀 Feature

Consider to add one pair of methods: Trainer.trigger(message), and Callback.on_trigger(message).

Motivation

When I finish my experiment, I want the trainer to trigger a callback to do something (for example, rename output dir).
For example:

# main.py
if args.do_train:
    trainer.fit(...)
    trainer.test(...)
    trainer.trigger("train success")
if args.do_test_only:
    trainer.test(...)
    trainer.trigger("test success")
if args.do_predict_only:
    trainer.predict(...)
    trainer.trigger("predict success")

# xxx_callback.py
class OutputDirManager(Callback)
...
def on_exception(trainer, pl_module, exception)
      ...  # rename output dir to mark this is a fail experiment

def on_trigger(trainer, pl_module, message)
    # rename output dir to mark this is a success experiment
    if message=="train success"
        ...
    if message=="test success"
        ...
    if message=="predict success"
        ...

I think this pair of methods are flexible for wider usages.

cc @awaelchli @ananthsub @rohitgr7

Start by locating the Trainer and Callback definitions and reviewing how existing hooks are dispatched. Define the intended trigger message flow for the fit, test, and predict examples, then verify that callbacks receive the message and that the new API is covered by project tests.