Improve finish tasks

Is your feature request related to a problem? Please describe.

While trying to solve #10299, that is adding domain index entries based on late build processing, I've found that there is no event before finishing tasks and after individual documents a processed. The only event is when everything is finished, hence we cannot hook in processing happening in finishing tasks, such as writing indices (for HTML builds). There is also html-collect-pages event which happens too late.

It is possible to add finish task, but they will be executed immediately when using the serial task

https://github.com/sphinx-doc/sphinx/blob/35381cc9b6c49d8369cf3af5098bcce63fc383eb/sphinx/util/parallel.py#L24-L39

Describe the solution you'd like

A way to subscribe finish tasks. That could be done in two ways:

• A Sphinx Core event to connect to where we can work with finishing tasks.
  • Event name ideas:
    • doctree-written(app)
    • before-finish(app)
• Make the tasks (SerialTasks, ParallelTasks) only start when instructed, that would be on finish()

Describe alternatives you've considered

I've tried to change the time domain indices are generated, but that would have broken the rellinks context.

Additional context

Explaination of Sphinx core events, at the time of opening this issue:

1. event.config-inited(app,config)
2. event.builder-inited(app)
3. event.env-get-outdated(app, env, added, changed, removed)
4. event.env-before-read-docs(app, env, docnames)

for docname in docnames:
   5. event.env-purge-doc(app, env, docname)

   if doc changed and not removed:
      6. source-read(app, docname, source)
      7. run source parsers: text -> docutils.document
         - parsers can be added with the app.add_source_parser() API
      8. apply transforms based on priority: docutils.document -> docutils.document
         - event.doctree-read(app, doctree) is called in the middle of transforms,
           transforms come before/after this event depending on their priority.

9. event.env-merge-info(app, env, docnames, other)
   - if running in parallel mode, this event will be emitted for each process

10. event.env-updated(app, env)
11. event.env-get-updated(app, env)
12. event.env-check-consistency(app, env)

# The updated-docs list can be builder dependent, but generally includes all new/changed documents,
# plus any output from `env-get-updated`, and then all "parent" documents in the ToC tree
# For builders that output a single page, they are first joined into a single doctree before post-transforms
# or the doctree-resolved event is emitted
for docname in updated-docs:
   13. apply post-transforms (by priority): docutils.document -> docutils.document
   14. event.doctree-resolved(app, doctree, docname)
       - In the event that any reference nodes fail to resolve, the following may emit:
       - event.missing-reference(env, node, contnode)
       - event.warn-missing-reference(domain, node)

15. Generate output files
16. event.build-finished(app, exception)

Start by reading sphinx/util/parallel.py around SerialTasks and ParallelTasks, then inspect the finish() entry point in sphinx/builders/init.py and the documented Sphinx core events. Compare the proposed event and deferred-task approaches; done means finish tasks can be subscribed to and run at the intended stage in both serial and parallel builds without breaking existing processing.