add callbacks dynamically

Hi!

this is somewhat of a mixture of a feature request and a bug report. here is my example:

import dash
import dash_core_components as dcc
import dash_html_components as html
from dash.dependencies import Input, Output, State
import json
import os

external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css']
app = dash.Dash(__name__, external_stylesheets=external_stylesheets)
app.config.suppress_callback_exceptions=True

try:
    os.remove('cities.json')
except OSError:
    pass


def make_cb(city):
    @app.callback(Output('%s-pushed' % city, 'children'),
                  [Input('%s-but' % city, 'n_clicks')],
                  [State('%s-but' % city, 'children')])
    def fing(nc,cval):
        if nc:
            return cval
        return ''


def get_layout():
    
    try:
        cities = json.load(open('cities.json'))
    except FileNotFoundError:
        cities = []
    
    return html.Div([
                dcc.Input(id='input-1', type='text', value='Montréal'),
                html.Button(id='submit-button', n_clicks=0, children='Submit'),
                html.Div(id='pushed-city'),
                html.Div([html.Div([html.Button(c,id='%s-but' % c),
                                    html.Div(id='%s-pushed' % c)])
                          for c in cities],id='output-state')
            ])


app.layout = get_layout


@app.callback(Output('output-state', 'children'),
              [Input('submit-button', 'n_clicks')],
              [State('input-1', 'value')])
def update_output(n_clicks, input1):
    try:
        cities = set(json.load(open('cities.json')))
    except FileNotFoundError:
        cities = set()
    
    if input1 not in cities and input1 is not None:
        cities.add(input1)
        cities = list(cities)
        json.dump(cities,open('cities.json','w'))
        make_cb(input1)
    print(app.callback_map)
    return [html.Div([html.Button(c,id='%s-but' % c),
                                    html.Div(id='%s-pushed' % c)])
                          for c in cities]


if __name__ == '__main__':
    app.run_server(debug=True)

this is a simple app based on the documentation, that allows the user to add interactive elements, with some caching. There are two issues. The first one I don't understand, the second one I do.

1. If I click the submit button, to add a city, everything appears fine, the print(app.callback_map) line shows me that the callback has been registered, i can see everything in the browser, however the callback does not become active, until I refresh the browser.

2. If I use a proper server like gunicorn not the flask development server, with parallel processes, the callback_map dictionary is not shared, so the whole thing breaks down, and the callbacks basically don't work.

Now, I'm quite sure that 1, has some easy explanation that I'm just a little too tired to figure out, I wrote this whole thing because of 2,

I also hacked together a solution for myself for the second one, that allows for declaring the callback map as a shared key-value store, using redis. If anyone's interested, I can post a PR. As far as i can see, only __iter__, __getitem__, __setitem__ and items() are utilized from the callback_map.

I looked through #577 #475 #373 this and this. Based on these, creating a shared key-value store for the callbacks is a well isolated issue that fits the main arch of these topics.

Start by tracing how Dash registers and consumes app.callback_map during callback dispatch, using the dynamic-callback example in the issue as the reproduction. Investigate the implications of storing callback metadata in a shared Redis-backed key-value store across worker processes. Done means dynamically added callbacks work without a browser refresh and remain available with multiple server processes.