Need more clear documentation regarding cg.backward() and sgd.update() connection

If I renew the computation graph and perform cg.forward() and cg.backward() after each sentence but perform the sgd.update() after say 10 sentences. Will the update be according to all of the sentences or just the last one i.e. are the gradients computed from the backward() step stored somewhere else or are they dependent on the computation graph?