contribute checking for "miscaptured" variables in loops

Original report by daira (@daira?) on Launchpad:

Tahoe-LAFS, a large Python codebase implementing a distributed filesystem, uses a script called "check-miscaptures" ( https://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/misc/coding_tools/check-miscaptures.py) to catch a class of errors resulting in unexpected behaviour when capturing variables in loops.

An example of incorrect code that it would catch is:

  a = [None]*10
  for i in range(0, 10):
       a[i] = lambda x: x+i

  # The programmer expected a[x](y) to be equivalent to x+y


  print a[2](2)  # not 4, but 11

This happens because all of the a[i] capture the same mutable variable i, which is 9 after the loop when the lambda is called.

For this example, the output of check-miscaptures is:

Checking miscapture.py...
miscapture.py:3 captures 'i' assigned at line 2
1 suspiciously captured variables in 1 out of 1 file(s).

Changing the loop body to "a[i] = lambda x, i=i: x+i" fixes the problem, and check-miscaptures will not report a warning for the fixed code (it rarely gives false positives, and when it does they are easy to suppress).

The same problem can happen with named functions rather than lambdas, and with other kinds of loop or list comprehensions. More realistic examples are common in programs that define lots of callback functions, for example when using asynchronous libraries such as Twisted.

This ticket is to integrate the checking done by this script into pyflakes.

Start by reading the linked check-miscaptures.py script and its example output to understand the warnings it produces for lambdas, named functions, loops, and comprehensions. Then inspect pyflakes to find the existing source-checking entry points; done means the same suspicious captures are reported by pyflakes, while the shown default-argument form is not.