cindex needs vast amounts of temporary space

```
1. Run cindex on about 9 GiB of files (where about half of them are text files)
2. Observe space usage of cindex

What is the expected output?

cindex takes a reasonable amount of temporary space and provides an option to
specify an alternative place to store temporary files.

What do you see instead?

cindex eats up to 5 GiB of space on /tmp while generating a 600 MiB
.csearchindex file.

What version of the product are you using? On what operating system?

tip on Linux with go tip

Please provide any additional information below.

It would be nice if cindex could provide an option to change the temporary
directory as /tmp might not be large enough. Additionally, it might be possible
to lower the space usage of cindex, for instance, it might be possible to merge
(and therefore compress) temporary indices before processing the next set of
files.
```

Original issue reported on code.google.com by `fuz...@gmail.com` on 23 Sep 2013 at 1:23