Clean up preferences handling and make them more consistent

Currently, there are a lot of places where configuration values can come from. There is preferences.txt, boards.txt, platform.txt, programmers.txt and some runtime generated values. Each of these somehow contribute to the values used for building, uploading, showing the interface, etc., but I believe this process is not entirely clear. Some values can be set in one place, some can be overridden in other places (but not all), etc.

It would be good to:
- Have better documentation about the available options
- Make the way options are used more consistent
- Try to keep backward compatibility as much as possible (perhaps by adding an explicit version number to all these files?)

As part of this, overriding options should probably be more explicit, and less accidental or ad-hoc as it is now. Explicitely specifying overrides (using e.g. an `override.` prefix) could help here, and/or making the source of a preference (or perhaps more the intended use of it) using prefixes like `board.`, `build.`, `platform.`, `pref.` etc. I'm not sure how this should work exactly yet, but some thought should go into this in any case.

@lmihalkovic voiced similar ideas here: https://github.com/arduino/Arduino/pull/4102#issuecomment-171289157

Another related issue that should probably be fixed as part of this, is allowing custom compiler flags, as discussed in #421.

Start by inventorying how preferences.txt, boards.txt, platform.txt, programmers.txt, and runtime-generated values contribute to building, uploading, and the interface. Read the linked pull-request discussion and related issue #421; done requires an agreed, documented, consistent preference and override model that preserves backward compatibility and addresses custom compiler flags.

Written by the indexing model from the issue text.