Auto-tuner infrastructure: Auto-tiling

Start with the dataflow graph datastructure described in issue #1002, then trace the memory coloring algorithm and how custom nodes are represented. Define how tile and untile nodes can represent pointer-only transformations and how candidate tiled paths are evaluated. Done means the autotuner can choose which layers and tile counts to use based on cache and kernel information.

ML tasks on large images present a very interesting problem: They thrash the caches. This means that data produced by a kernel may not be found in the cache even by the kernel that immediately follows. This is a huge problem.

The naive mitigation is to tile the image, and run multiple instances of the network, on a SKU dependent manner. SKUs with small caches may run the tiles sequentially, or SKUs with larger caches may run them in parallel. Tiling is not a no-brainer win however: If you tile too much, then you may reduce the occupancy too much, and end up wiping out the gains from higher cache locality and increased hit rates (and possibly even overall reduce the performance). But obviously there will be an optimum, between those two conflicting trends.

The autotuning problem:
To decide which layers to tile. Not necessarily everything should be tiled, maybe its only a few layers that lead to cache thrashing. This could be knowable, using info of the cache sizes in any SKU, and the buffer sizes of each kernel.

Background: The dataflow graph datastructure described in
https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/issues/1002

A way to implement tiling auto-tuning is to just create new nodes in the graph, for example: "tile" custom node, and "untile" custom node.

For example, kernel A acting on D1 and producing D2:
A
D1 -> D2

can also be implemented as:

D1 -> tile(4) -> 4x( "A/4" ) -> untile(4) -> D2

Then the optimization problem is still finding the minimum path from D1 to D2, but now there are more paths to consider!! Note that "tile" and "untile" should not act on data, but only on base pointers in the memory coloring algorithm.