`persistent-template` TH performance seems to get worse the more embedded entities are added to entities

I have been doing some of my own benchmarking around the performance of persistent-template and I've noticed that the perf for the TH gets worse the more relations you add.

You can see what I've been doing here https://github.com/yesodweb/persistent/compare/master...danbroooks:bench?expand=1#diff-4ce9d2d6083cd55da50aa29369fbc263

But specifically, I've added some model declarations that add more and more relations like so:

King
  field1 Text Maybe
  field2 Text Maybe
  field3 Text Maybe
  field4 Text Maybe
  field5 Text Maybe
  queen Queen Maybe

Queen
  field1 Text Maybe
  field2 Text Maybe
  field3 Text Maybe
  field4 Text Maybe
  field5 Text Maybe
  jack Jack Maybe

Jack
  field1 Text Maybe
  field2 Text Maybe
  field3 Text Maybe
  field4 Text Maybe
  field5 Text Maybe
  king King Maybe

And for each one, the relations at the bottom get upped by one, up until the entities having 9 relations each.

So here is the output when it is run through the benchmarks:

persistent-template> benchmarks
Running 1 benchmarks...          
Benchmark persistent-th-bench: RUNNING...
benchmarking mkPersist/From File/model-relations-2
time                 8.886 ms   (8.807 ms .. 8.968 ms)
                     0.999 R²   (0.997 R² .. 1.000 R²)
mean                 8.939 ms   (8.879 ms .. 9.047 ms)
std dev              223.0 μs   (108.8 μs .. 347.6 μs)
                                 
benchmarking mkPersist/From File/model-relations-3
time                 18.68 ms   (18.42 ms .. 18.91 ms)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 18.69 ms   (18.53 ms .. 18.89 ms)
std dev              442.0 μs   (302.9 μs .. 624.1 μs)
                                 
benchmarking mkPersist/From File/model-relations-4
time                 35.18 ms   (34.68 ms .. 35.69 ms)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 35.49 ms   (35.10 ms .. 36.06 ms)
std dev              941.8 μs   (611.8 μs .. 1.396 ms)
                                 
benchmarking mkPersist/From File/model-relations-5
time                 61.99 ms   (60.48 ms .. 63.30 ms)
                     0.999 R²   (0.997 R² .. 1.000 R²)
mean                 61.38 ms   (58.43 ms .. 62.88 ms)
std dev              3.836 ms   (1.548 ms .. 6.667 ms)
variance introduced by outliers: 16% (moderately inflated)
                                 
benchmarking mkPersist/From File/model-relations-6
time                 99.56 ms   (94.91 ms .. 104.2 ms)
                     0.996 R²   (0.991 R² .. 0.999 R²)
mean                 100.2 ms   (94.15 ms .. 103.7 ms)
std dev              7.404 ms   (3.296 ms .. 11.29 ms)
variance introduced by outliers: 21% (moderately inflated)
                                 
benchmarking mkPersist/From File/model-relations-7
time                 158.8 ms   (141.5 ms .. 165.7 ms)
                     0.994 R²   (0.981 R² .. 0.999 R²)
mean                 155.6 ms   (144.1 ms .. 162.1 ms)
std dev              12.90 ms   (5.297 ms .. 19.98 ms)
variance introduced by outliers: 26% (moderately inflated)
                                 
benchmarking mkPersist/From File/model-relations-8
time                 234.1 ms   (202.1 ms .. 263.3 ms)
                     0.991 R²   (0.977 R² .. 1.000 R²)
mean                 230.6 ms   (208.6 ms .. 242.7 ms)
std dev              22.28 ms   (8.377 ms .. 33.76 ms)
variance introduced by outliers: 30% (moderately inflated)
                                 
benchmarking mkPersist/From File/model-relations-9
time                 359.2 ms   (250.4 ms .. 437.8 ms)
                     0.986 R²   (0.979 R² .. 1.000 R²)
mean                 330.9 ms   (270.7 ms .. 354.1 ms)
std dev              41.81 ms   (9.543 ms .. 53.53 ms)
variance introduced by outliers: 23% (moderately inflated)
                                 
Benchmark persistent-th-bench: FINISH
Completed 3 action(s).     

As you can see the difference between each average time seems to increase in a non-linear fashion. This might be unavoidable though, and perhaps just the nature of how TH is working under the hood.

My first thought was there might be something in persistent-template that is implemented in an inefficient way, but I've not been able to find it... but it must be down to additional work it is doing because these fields are relations... if I switch the fields to all Text the time it takes to run is far far lower (though I think this is because it is generating less haskell, I think?), and doesnt increase in the same fashion. Here is model-relations-1 compared to model-relations-9 when all fields are set to Text:

persistent-template> benchmarks
Running 1 benchmarks...          
Benchmark persistent-th-bench: RUNNING...
benchmarking mkPersist/From File/model-relations-1
time                 1.735 ms   (1.715 ms .. 1.757 ms)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 1.723 ms   (1.709 ms .. 1.741 ms)
std dev              52.89 μs   (41.79 μs .. 68.08 μs)
variance introduced by outliers: 17% (moderately inflated)
                                 
benchmarking mkPersist/From File/model-relations-9
time                 9.769 ms   (9.656 ms .. 9.907 ms)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 9.746 ms   (9.674 ms .. 9.812 ms)
std dev              188.0 μs   (135.6 μs .. 274.2 μs)
                                 
Benchmark persistent-th-bench: FINISH
Completed 3 action(s).           

I wanted to share what I had found here, in case there was some optimisation that could be made here, or if anybody who is closer to the project would be able to shed some light on why what I have found is the way it is.

Start with the linked benchmark comparison and the persistent-th-bench cases for model-relations-2 through model-relations-9. Compare the relation-based models with the all-Text models, then determine whether the nonlinear compile-time growth is avoidable; done means an identified cause with an optimization or a documented explanation.