Method profiles

@dpo
# Summary:

The following script profiles:
- `evaluate_expr_tree()`;
- `get_expression_tree()`;
- `extract_element_functions()`;
- `complete_tree()`;
- `get_elemental_variables()`;
- `cast_type_of_constant()`;
- `set_bounds!()`;
- `set_convexity!()`.

All these methods walk recursively in the expression tree, which produce a truncated flame graph (the stack of call is too big).

Only `evaluate_expr_tree()`, `get_expression_tree()` and `extract_element_functions()` are applied on the expression tree extracted from the `NLPModel` in PartiallySeparableNLPModel.
The other methods are usually call on the distinct element expression trees.
I called these methods on the original expression trees to get comparable results.
The details of the benchmark and the flame graph are respectively in the code and in join pieces.

## Conclusions:
- All the functions manipulating the expression tree have the same performances, around 2~8ms;
- Some methods modifying the tree are type unstable, but it is the cost of the flexibility.
The only problematic method with unstable typing is `evaluate_expr_tree`.
- The evaluation of the expression tree is the fastest method (800 μs), but it is slow compare to ADNLPModel (130 μs) or MathOptNLPModel (440 ns);
- The creation of the expression tree is the slowest method ~20 ms.

### Notes
I made profiles from expression trees extract from both `JuMP.Model` or `ADNLPModel`.
The `Expr` returned by a JuMP model is more poised than the `Expr` returned by an ADNLP (see the end of the file).
It induces a small overhead for every method called on `ADNLPModel`.

```julia
using ExpressionTreeForge
using BenchmarkTools, ProfileSVG
using NLPModels, ADNLPModels, NLPModelsJuMP
using OptimizationProblems, OptimizationProblems.PureJuMP, OptimizationProblems.ADNLPProblems
using ModelingToolkit

path = pwd()*"/dvpt/profilage/profiles/"

n = 500
jump_model = PureJuMP.arwhead(; n)
nlp_ad = ADNLPProblems.arwhead(; n)
nlp_mo = MathOptNLPModel(jump_model)

expr_jump = get_expression_tree(jump_model)
expr_ad = get_expression_tree(nlp_ad)

ProfileSVG.set_default(maxdepth=100)

## evaluate_expr_tree
x = ones(n)

@code_warntype evaluate_expr_tree(complete_jump, x)
@benchmark evaluate_expr_tree(complete_jump, x)
p = ProfileSVG.@profview (@benchmark evaluate_expr_tree(complete_jump, x))
ProfileSVG.save(path * "evaluate_expr_tree" * "_jump" * ".svg")
# BenchmarkTools.Trial: 5761 samples with 1 evaluation.
# Range (min … max): 721.900 μs … 13.861 ms ┊ GC (min … max): 0.00% … 93.24%
# Time (median): 786.700 μs ┊ GC (median): 0.00%
# Time (mean ± σ): 865.044 μs ± 623.942 μs ┊ GC (mean ± σ): 3.53% ± 4.65%
# █▄▁▇▃ ▁
# ▆█████▅▅▄▅██▅▃▃▃▃▄▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▂
# 722 μs Histogram: frequency by time 1.35 ms <
# Memory estimate: 347.14 KiB, allocs estimate: 9483.

@code_warntype evaluate_expr_tree(complete_ad, x)
@benchmark evaluate_expr_tree(complete_ad, x)
p = ProfileSVG.@profview (@benchmark evaluate_expr_tree(complete_ad, x))
ProfileSVG.save(path * "evaluate_expr_tree" * "_ad" * ".svg")
# BenchmarkTools.Trial: 5611 samples with 1 evaluation.
# Range (min … max): 782.500 μs … 13.278 ms ┊ GC (min … max): 0.00% … 92.06%
# Time (median): 812.900 μs ┊ GC (median): 0.00%
# Time (mean ± σ): 888.247 μs ± 632.920 μs ┊ GC (mean ± σ): 3.64% ± 4.84%
# ▇█▇▅▅▄▄▅▄▃▂▂▁▁▂▁ ▂
# ████████████████████▇██▇▇▇▆▅▇▅▇▅▅▆▆▆▇▅▅▅▅▆▅▆▅▆▆▅▅▅▃▅▁▃▄▄▁▃▆▃▅ █
# 782 μs Histogram: log(frequency) by time 1.57 ms <
# Memory estimate: 374.16 KiB, allocs estimate: 9479.

#= SUMMARY
Unstable type.
This a the weakest method of ExpressionTreeForge compare to ADNLPModel or JuMP.
If we compare to the
- MathOptNLPModel @benchmark NLPModels.obj(nlp_jump, x)
BenchmarkTools.Trial: 10000 samples with 199 evaluations.
Range (min … max): 425.126 ns … 2.141 μs ┊ GC (min … max): 0.00% … 0.00%
Time (median): 440.704 ns ┊ GC (median): 0.00%
Time (mean ± σ): 489.979 ns ± 120.268 ns ┊ GC (mean ± σ): 0.00% ± 0.00%
█▆▄▁ ▄▅▃▂▄▃ ▁▁▁ ▁
████▇███████▇▇▆▇█████████▇█▆▇▆▆▇▆▇▆█▇▇▇▇▆▆▇▅▆▆▆▄▅▅▆▅▅▄▅▅▄▅▃▄▅ █
425 ns Histogram: log(frequency) by time 991 ns <
Memory estimate: 16 bytes, allocs estimate: 1.

- ADNLPModel @benchmark NLPModels.obj(nlp_ad, x)
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
Range (min … max): 97.000 μs … 15.615 ms ┊ GC (min … max): 0.00% … 98.78%
Time (median): 129.800 μs ┊ GC (median): 0.00%
Time (mean ± σ): 136.756 μs ± 337.880 μs ┊ GC (mean ± σ): 5.50% ± 2.21%
█ ▂▅
▁▁▁▂▁▁▂▅▄▅██▂▂▂▂▂▁▁▅▆▄▃▃▂▂███▃▄▄▃▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▂
97 μs Histogram: frequency by time 186 μs <
Memory estimate: 70.27 KiB, allocs estimate: 4493.

=#

## get_expression_tree
@code_warntype get_expression_tree(jump_model)
@benchmark get_expression_tree(jump_model)
p = ProfileSVG.@profview (@benchmark get_expression_tree(jump_model))
ProfileSVG.save(path * string(get_expression_tree) * "_jump" * ".svg")
# BenchmarkTools.Trial: 569 samples with 1 evaluation.
# Range (min … max): 6.426 ms … 38.781 ms ┊ GC (min … max): 0.00% … 72.79%
# Time (median): 6.994 ms ┊ GC (median): 0.00%
# Time (mean ± σ): 8.781 ms ± 4.357 ms ┊ GC (mean ± σ): 6.05% ± 11.68%
# █▇▇▃▁
# ██████▆▇▅▅▅▇▅▅▄▆▅▄▅▄▆▅▅▅▅▅▅▅▆▆▅▇██▇█▆▅▅▁▄▁▁▁▁▁▁▄▁▁▁▁▁▄▄▇▄▆ ▇
# 6.43 ms Histogram: log(frequency) by time 23.7 ms <
# Memory estimate: 3.62 MiB, allocs estimate: 68496.

@code_warntype get_expression_tree(nlp_ad)
@benchmark get_expression_tree(nlp_ad)
p = ProfileSVG.@profview (@benchmark get_expression_tree(nlp_ad))
ProfileSVG.save(path * string(get_expression_tree) * "_ad" * ".svg")
# BenchmarkTools.Trial: 191 samples with 1 evaluation.
# Range (min … max): 17.895 ms … 77.558 ms ┊ GC (min … max): 0.00% … 38.78%
# Time (median): 19.463 ms ┊ GC (median): 0.00%
# Time (mean ± σ): 26.211 ms ± 11.898 ms ┊ GC (mean ± σ): 3.94% ± 9.29%
# █▇▃
# ███▇▆█▄▇▇▄▇▇▆▄▆▄▁▁▄▆▆▇▄▁▄▆█▆▇▇█▇▄▆▁▁▁▄▁▁▁▁▁▄▁▄▁▁▁▁▁▁▁▄▁▁▁▁▄ ▄
# 17.9 ms Histogram: log(frequency) by time 69.5 ms <
# Memory estimate: 6.28 MiB, allocs estimate: 146122.

#= SUMMARY:
- the function is slow.
It is only at the first allocation of a PartiallySeparableSolver;
- unstable types, designed to be like this!
Side effect of the flexibility;
- the recursive calls obstruct the analysis of the flame graph;
- get_expression_tree runs faster on jump model than ADLNPModel.
It is not due to the ModelingToolkit, used to get the Expr from the ADNLPModel.
The lines of code:
ModelingToolkit.@variables x[1:n]
@benchmark nlp_ad.f(x)

BenchmarkTools.Trial: 10000 samples with 1 evaluation.
Range (min … max): 216.000 μs … 22.164 ms ┊ GC (min … max): 0.00% … 96.54%
Time (median): 250.700 μs ┊ GC (median): 0.00%
Time (mean ± σ): 403.276 μs ± 954.136 μs ┊ GC (mean ± σ): 15.70% ± 6.71%

█▆▅▄▄▄▃▃▂▂▁▂▃▃▂▂ ▂ ▁▁▁▁ ▁▁ ▃▄▃▁▁▁▁ ▂
█████████████████████████████████████▇▇▇▆▆▅▆▇▇▅▆▅▆▆▅▅▅▃▅▄▆▇▅▅ █
216 μs Histogram: log(frequency) by time 986 μs <
Memory estimate: 600.31 KiB, allocs estimate: 16466.
are not the main issue.
=#

## extract_element_functions
@code_warntype extract_element_functions(expr_jump)
@benchmark extract_element_functions(expr_jump)
p = ProfileSVG.@profview (@benchmark extract_element_functions(expr_jump))
ProfileSVG.save(path * string(extract_element_functions) * "_jump" * ".svg")
#=
BenchmarkTools.Trial: 3093 samples with 1 evaluation.
Range (min … max): 874.400 μs … 103.354 ms ┊ GC (min … max): 0.00% … 97.48%
Time (median): 1.357 ms ┊ GC (median): 0.00%
Time (mean ± σ): 1.598 ms ± 3.038 ms ┊ GC (mean ± σ): 6.67% ± 3.51%
▁▃█▅▄▅▄▂▃▁ ▂ ▁
▁▂▅██████████████▇▆▆▇▇▇█▇▆▅▅▃▃▄▄▂▃▃▃▃▃▃▃▃▂▃▂▂▂▂▂▂▂▂▂▂▂▂▃▂▂▃▂▂ ▃
874 μs Histogram: frequency by time 2.82 ms <
Memory estimate: 653.91 KiB, allocs estimate: 16992.
=#

@code_warntype extract_element_functions(expr_ad)
@benchmark extract_element_functions(expr_ad)
p = ProfileSVG.@profview (@benchmark extract_element_functions(expr_ad))
ProfileSVG.save(path * string(extract_element_functions) * "_ad" * ".svg")
#=
BenchmarkTools.Trial: 951 samples with 1 evaluation.
Range (min … max): 2.985 ms … 29.612 ms ┊ GC (min … max): 0.00% … 61.10%
Time (median): 3.787 ms ┊ GC (median): 0.00%
Time (mean ± σ): 5.245 ms ± 3.185 ms ┊ GC (mean ± σ): 7.04% ± 11.75%
█▆▅▅▂▃ ▁ ▁ ▂▂▃▃▃▃▁▁
████████████████████████▇▅▆▄▅▄▅▁▁▁▁▁▅▁▅▄▅▅▄▅▅▄▁▁▁▄▅▁▄▁▁▁▁▅ █
2.98 ms Histogram: log(frequency) by time 17.6 ms <
Memory estimate: 3.63 MiB, allocs estimate: 19974.

Which turns into:
BenchmarkTools.Trial: 1949 samples with 1 evaluation.
Range (min … max): 1.917 ms … 17.456 ms ┊ GC (min … max): 0.00% … 86.07%
Time (median): 2.143 ms ┊ GC (median): 0.00%
Time (mean ± σ): 2.559 ms ± 1.655 ms ┊ GC (mean ± σ): 10.94% ± 13.46%
▇█▄▃▅▄▂
████████▇▇▅▄▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▅▅▇▆▇▇ █
1.92 ms Histogram: log(frequency) by time 11.6 ms <
Memory estimate: 3.46 MiB, allocs estimate: 11974.
if I remove the parallelized loop.
=#

#= SUMMARY
Unstable type.
Run faster than the transformation into `Type_expr_tree`.
=#

## complete_tree
@code_warntype complete_tree(expr_jump)
@benchmark complete_tree(expr_jump)
p = ProfileSVG.@profview (@benchmark complete_tree(expr_jump))
ProfileSVG.save(path * string(complete_tree) * "_jump" * ".svg")
# BenchmarkTools.Trial: 1024 samples with 1 evaluation.
# Range (min … max): 3.949 ms … 19.875 ms ┊ GC (min … max): 0.00% … 75.16%
# Time (median): 4.423 ms ┊ GC (median): 0.00%
# Time (mean ± σ): 4.878 ms ± 2.120 ms ┊ GC (mean ± σ): 6.58% ± 11.26%
# ▆█▇▆▅▃▂
# ████████▇▄▄▄▄▁▅▄▁▁▁▄▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▄▆▆▄▅▅ █
# 3.95 ms Histogram: log(frequency) by time 17.8 ms <
# Memory estimate: 2.90 MiB, allocs estimate: 70873.
@code_warntype complete_tree(expr_ad)
@benchmark complete_tree(expr_ad)
p = ProfileSVG.@profview (@benchmark complete_tree(expr_ad))
ProfileSVG.save(path * string(complete_tree) * "_ad" * ".svg")
# BenchmarkTools.Trial: 800 samples with 1 evaluation.
# Range (min … max): 4.655 ms … 21.341 ms ┊ GC (min … max): 0.00% … 67.12%
# Time (median): 5.724 ms ┊ GC (median): 0.00%
# Time (mean ± σ): 6.243 ms ± 2.400 ms ┊ GC (mean ± σ): 6.01% ± 11.25%
# ▄█▆▆▃
# ▄██████▇▄▃▃▂▃▂▂▂▂▂▂▂▂▁▁▁▁▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▂▂▂▂▂ ▃
# 4.65 ms Histogram: frequency by time 19.8 ms <
# Memory estimate: 3.24 MiB, allocs estimate: 78825.

#= SUMMARY
Type stable
We will have to improve the constructor of CompleteTree.
Usually this function is call on element tree, that are much smaller and in less quantity.
=#

# get_elemental_variables
@code_warntype get_elemental_variables(expr_jump)
@benchmark get_elemental_variables(expr_jump)
p = ProfileSVG.@profview (@benchmark get_elemental_variables(expr_jump))
ProfileSVG.save(path * string(get_elemental_variables) * "_jump" * ".svg")
# BenchmarkTools.Trial: 1171 samples with 1 evaluation.
# Range (min … max): 3.447 ms … 18.221 ms ┊ GC (min … max): 0.00% … 74.66%
# Time (median): 3.931 ms ┊ GC (median): 0.00%
# Time (mean ± σ): 4.266 ms ± 1.797 ms ┊ GC (mean ± σ): 5.11% ± 9.59%
# ▇█▇▆▄▃▂
# ████████▇▆▅▁▆▅▅▁▄▁▁▄▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆▅ █
# 3.45 ms Histogram: log(frequency) by time 17.5 ms <
# Memory estimate: 1.86 MiB, allocs estimate: 39942.

@code_warntype get_elemental_variables(expr_ad)
@benchmark get_elemental_variables(expr_ad)
p = ProfileSVG.@profview (@benchmark get_elemental_variables(expr_ad))
ProfileSVG.save(path * string(get_elemental_variables) * "_ad" * ".svg")
# BenchmarkTools.Trial: 603 samples with 1 evaluation.
# Range (min … max): 6.408 ms … 23.576 ms ┊ GC (min … max): 0.00% … 51.44%
# Time (median): 7.371 ms ┊ GC (median): 0.00%
# Time (mean ± σ): 8.282 ms ± 2.871 ms ┊ GC (mean ± σ): 8.10% ± 13.58%
# ▄█▇▃▁
# ▃█████▇▅▅▃▄▃▃▃▂▃▂▂▂▁▁▁▂▁▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▂▃▃▂▂▂▂▁▃▂▂▂▁▂ ▃
# 6.41 ms Histogram: frequency by time 20.2 ms <
# Memory estimate: 7.66 MiB, allocs estimate: 50725.

#= SUMMARY
Type stable.
Run on element expression trees.
=#

T=Float64
@code_warntype ExpressionTreeForge.cast_type_of_constant(expr_jump, T)
@benchmark ExpressionTreeForge.cast_type_of_constant(expr_jump, T)
p = ProfileSVG.@profview (@benchmark ExpressionTreeForge.cast_type_of_constant(expr_jump, T))
ProfileSVG.save(path * "cast_type_of_constant" * "_jump" * ".svg")
# BenchmarkTools.Trial: 626 samples with 1 evaluation.
# Range (min … max): 6.629 ms … 25.828 ms ┊ GC (min … max): 0.00% … 62.93%
# Time (median): 7.589 ms ┊ GC (median): 0.00%
# Time (mean ± σ): 7.980 ms ± 2.004 ms ┊ GC (mean ± σ): 2.82% ± 7.90%
# █▇▆▄▃
# ▇██████▅▄▄▄▃▃▂▁▂▁▁▁▁▁▁▂▁▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▂▂ ▃
# 6.63 ms Histogram: frequency by time 21.7 ms <
# Memory estimate: 1.98 MiB, allocs estimate: 55402.

@code_warntype ExpressionTreeForge.cast_type_of_constant(expr_ad, T)
@benchmark ExpressionTreeForge.cast_type_of_constant(expr_ad, T)
p = ProfileSVG.@profview (@benchmark ExpressionTreeForge.cast_type_of_constant(expr_ad, T))
ProfileSVG.save(path * "cast_type_of_constant" * "_ad" * ".svg")
# BenchmarkTools.Trial: 564 samples with 1 evaluation.
# Range (min … max): 7.546 ms … 24.218 ms ┊ GC (min … max): 0.00% … 56.12%
# Time (median): 8.480 ms ┊ GC (median): 0.00%
# Time (mean ± σ): 8.867 ms ± 1.987 ms ┊ GC (mean ± σ): 2.71% ± 7.97%
# ▅▇█▆▃
# ▇█████▆▆▄▄▃▃▃▂▃▂▂▁▁▂▁▁▁▂▁▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▂ ▃
# 7.55 ms Histogram: frequency by time 21.7 ms <
# Memory estimate: 2.22 MiB, allocs estimate: 61863.

#= SUMMARY
Unstable type.
Applied on element expression trees.
=#
complete_jump = complete_tree(expr_jump)
complete_ad = complete_tree(expr_ad)

@code_warntype set_bounds!(complete_jump)
@benchmark set_bounds!(complete_jump)
p = ProfileSVG.@profview (@benchmark set_bounds!(complete_jump))
ProfileSVG.save(path * "set_bounds!" * "_jump" * ".svg")
# BenchmarkTools.Trial: 545 samples with 1 evaluation.
# Range (min … max): 6.321 ms … 35.232 ms ┊ GC (min … max): 0.00% … 58.28%
# Time (median): 7.898 ms ┊ GC (median): 0.00%
# Time (mean ± σ): 9.169 ms ± 3.634 ms ┊ GC (mean ± σ): 5.64% ± 11.52%
# ▂▇▇▇█▇▆▅▂▁▅▅▅ ▁
# █████████████▇▄▆▇▄▇▁▁▇▇▆▇█▇▇▄▆▄▄▁▄▆▁▇▄▁▁▆▁▁▄▁▁▄▁▆▄▄▆▆▄▆▄▁▆ █
# 6.32 ms Histogram: log(frequency) by time 23.7 ms <
# Memory estimate: 3.94 MiB, allocs estimate: 101318.

@code_warntype set_bounds!(complete_ad)
@benchmark set_bounds!(complete_ad)
p = ProfileSVG.@profview (@benchmark set_bounds!(complete_ad))
ProfileSVG.save(path * "set_bounds!" * "_ad" * ".svg")
# BenchmarkTools.Trial: 257 samples with 1 evaluation.
# Range (min … max): 10.684 ms … 155.810 ms ┊ GC (min … max): 0.00% … 0.00%
# Time (median): 12.555 ms ┊ GC (median): 0.00%
# Time (mean ± σ): 19.564 ms ± 18.780 ms ┊ GC (mean ± σ): 3.51% ± 10.55%
# ▇█▃
# ███▇▆▄▆▅▄▄▅▁▁▅▅▄▄▅▄▁▄▁▁▁▄▆▅▄▄▄▁▄▄▁▄▁▁▆▄▅▄▁▁▅▁▁▅▁▁▁▁▁▁▄▁▁▁▁▄▄ ▅
# 10.7 ms Histogram: log(frequency) by time 90.9 ms <
# Memory estimate: 4.42 MiB, allocs estimate: 112252.

#= SUMMARY
Type stable.
=#

@code_warntype set_convexity!(complete_jump)
@benchmark set_convexity!(complete_jump)
p = ProfileSVG.@profview (@benchmark set_convexity!(complete_jump))
ProfileSVG.save(path * "set_convexity!" * "_jump" * ".svg")
# BenchmarkTools.Trial: 789 samples with 1 evaluation.
# Range (min … max): 4.957 ms … 24.374 ms ┊ GC (min … max): 0.00% … 61.36%
# Time (median): 5.820 ms ┊ GC (median): 0.00%
# Time (mean ± σ): 6.333 ms ± 2.394 ms ┊ GC (mean ± σ): 5.70% ± 10.83%
# ▅█▃▁
# ▅████▇▅▅▃▃▂▂▂▁▂▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▂▂▂▂▁▂ ▃
# 4.96 ms Histogram: frequency by time 20.4 ms <
# Memory estimate: 3.10 MiB, allocs estimate: 71373.

@code_warntype set_convexity!(complete_ad)
@benchmark set_convexity!(complete_ad)
p = ProfileSVG.@profview (@benchmark set_convexity!(complete_ad))
ProfileSVG.save(path * "set_convexity!" * "_ad" * ".svg")
# BenchmarkTools.Trial: 707 samples with 1 evaluation.
# Range (min … max): 5.604 ms … 23.188 ms ┊ GC (min … max): 0.00% … 68.20%
# Time (median): 6.537 ms ┊ GC (median): 0.00%
# Time (mean ± σ): 7.066 ms ± 2.445 ms ┊ GC (mean ± σ): 5.66% ± 11.15%
# ▆██▇▆▅▃▁
# █████████▇▇▆▅▁▄▄▄▁▅▁▁▁▁▁▁▁▁▁▄▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▄▁▆▆▅▆▄ █
# 5.6 ms Histogram: log(frequency) by time 20.6 ms <
# Memory estimate: 3.47 MiB, allocs estimate: 79822.

#= SUMMARY
Type stable.
=#

#=
In the case of arwhead(;n=5), the expression tree from a JuMP model is:
+
+
-
^2.0
+
^2.0
x[1]
^2.0
x[5]
*
4.0
x[1]
3.0
+
-
^2.0
+
^2.0
x[2]
^2.0
x[5]
*
4.0
x[2]
3.0
+
-
^2.0
+
^2.0
x[3]
^2.0
x[5]
*
4.0
x[3]
3.0
+
-
^2.0
+
^2.0
x[4]
^2.0
x[5]
*
4.0
x[4]
3.0
and the expression tree from a ADNLPModel:
+
+
+
+
-
^2
+
^2
x[1]
^2
x[5]
*
4
x[1]
3
+
-
^2
+
^2
x[2]
^2
x[5]
*
4
x[2]
3
+
-
^2
+
^2
x[3]
^2
x[5]
*
4
x[3]
3
+
-
^2
+
^2
x[4]
^2
x[5]
*
4
x[4]
3
=#
```

![extract_element_functions_ad](https://user-images.githubusercontent.com/24891028/181631462-0e6a4baf-234f-498c-
a406-8f721c33f728.svg)