Enzyme introduces additional runtime overhead on function calls and for loops
- Dominant language
- LLVM
- Stars
- 1.7k
- Forks
- 188
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 22
Description
Hi,
I've been using Enzyme to differentiate some of my existing c++ code. I found that when there are for loops or function calls existing inside the function, the reverse pass would be very slow. Then I tried to move the for loop outside the function being differentiated, the reverse pass would run 2x faster. I made a simple example below to show this problem, In `d_sum2`, the for loop is inside `__enzyme_autodiff`, while in `d_sum1`, the for loop is outside `__enzyme_autodiff`. `d_sum1` ran 2x faster than `d_sum2'. I'm not sure if I'm using Enzyme incorrectly or Enzyme does introduce some additional overhead. Thank you.
``` cpp
#include
#include "timer.h"
#include
#include
#include
using namespace std;
void __enzyme_autodiff(...);
int enzyme_dup, enzyme_const, enzyme_out, enzyme_dupnoneed;
__attribute__((optnone)) double sum1(std::vector &v)
{
double ret = 0.;
for (int i = 0; i < v.size(); i++)
{
ret += v[i];
}
ret /= v.size();
return ret;
}
__attribute__((optnone)) double sum2(std::vector &v)
{
double res = 0.;
for (int i = 0; i < v.size(); i++)
{
res += sum1(v);
}
return res;
}
__attribute__((optnone)) void d_sum2(std::vector &v, std::vector &d_v)
{
__enzyme_autodiff((void *)(sum2),
enzyme_dup, &v, &d_v);
}
__attribute__((optnone)) void d_sum1(std::vector &v, std::vector &d_v)
{
double res = 1.;
for (int i = 0; i < v.size(); i++)
{
__enzyme_autodiff((void *)(sum1),
enzyme_dup, &v, &d_v);
}
}
int main()
{
std::vector v(1000);
std::vector dv(1000);
int size = 100;
for (auto &e : v)
e = 1.;
for (auto &e : dv)
e = 0.;
{
Timer timer("s");
for (int i = 0; i < 100; i++)
d_sum1(v, dv);
}
printf("%f\n", dv[0]);
{
Timer timer("s");
for (int i = 0; i < 100; i++)
d_sum2(v, dv);
}
printf("%f\n", dv[0]);
}
```
Contributor guide
Assessment
This issue has not been assessed yet.