Can't build on Rpi 3+ with AArch64

I am not sure if this is an issue with arch, my setup or ffmpegdirect. I am trying to build but I get an -fpic error. I noticed someone else had the same issue and they compiled with -flto. In my PKGBUILD, I added the option of lto but it gave me a different error. I tried to add -fpic to my flags through /etc/makepkg.conf but it didn't work

without lto
/home/alarm/packages/ffmpeg/src/inputstream.ffmpegdirect-19.0.1-Nexus/build/build/depends/lib/libhogweed.a(bignum.o): in function nettle_mpz_sizeinbase_256_s': /home/alarm/packages/ffmpeg/src/inputstream.ffmpegdirect-19.0.1-Nexus/build/build/nettle/src/nettle/bignum.c:61:(.text+0xd8): relocation truncated to fit: R_AARCH64_LD64_GOTPAGE_LO15 against symbol __gmpz_sizeinbase' defined in .text section in /home/alarm/packages/ffmpeg/src/inputstream.ffmpegdirect-19.0.1-Nexus/build/build/depends/lib/libgmp.a(sizeinbase.o)
/usr/bin/ld: /home/alarm/packages/ffmpeg/src/inputstream.ffmpegdirect-19.0.1-Nexus/build/build/nettle/src/nettle/bignum.c:61: warning: too many GOT entries for -fpic, please recompile with -fPIC
collect2: error: ld returned 1 exit status
make[5]: *** [CMakeFiles/inputstream.ffmpegdirect.dir/build.make:330: inputstream.ffmpegdirect.so.20.1.0] Error 1
make[4]: *** [CMakeFiles/Makefile2:110: CMakeFiles/inputstream.ffmpegdirect.dir/all] Error 2
make[3]: *** [Makefile:156: all] Error 2
make[2]: *** [CMakeFiles/inputstream.ffmpegdirect.dir/build.make:88: inputstream.ffmpegdirect-prefix/src/inputstream.ffmpegdirect-stamp/inputstream.ffmpegdirect-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:163: CMakeFiles/inputstream.ffmpegdirect.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
==> ERROR: A failure occurred in build().

With lto:
make[5]: *** [CMakeFiles/inputstream.ffmpegdirect.dir/build.make:330: inputstream.ffmpegdirect.so.20.1.0] Error 1
make[5]: *** Deleting file 'inputstream.ffmpegdirect.so.20.1.0'
make[4]: *** [CMakeFiles/Makefile2:110: CMakeFiles/inputstream.ffmpegdirect.dir/all] Error 2
make[3]: *** [Makefile:156: all] Error 2
make[2]: *** [CMakeFiles/inputstream.ffmpegdirect.dir/build.make:88: inputstream.ffmpegdirect-prefix/src/inputstream.ffmpegdirect-stamp/inputstream.ffmpegdirect-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:163: CMakeFiles/inputstream.ffmpegdirect.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
==> ERROR: A failure occurred in build().

Can someone point me in the right direction?

Start by reproducing the build on an RPi 3+ with AArch64, then inspect the PKGBUILD flags and the CMake build output around inputstream.ffmpegdirect.so. Compare the no-LTO and LTO failures and verify whether the dependency libraries under build/build/depends need different PIC settings. Done means the package builds successfully on the reported platform.