ARM: big performance impact with enabled neon but -mfloat-abi=softfp

I've tried to make little benchmark using QEMU under Ubuntu 18.04. Because I want measure aarch64 (which I do not have) + it's easier measure small differences (every run is deterministic).
Runtime CPU detection do not work with QEMU semihosting, so I enable it manually:

@@ -142,11 +142,11 @@ opus_uint32 opus_cpu_capabilities(void)
 # endif
     }
 
     fclose(cpuinfo);
   }
-  return flags;
+  return flags | OPUS_CPU_ARM_EDSP_FLAG | OPUS_CPU_ARM_MEDIA_FLAG | OPUS_CPU_ARM_NEON_FLAG;
 }
 #else
 /* The feature registers which can tell us what the processor supports are
  * accessible in priveleged modes only, so we can't have a general user-space
  * detection method like on x86.*/

Both variants with neon optimized functions and without is measured.

First I've notice that when I build with arm-linux-gnueabi-gcc:

CC=arm-linux-gnueabi-gcc CFLAGS="-O2 -mcpu=cortex-a15 -mfpu=neon -mfloat-abi=softfp" ../configure --host=arm-linux-gnueabi --disable-shared

Runtime float point functions heavily used:

Total executed instructions: 9334798638
__aeabi_dadd 1095866862 11.740%  <--- libgcc float point support
celt_encode_with_ec 745843070 7.990%
opus_fft_impl 712053747 7.628%
tonality_analysis.isra.0 629640675 6.745%
celt_pitch_xcorr_float_neon 607909464 6.512%
op_pvq_search_c 442786713 4.743%
__aeabi_dmul 431374552 4.621%
compute_gru 342358288 3.668%
__subsf3 327752096 3.511%

This functions come not by itself but form math functions like log10, cos, pow, sqrt. Note that double variants used (for example in silk_process_gains_FLP, silk_noise_shape_analysis_FLP.c).
Then I try --enable-float-approx and it somehow helps:

Total executed instructions: 8746436647
__aeabi_dadd 784876796 8.974%
celt_encode_with_ec 749657806 8.571%
opus_fft_impl 712053747 8.141%
tonality_analysis.isra.0 629270601 7.195%
celt_pitch_xcorr_float_neon 607909464 6.950%
op_pvq_search_c 442807791 5.063%
compute_gru 340887999 3.897%
__subsf3 327752096 3.747%
__aeabi_dmul 323995896 3.704%

But still math functions (even double) heavily used.
Then I tried to replace all double functions to float equivalents log10f, cosf etc. Except initialization functions. And this helps even more:

Total executed instructions: 8462849809
celt_encode_with_ec 749638691 8.858%
opus_fft_impl 712053747 8.414%
tonality_analysis.isra.0 629040555 7.433%
celt_pitch_xcorr_float_neon 607909464 7.183%
__adddf3 601537247 7.108%  <---- still here
op_pvq_search_c 442807791 5.232%
compute_gru 340887999 4.028%
__addsf3 294458581 3.479%
__aeabi_dmul 267024861 3.155%

But still big difference with -mfloat-abi=hard:

Total executed instructions: 7311795843
celt_encode_with_ec 797797095 10.911%
opus_fft_impl 712338094 9.742%
tonality_analysis.isra.0 645539978 8.829%
celt_pitch_xcorr_float_neon 609168730 8.331%
op_pvq_search_c 441154759 6.033%
compute_gru 382336287 5.229%
pitch_downsample 248975975 3.405%
haar1 246039992 3.365%
clt_mdct_forward_c 233810038 3.198%
main 165168530 2.259%
silk_biquad_float 163545392 2.237%
find_best_pitch 157104294 2.149%
dual_inner_prod_neon 156394521 2.139%
celt_inner_prod_neon 156178844 2.136%
__lrintf 150633156 2.060%

aarch64 and aarch64 + flto gives another speedups:

Total executed instructions: 6078286128

So:

• NEON optimizations gives not so much speedup. Non of optimizations gives too much, but all of them gives 42% (arm-noneon vs aarch64-flto with double functions fix).
• -mfloat-abi=softfp have big performance penalty, because of float math functions used while encoding in porcess.
• Of course QEMU approach is in question, it measures instructions, not clockticks. But neon vs non-neon should be measured perfectly by instructions (I've measure nearly 4x on minimp3). And libgcc __aeabi* usage instead of neon\vfp should definetly harm hardware performance too.

Here all profiling info:
profile.zip

Start by reproducing the arm-linux-gnueabi-gcc configurations under QEMU, comparing softfp, hard-float, NEON, and aarch64 results from the supplied profile.zip. Read opus_cpu_capabilities and the floating-point-heavy paths mentioned in silk_process_gains_FLP.c and silk_noise_shape_analysis_FLP.c; done means identifying and correcting or clearly documenting the source of the softfp performance gap.