GoogleCloudPlatform / GoogleCloudPlatform/PerfKitBenchmarker
Influxdb publishing broken
- Dominant language
- Python
- Stars
- 2k
- Forks
- 562
- Avg merge
- 4h 55m
- Merged PRs (30d)
- 69
Description
Using commit b51162e18763e5ad86fbc02c4ba5ebe57085e7ad and influxdb publisher with influxdb v1.8.10.
Influxdb publisher publishes some metrics but not all. Logs show an error:
```
2022-08-17 16:05:37,140 78f51f1d MainThread ERROR 400 Request could not be completed due to: Bad Request perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=800b2ca0-c787-4b3b-ac0b-0ca6efe00aa2,metric=Coremark\ Score,unit=\"\",product_name=PerfKitBenchmarker,summary=CoreMark\ 1.0\ :\ 110338.739932\ /\ GCC12.1.0\ -O2\ -g\ -O2\ -DMULTITHREAD=4\ -DUSE_PTHREAD\ -DPERFORMANCE_RUN=1\ -DPERFORMANCE_RUN=1\ \ -lrt\ -lpthread\ /\ Heap\ /\ 4:PThreads,size=666,total_ticks=36252,total_time_sec=36.252,iterations=4000000,iterations_per_cpu=1000000,parallelism_method=PTHREAD,run_number=0,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=110338.739932 1660766736251683072
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=174040d2-7e76-4807-870e-16f670168b38,metric=lscpu,unit=\"\",product_name=PerfKitBenchmarker,node_name=pkb-78f51f1d-0,Architecture=x86_64,CPU op-mode(s)=32-bit\,\ 64-bit,Address sizes=48\ bits\ physical\,\ 48\ bits\ virtual,Byte Order=Little\ Endian,CPU(s)=4,On-line CPU(s) list=0-3,Vendor ID=AuthenticAMD,BIOS Vendor ID=Red\ Hat,Model name=AMD\ EPYC\ 7763\ 64-Core\ Processor,BIOS Model name=RHEL-8.6.0\ PC\ (Q35\ +\ ICH9\,\ 2009)\ \ CPU\ @\ 2.0GHz,BIOS CPU family=1,CPU family=25,Model=1,Thread(s) per core=1,Core(s) per socket=1,Socket(s)=4,Stepping=1,BogoMIPS=4900.00,Flags=fpu\ vme\ de\ pse\ tsc\ msr\ pae\ mce\ cx8\ apic\ sep\ mtrr\ pge\ mca\ cmov\ pat\ pse36\ clflush\ mmx\ fxsr\ sse\ sse2\ syscall\ nx\ mmxext\ fxsr_opt\ pdpe1gb\ rdtscp\ lm\ rep_good\ nopl\ cpuid\ extd_apicid\ tsc_known_freq\ pni\ pclmulqdq\ ssse3\ fma\ cx16\ pcid\ sse4_1\ sse4_2\ x2apic\ movbe\ popcnt\ tsc_deadline_timer\ aes\ xsave\ avx\ f16c\ rdrand\ hypervisor\ lahf_lm\ cmp_legacy\ svm\ cr8_legacy\ abm\ sse4a\ misalignsse\ 3dnowprefetch\ osvw\ perfctr_core\ invpcid_single\ ssbd\ ibrs\ ibpb\ stibp\ vmmcall\ fsgsbase\ tsc_adjust\ bmi1\ avx2\ smep\ bmi2\ invpcid\ rdseed\ adx\ smap\ clflushopt\ clwb\ sha_ni\ xsaveopt\ xsavec\ xgetbv1\ xsaves\ clzero\ xsaveerptr\ wbnoinvd\ arat\ npt\ nrip_save\ umip\ pku\ ospke\ vaes\ vpclmulqdq\ rdpid\ arch_capabilities,Virtualization=AMD-V,Hypervisor vendor=KVM,Virtualization type=full,L1d cache=256\ KiB\ (4\ instances),L1i cache=256\ KiB\ (4\ instances),L2 cache=2\ MiB\ (4\ instances),L3 cache=64\ MiB\ (4\ instances),NUMA node(s)=1,NUMA node0 CPU(s)=0-3,Vulnerability Itlb multihit=Not\ affected,Vulnerability L1tf=Not\ affected,Vulnerability Mds=Not\ affected,Vulnerability Meltdown=Not\ affected,Vulnerability Mmio stale data=Not\ affected,Vulnerability Retbleed=Not\ affected,Vulnerability Spec store bypass=Mitigation;\ Speculative\ Store\ Bypass\ disabled\ via\ prctl,Vulnerability Spectre v1=Mitigation;\ usercopy/swapgs\ barriers\ and\ __user\ pointer\ sanitization,Vulnerability Spectre v2=Mitigation;\ Retpolines\,\ IBPB\ conditional\,\ IBRS_FW\,\ STIBP\ disabled\,\ RSB\ filling\,\ PBRSB-eIBRS\ Not\ affected,Vulnerability Srbds=Not\ affected,Vulnerability Tsx async abort=Not\ affected,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=0.0 1660766736252098048
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=5aba2937-9e63-400b-b5c8-3ad7aca2cf6d,metric=proccpu,unit=\"\",product_name=PerfKitBenchmarker,node_name=pkb-78f51f1d-0,vendor_id=AuthenticAMD,cpu family=25,model=1,model name=AMD\ EPYC\ 7763\ 64-Core\ Processor,stepping=1,microcode=0xa001144,cpu MHz=2450.000,cache size=512\ KB,siblings=1,cpu cores=1,fpu=yes,fpu_exception=yes,cpuid level=16,wp=yes,flags=3dnowprefetch\ abm\ adx\ aes\ apic\ arat\ arch_capabilities\ avx\ avx2\ bmi1\ bmi2\ clflush\ clflushopt\ clwb\ clzero\ cmov\ cmp_legacy\ cpuid\ cr8_legacy\ cx16\ cx8\ de\ extd_apicid\ f16c\ fma\ fpu\ fsgsbase\ fxsr\ fxsr_opt\ hypervisor\ ibpb\ ibrs\ invpcid\ invpcid_single\ lahf_lm\ lm\ mca\ mce\ misalignsse\ mmx\ mmxext\ movbe\ msr\ mtrr\ nopl\ npt\ nrip_save\ nx\ ospke\ osvw\ pae\ pat\ pcid\ pclmulqdq\ pdpe1gb\ perfctr_core\ pge\ pku\ pni\ popcnt\ pse\ pse36\ rdpid\ rdrand\ rdseed\ rdtscp\ rep_good\ sep\ sha_ni\ smap\ smep\ ssbd\ sse\ sse2\ sse4_1\ sse4_2\ sse4a\ ssse3\ stibp\ svm\ syscall\ tsc\ tsc_adjust\ tsc_deadline_timer\ tsc_known_freq\ umip\ vaes\ vme\ vmmcall\ vpclmulqdq\ wbnoinvd\ x2apic\ xgetbv1\ xsave\ xsavec\ xsaveerptr\ xsaveopt\ xsaves,bugs=null_seg\ spec_store_bypass\ spectre_v1\ spectre_v2\ sysret_ss_attrs,bogomips=4900.00,TLB size=1024\ 4K\ pages,clflush size=64,cache_alignment=64,address sizes=48\ bits\ physical\,\ 48\ bits\ virtual,power management=\"\",proccpu=TLB\ size\,address\ sizes\,bogomips\,bugs\,cache\ size\,cache_alignment\,clflush\ size\,cpu\ MHz\,cpu\ cores\,cpu\ family\,cpuid\ level\,flags\,fpu\,fpu_exception\,microcode\,model\,model\ name\,power\ management\,siblings\,stepping\,vendor_id\,wp,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=0.0 1660766736335831040
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=ae3bd582-31ec-4e01-ba77-ae3e00a1985c,metric=proccpu_mapping,unit=\"\",product_name=PerfKitBenchmarker,node_name=pkb-78f51f1d-0,proc_0=apicid=0;core\ id=0;initial\ apicid=0;physical\ id=0,proc_1=apicid=1;core\ id=0;initial\ apicid=1;physical\ id=1,proc_2=apicid=2;core\ id=0;initial\ apicid=2;physical\ id=2,proc_3=apicid=3;core\ id=0;initial\ apicid=3;physical\ id=3,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=0.0 1660766736336011008
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=8be96d94-4bbf-42c9-b6e5-7ad8e6d2dcb8,metric=cpu_vuln,unit=\"\",product_name=PerfKitBenchmarker,vm_name=pkb-78f51f1d-0,mitigations=spec_store_bypass\,spectre_v1\,spectre_v2,mitigation_spec_store_bypass=Speculative\ Store\ Bypass\ disabled\ via\ prctl,mitigation_spectre_v1=usercopy/swapgs\ barriers\ and\ __user\ pointer\ sanitization,mitigation_spectre_v2=Retpolines\,\ IBPB:\ conditional\,\ IBRS_FW\,\ STIBP:\ disabled\,\ RSB\ filling\,\ PBRSB-eIBRS:\ Not\ affected,notaffecteds=itlb_multihit\,l1tf\,mds\,meltdown\,mmio_stale_data\,retbleed\,srbds\,tsx_async_abort,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=0.0 1660766736419550976
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=25feb639-d1ac-420e-8d9e-d7fe75ead6c6,metric=gcc_version,unit=\"\",product_name=PerfKitBenchmarker,name=pkb-78f51f1d-0,versiondump=12.1.0,versioninfo=gcc\ (Debian\ 12.1.0-8)\ 12.1.0,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=0.0 1660766736606948096
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=ddd6412a-9ffb-443f-9aa7-403d0b5a784d,metric=glibc_version,unit=\"\",product_name=PerfKitBenchmarker,name=pkb-78f51f1d-0,versioninfo=ldd\ (Debian\ GLIBC\ 2.34-3)\ 2.34,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=0.0 1660766736751962112
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=f83cb277-70ca-45cb-ae63-0f5ddf9aae3d,metric=End\ to\ End\ Runtime,unit=seconds,product_name=PerfKitBenchmarker,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=83.10814499855042 1660766737001362944
2022-08-17 16:05:37,143 78f51f1d MainThread ERROR Error connecting to the database:
```
Some metrics make it into the DB:
```
> show databases
name: databases
name
----
_internal
perfkit
> use perfkit
Using database perfkit
> select value, metric from perfkitbenchmarker
name: perfkitbenchmarker
time value metric
---- ----- ------
1660766736419550976 0 cpu_vuln
1660766736606948096 0 gcc_version
1660766736751962112 0 glibc_version
1660766737001362944 83.10814499855042 End to End Runtime
>
```
Missing metrics:
* Coremark Score
* lscpu
* proccpu
* proccpu_mapping
Run command line:
```
./pkb.py --benchmarks=coremark --benchmark_config_file=./local.yml --temp_dir=/tmp/pkb --influx_uri grafana1-nj1.xxxxx.net:8086 --influx_db_name perfkit
```
Contributor guide
Assessment
This issue has not been assessed yet.