GoogleCloudPlatform / GoogleCloudPlatform/PerfKitBenchmarker

Influxdb publishing broken

Open
#3,713 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2k
Forks
562
Avg merge
4h 55m
Merged PRs (30d)
69

Description

Using commit b51162e18763e5ad86fbc02c4ba5ebe57085e7ad and influxdb publisher with influxdb v1.8.10.

Influxdb publisher publishes some metrics but not all. Logs show an error:
```
2022-08-17 16:05:37,140 78f51f1d MainThread ERROR 400 Request could not be completed due to: Bad Request perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=800b2ca0-c787-4b3b-ac0b-0ca6efe00aa2,metric=Coremark\ Score,unit=\"\",product_name=PerfKitBenchmarker,summary=CoreMark\ 1.0\ :\ 110338.739932\ /\ GCC12.1.0\ -O2\ -g\ -O2\ -DMULTITHREAD=4\ -DUSE_PTHREAD\ -DPERFORMANCE_RUN=1\ -DPERFORMANCE_RUN=1\ \ -lrt\ -lpthread\ /\ Heap\ /\ 4:PThreads,size=666,total_ticks=36252,total_time_sec=36.252,iterations=4000000,iterations_per_cpu=1000000,parallelism_method=PTHREAD,run_number=0,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=110338.739932 1660766736251683072
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=174040d2-7e76-4807-870e-16f670168b38,metric=lscpu,unit=\"\",product_name=PerfKitBenchmarker,node_name=pkb-78f51f1d-0,Architecture=x86_64,CPU op-mode(s)=32-bit\,\ 64-bit,Address sizes=48\ bits\ physical\,\ 48\ bits\ virtual,Byte Order=Little\ Endian,CPU(s)=4,On-line CPU(s) list=0-3,Vendor ID=AuthenticAMD,BIOS Vendor ID=Red\ Hat,Model name=AMD\ EPYC\ 7763\ 64-Core\ Processor,BIOS Model name=RHEL-8.6.0\ PC\ (Q35\ +\ ICH9\,\ 2009)\ \ CPU\ @\ 2.0GHz,BIOS CPU family=1,CPU family=25,Model=1,Thread(s) per core=1,Core(s) per socket=1,Socket(s)=4,Stepping=1,BogoMIPS=4900.00,Flags=fpu\ vme\ de\ pse\ tsc\ msr\ pae\ mce\ cx8\ apic\ sep\ mtrr\ pge\ mca\ cmov\ pat\ pse36\ clflush\ mmx\ fxsr\ sse\ sse2\ syscall\ nx\ mmxext\ fxsr_opt\ pdpe1gb\ rdtscp\ lm\ rep_good\ nopl\ cpuid\ extd_apicid\ tsc_known_freq\ pni\ pclmulqdq\ ssse3\ fma\ cx16\ pcid\ sse4_1\ sse4_2\ x2apic\ movbe\ popcnt\ tsc_deadline_timer\ aes\ xsave\ avx\ f16c\ rdrand\ hypervisor\ lahf_lm\ cmp_legacy\ svm\ cr8_legacy\ abm\ sse4a\ misalignsse\ 3dnowprefetch\ osvw\ perfctr_core\ invpcid_single\ ssbd\ ibrs\ ibpb\ stibp\ vmmcall\ fsgsbase\ tsc_adjust\ bmi1\ avx2\ smep\ bmi2\ invpcid\ rdseed\ adx\ smap\ clflushopt\ clwb\ sha_ni\ xsaveopt\ xsavec\ xgetbv1\ xsaves\ clzero\ xsaveerptr\ wbnoinvd\ arat\ npt\ nrip_save\ umip\ pku\ ospke\ vaes\ vpclmulqdq\ rdpid\ arch_capabilities,Virtualization=AMD-V,Hypervisor vendor=KVM,Virtualization type=full,L1d cache=256\ KiB\ (4\ instances),L1i cache=256\ KiB\ (4\ instances),L2 cache=2\ MiB\ (4\ instances),L3 cache=64\ MiB\ (4\ instances),NUMA node(s)=1,NUMA node0 CPU(s)=0-3,Vulnerability Itlb multihit=Not\ affected,Vulnerability L1tf=Not\ affected,Vulnerability Mds=Not\ affected,Vulnerability Meltdown=Not\ affected,Vulnerability Mmio stale data=Not\ affected,Vulnerability Retbleed=Not\ affected,Vulnerability Spec store bypass=Mitigation;\ Speculative\ Store\ Bypass\ disabled\ via\ prctl,Vulnerability Spectre v1=Mitigation;\ usercopy/swapgs\ barriers\ and\ __user\ pointer\ sanitization,Vulnerability Spectre v2=Mitigation;\ Retpolines\,\ IBPB\ conditional\,\ IBRS_FW\,\ STIBP\ disabled\,\ RSB\ filling\,\ PBRSB-eIBRS\ Not\ affected,Vulnerability Srbds=Not\ affected,Vulnerability Tsx async abort=Not\ affected,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=0.0 1660766736252098048
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=5aba2937-9e63-400b-b5c8-3ad7aca2cf6d,metric=proccpu,unit=\"\",product_name=PerfKitBenchmarker,node_name=pkb-78f51f1d-0,vendor_id=AuthenticAMD,cpu family=25,model=1,model name=AMD\ EPYC\ 7763\ 64-Core\ Processor,stepping=1,microcode=0xa001144,cpu MHz=2450.000,cache size=512\ KB,siblings=1,cpu cores=1,fpu=yes,fpu_exception=yes,cpuid level=16,wp=yes,flags=3dnowprefetch\ abm\ adx\ aes\ apic\ arat\ arch_capabilities\ avx\ avx2\ bmi1\ bmi2\ clflush\ clflushopt\ clwb\ clzero\ cmov\ cmp_legacy\ cpuid\ cr8_legacy\ cx16\ cx8\ de\ extd_apicid\ f16c\ fma\ fpu\ fsgsbase\ fxsr\ fxsr_opt\ hypervisor\ ibpb\ ibrs\ invpcid\ invpcid_single\ lahf_lm\ lm\ mca\ mce\ misalignsse\ mmx\ mmxext\ movbe\ msr\ mtrr\ nopl\ npt\ nrip_save\ nx\ ospke\ osvw\ pae\ pat\ pcid\ pclmulqdq\ pdpe1gb\ perfctr_core\ pge\ pku\ pni\ popcnt\ pse\ pse36\ rdpid\ rdrand\ rdseed\ rdtscp\ rep_good\ sep\ sha_ni\ smap\ smep\ ssbd\ sse\ sse2\ sse4_1\ sse4_2\ sse4a\ ssse3\ stibp\ svm\ syscall\ tsc\ tsc_adjust\ tsc_deadline_timer\ tsc_known_freq\ umip\ vaes\ vme\ vmmcall\ vpclmulqdq\ wbnoinvd\ x2apic\ xgetbv1\ xsave\ xsavec\ xsaveerptr\ xsaveopt\ xsaves,bugs=null_seg\ spec_store_bypass\ spectre_v1\ spectre_v2\ sysret_ss_attrs,bogomips=4900.00,TLB size=1024\ 4K\ pages,clflush size=64,cache_alignment=64,address sizes=48\ bits\ physical\,\ 48\ bits\ virtual,power management=\"\",proccpu=TLB\ size\,address\ sizes\,bogomips\,bugs\,cache\ size\,cache_alignment\,clflush\ size\,cpu\ MHz\,cpu\ cores\,cpu\ family\,cpuid\ level\,flags\,fpu\,fpu_exception\,microcode\,model\,model\ name\,power\ management\,siblings\,stepping\,vendor_id\,wp,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=0.0 1660766736335831040
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=ae3bd582-31ec-4e01-ba77-ae3e00a1985c,metric=proccpu_mapping,unit=\"\",product_name=PerfKitBenchmarker,node_name=pkb-78f51f1d-0,proc_0=apicid=0;core\ id=0;initial\ apicid=0;physical\ id=0,proc_1=apicid=1;core\ id=0;initial\ apicid=1;physical\ id=1,proc_2=apicid=2;core\ id=0;initial\ apicid=2;physical\ id=2,proc_3=apicid=3;core\ id=0;initial\ apicid=3;physical\ id=3,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=0.0 1660766736336011008
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=8be96d94-4bbf-42c9-b6e5-7ad8e6d2dcb8,metric=cpu_vuln,unit=\"\",product_name=PerfKitBenchmarker,vm_name=pkb-78f51f1d-0,mitigations=spec_store_bypass\,spectre_v1\,spectre_v2,mitigation_spec_store_bypass=Speculative\ Store\ Bypass\ disabled\ via\ prctl,mitigation_spectre_v1=usercopy/swapgs\ barriers\ and\ __user\ pointer\ sanitization,mitigation_spectre_v2=Retpolines\,\ IBPB:\ conditional\,\ IBRS_FW\,\ STIBP:\ disabled\,\ RSB\ filling\,\ PBRSB-eIBRS:\ Not\ affected,notaffecteds=itlb_multihit\,l1tf\,mds\,meltdown\,mmio_stale_data\,retbleed\,srbds\,tsx_async_abort,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=0.0 1660766736419550976
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=25feb639-d1ac-420e-8d9e-d7fe75ead6c6,metric=gcc_version,unit=\"\",product_name=PerfKitBenchmarker,name=pkb-78f51f1d-0,versiondump=12.1.0,versioninfo=gcc\ (Debian\ 12.1.0-8)\ 12.1.0,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=0.0 1660766736606948096
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=ddd6412a-9ffb-443f-9aa7-403d0b5a784d,metric=glibc_version,unit=\"\",product_name=PerfKitBenchmarker,name=pkb-78f51f1d-0,versioninfo=ldd\ (Debian\ GLIBC\ 2.34-3)\ 2.34,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=0.0 1660766736751962112
perfkitbenchmarker,test=coremark,official=False,owner=brett,run_uri=78f51f1d-1d773331-e686-4dd4-9e30-66167bfcd35f,sample_uri=f83cb277-70ca-45cb-ae63-0f5ddf9aae3d,metric=End\ to\ End\ Runtime,unit=seconds,product_name=PerfKitBenchmarker,perfkitbenchmarker_version=v1.12.0-3363-gb51162e1,image=None,zone=Static\ -\ root@debtest,cloud=Static,os_type=ubuntu1804,tcp_congestion_control=cubic,numa_node_count=1,num_cpus=4,vm_count=1,threads_per_core=1,os_info=Debian\ GNU/Linux\ bookworm/sid,kernel_release=5.18.0-4-cloud-amd64,cpu_arch=x86_64,/dev/vda=88046829568,mtu=1500 value=83.10814499855042 1660766737001362944
2022-08-17 16:05:37,143 78f51f1d MainThread ERROR Error connecting to the database:
```

Some metrics make it into the DB:
```
> show databases
name: databases
name
----
_internal
perfkit
> use perfkit
Using database perfkit
> select value, metric from perfkitbenchmarker
name: perfkitbenchmarker
time value metric
---- ----- ------
1660766736419550976 0 cpu_vuln
1660766736606948096 0 gcc_version
1660766736751962112 0 glibc_version
1660766737001362944 83.10814499855042 End to End Runtime
>
```

Missing metrics:
* Coremark Score
* lscpu
* proccpu
* proccpu_mapping

Run command line:
```
./pkb.py --benchmarks=coremark --benchmark_config_file=./local.yml --temp_dir=/tmp/pkb --influx_uri grafana1-nj1.xxxxx.net:8086 --influx_db_name perfkit
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.