Error when running a local experiment on Fuzzbench
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 302
- PR merge metrics
- No merged PRs in 30d
Description
I have set up a fuzzbench on a Nectar Server and integrated my AFL version (tested separately) to run on several benchmarks locally. `make presubmit` was successful.
When I trigger fuzzing executions manually using say `make run-afl-freetype2-2017`, it works quite well.
Then I wanted to run the experiment for multiple trials and I tried it by placing the `experiment-config.yaml` file. The following command was run thereafter.
`PYTHONPATH=. python3 experiment/run_experiment.py --experiment-config experiment-config.yaml --benchmarks freetype2-2017 bloaty_fuzz_target --experiment-name experiment-1 --fuzzers afl`
It was showing that the experiment was starting. i.e. `INFO:root:Starting experiment.`
Unfortunately, the following error popped up.
> ERROR:root:Executed command: "make -j base-image worker" returned: 2. Extras: {'output': ".0-py2.py3-none-any.whl (31 kB)\nCollecting h11==0.*\n Downloading h11-0.12.0-py3-none-any.whl (54 kB)\nCollecting typing-extensions>=3.7.4.2\n Downloading typing_extensions-3.10.0.0-py3-none-any.whl (26 kB)\nCollecting typing-inspect>=0.4.0\n Downloading typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)\nCollecting kiwisolver>=1.0.1\n Using cached kiwisolver-1.3.1-cp38-cp38-manylinux1_x86_64.whl (1.2 MB)\nCollecting pillow>=6.2.0\n Downloading Pillow-8.3.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (3.0 MB)\nCollecting cycler>=0.10\n Using cached cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)\nCollecting decorator<5,>=4.3\n Downloading decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)\nCollecting qasync\n Downloading qasync-0.17.0-py3-none-any.whl (13 kB)\nCollecting cachecontrol[filecache]\n Using cached CacheControl-0.12.6-py2.py3-none-any.whl (19 kB)\nCollecting commonmark>=0.8.1\n Using cached commonmark-0.9.1-py2.py3-none-any.whl (51 kB)\nCollecting dictdiffer\n Using cached dictdiffer-0.8.1-py2.py3-none-any.whl (16 kB)\nCollecting docutils\n Downloading docutils-0.17.1-py2.py3-none-any.whl (575 kB)\nCollecting pyasn1<0.5.0,>=0.4.6\n Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)\nCollecting PyQt5-Qt5>=5.15\n Downloading PyQt5_Qt5-5.15.2-py3-none-manylinux2014_x86_64.whl (59.9 MB)\nCollecting PyQt5-sip<13,>=12.8\n Downloading PyQt5_sip-12.9.0-cp38-cp38-manylinux1_x86_64.whl (332 kB)\nCollecting PyQtWebEngine-Qt5>=5.15\n Downloading PyQtWebEngine_Qt5-5.15.2-py3-none-manylinux2014_x86_64.whl (67.5 MB)\nCollecting idna<4,>=2.5\n Downloading idna-3.2-py3-none-any.whl (59 kB)\nCollecting urllib3<1.27,>=1.21.1\n Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB)\nCollecting charset-normalizer~=2.0.0\n Downloading charset_normalizer-2.0.3-py3-none-any.whl (35 kB)\nCollecting threadpoolctl>=2.0.0\n Downloading threadpoolctl-2.2.0-py3-none-any.whl (12 kB)\nCollecting mypy-extensions>=0.3.0\n Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)\nCollecting msgpack>=0.5.2\n Using cached msgpack-1.0.2-cp38-cp38-manylinux1_x86_64.whl (302 kB)\nCollecting lockfile>=0.9\n Using cached lockfile-0.12.2-py2.py3-none-any.whl (13 kB)\nCollecting requests-oauthlib>=0.7.0\n Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)\nCollecting oauthlib>=3.0.0\n Downloading oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)\nCollecting pymemcache\n Downloading pymemcache-3.5.0-py2.py3-none-any.whl (56 kB)\nCollecting google-cloud-datastore\n Downloading google_cloud_datastore-1.15.3-py2.py3-none-any.whl (134 kB)\nCollecting google-resumable-media<3.0dev,>=1.3.0\n Downloading google_resumable_media-1.3.1-py2.py3-none-any.whl (75 kB)\nCollecting google-crc32c<2.0dev,>=1.0\n Using cached google_crc32c-1.1.2-cp38-cp38-manylinux2014_x86_64.whl (38 kB)\nCollecting cffi>=1.0.0\n Downloading cffi-1.14.6-cp38-cp38-manylinux1_x86_64.whl (411 kB)\nCollecting pycparser\n Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)\nCollecting jeepney>=0.4.2\n Downloading jeepney-0.7.0-py3-none-any.whl (53 kB)\nCollecting SecretStorage>=3.2\n Using cached SecretStorage-3.3.1-py3-none-any.whl (15 kB)\nCollecting importlib-metadata>=3.6\n Downloading importlib_metadata-4.6.1-py3-none-any.whl (17 kB)\nCollecting zipp>=0.5\n Downloading zipp-3.5.0-py3-none-any.whl (5.7 kB)\nCollecting cryptography>=2.0\n Using cached cryptography-3.4.7-cp36-abi3-manylinux2014_x86_64.whl (3.2 MB)\nCollecting mozinfo\n Using cached mozinfo-1.2.2.tar.gz (5.8 kB)\nCollecting distro>=1.4.0\n Using cached distro-1.5.0-py2.py3-none-any.whl (18 kB)\nCollecting mozfile>=0.12\n Using cached mozfile-2.1.0.tar.gz (5.9 kB)\nCollecting et-xmlfile\n Downloading et_xmlfile-1.1.0-py3-none-any.whl (4.7 kB)\nCollecting patsy>=0.5\n Using cached patsy-0.5.1-py2.py3-none-any.whl (231 kB)\nUsing legacy 'setup.py install' for alembic, since package 'wheel' is not installed.\nUsing legacy 'setup.py install' for grpc-google-iam-v1, since package 'wheel' is not installed.\nUsing legacy 'setup.py install' for importlab, since package 'wheel' is not installed.\nUsing legacy 'setup.py install' for python-louvain, since package 'wheel' is not installed.\nUsing legacy 'setup.py install' for wrapt, since package 'wheel' is not installed.\nUsing legacy 'setup.py install' for google-cloud-profiler, since package 'wheel' is not installed.\nUsing legacy 'setup.py install' for mozinfo, since package 'wheel' is not installed.\nUsing legacy 'setup.py install' for mozfile, since package 'wheel' is not installed.\nUsing legacy 'setup.py install' for serverfiles, since package 'wheel' is not installed.\nInstalling collected packages: six, pyasn1, urllib3, rsa, pyparsing, pyasn1-modules, protobuf, idna, charset-normalizer, certifi, cachetools, requests, pytz, pycparser, packaging, msgpack, googleapis-common-protos, google-auth, numpy, lockfile, httplib2, grpcio, google-api-core, cffi, cachecontrol, zipp, uritemplate, typing-extensions, threadpoolctl, sniffio, scipy, rfc3986, qasync, python-dateutil, PyQt5-sip, PyQt5-Qt5, pillow, oauthlib, mypy-extensions, mozfile, kiwisolver, joblib, jeepney, h11, google-crc32c, google-cloud-core, google-auth-httplib2, docutils, distro, dictdiffer, decorator, cycler, cryptography, commonmark, AnyQt, wrapt, typing-inspect, SecretStorage, scikit-learn, requests-oauthlib, redis, PyYAML, PyQtWebEngine-Qt5, pyqtgraph, PyQt5, pymemcache, proto-plus, patsy, pandas, orange-canvas-core, networkx, mozinfo, matplotlib, MarkupSafe, lazy-object-proxy, importlib-metadata, httpcore, google-resumable-media, google-cloud-datastore, google-api-python-client, et-xmlfile, xlsxwriter, xlrd, typed-ast, toml, statsmodels, sqlalchemy, serverfiles, seaborn, python-louvain, python-editor, PyQtWebEngine, py, psutil, pluggy, orange-widget-base, openTSNE, openpyxl, oauth2client, ninja, mozprocess, mccabe, Mako, libcst, keyrings.alt, keyring, isort, iniconfig, importlab, httpx, grpc-google-iam-v1, google-cloud-storage, google-cloud-profiler, google-cloud-ndb, google-cloud-monitoring, google-cloud-logging, google-auth-oauthlib, click, chardet, bottleneck, baycomp, attrs, astroid, yapf, scikit-posthocs, rq, pytype, pytest, pylint, pyfakefs, psycopg2-binary, Orange3, Jinja2, google-cloud-secret-manager, google-cloud-error-reporting, clusterfuzz, alembic\n Running setup.py install for mozfile: started\n Running setup.py install for mozfile: finished with status 'done'\n Running setup.py install for wrapt: started\n Running setup.py install for wrapt: finished with status 'done'\n Running setup.py install for mozinfo: started\n Running setup.py install for mozinfo: finished with status 'done'\n Running setup.py install for serverfiles: started\n Running setup.py install for serverfiles: finished with status 'done'\n Running setup.py install for python-louvain: started\n Running setup.py install for python-louvain: finished with status 'done'\n Running setup.py install for importlab: started\n Running setup.py install for importlab: finished with status 'done'\n Running setup.py install for grpc-google-iam-v1: started\n Running setup.py install for grpc-google-iam-v1: finished with status 'done'\n Running setup.py install for google-cloud-profiler: started\n Running setup.py install for google-cloud-profiler: finished with status 'done'\n Running setup.py install for alembic: started\n Running setup.py install for alembic: finished with status 'done'\nSuccessfully installed AnyQt-0.0.13 Jinja2-2.11.3 Mako-1.1.4 MarkupSafe-2.0.1 Orange3-3.28.0 PyQt5-5.15.4 PyQt5-Qt5-5.15.2 PyQt5-sip-12.9.0 PyQtWebEngine-5.15.4 PyQtWebEngine-Qt5-5.15.2 PyYAML-5.4 SecretStorage-3.3.1 alembic-1.4.0 astroid-2.6.5 attrs-21.2.0 baycomp-1.0.2 bottleneck-1.3.2 cachecontrol-0.12.6 cachetools-4.2.2 certifi-2021.5.30 cffi-1.14.6 chardet-4.0.0 charset-normalizer-2.0.3 click-8.0.1 clusterfuzz-0.0.1a0 commonmark-0.9.1 cryptography-3.4.7 cycler-0.10.0 decorator-4.4.2 dictdiffer-0.8.1 distro-1.5.0 docutils-0.17.1 et-xmlfile-1.1.0 google-api-core-1.31.0 google-api-python-client-2.5.0 google-auth-1.30.1 google-auth-httplib2-0.1.0 google-auth-oauthlib-0.4.4 google-cloud-core-1.7.1 google-cloud-datastore-1.15.3 google-cloud-error-reporting-1.1.2 google-cloud-logging-1.15.1 google-cloud-monitoring-2.4.1 google-cloud-ndb-1.10.0 google-cloud-profiler-3.0.5 google-cloud-secret-manager-2.4.0 google-cloud-storage-1.41.1 google-crc32c-1.1.2 google-resumable-media-1.3.1 googleapis-common-protos-1.53.0 grpc-google-iam-v1-0.12.3 grpcio-1.39.0 h11-0.12.0 httpcore-0.12.3 httplib2-0.19.1 httpx-0.16.1 idna-3.2 importlab-0.6.1 importlib-metadata-4.6.1 iniconfig-1.1.1 isort-5.9.2 jeepney-0.7.0 joblib-1.0.1 keyring-23.0.1 keyrings.alt-4.1.0 kiwisolver-1.3.1 lazy-object-proxy-1.6.0 libcst-0.3.19 lockfile-0.12.2 matplotlib-3.4.2 mccabe-0.6.1 mozfile-2.1.0 mozinfo-1.2.2 mozprocess-1.3.0 msgpack-1.0.2 mypy-extensions-0.4.3 networkx-2.5.1 ninja-1.10.0.post3 numpy-1.18.1 oauth2client-4.1.3 oauthlib-3.1.1 openTSNE-0.6.0 openpyxl-3.0.7 orange-canvas-core-0.1.21 orange-widget-base-4.13.1 packaging-21.0 pandas-1.2.4 patsy-0.5.1 pillow-8.3.1 pluggy-0.13.1 proto-plus-1.19.0 protobuf-3.17.3 psutil-5.8.0 psycopg2-binary-2.8.4 py-1.10.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.20 pyfakefs-3.7.1 pylint-2.7.4 pymemcache-3.5.0 pyparsing-2.4.7 pyqtgraph-0.12.2 pytest-6.1.2 python-dateutil-2.8.1 python-editor-1.0.4 python-louvain-0.15 pytype-2021.4.15 pytz-2019.3 qasync-0.17.0 redis-3.5.3 requests-2.26.0 requests-oauthlib-1.3.0 rfc3986-1.5.0 rq-1.4.3 rsa-4.7.2 scikit-learn-0.24.2 scikit-posthocs-0.6.2 scipy-1.6.2 seaborn-0.11.1 serverfiles-0.3.1 six-1.16.0 sniffio-1.2.0 sqlalchemy-1.3.19 statsmodels-0.12.2 threadpoolctl-2.2.0 toml-0.10.2 typed-ast-1.4.3 typing-extensions-3.10.0.0 typing-inspect-0.7.1 uritemplate-3.0.1 urllib3-1.26.6 wrapt-1.12.1 xlrd-2.0.1 xlsxwriter-1.4.4 yapf-0.30.0 zipp-3.5.0\nsource .venv/bin/activate && PYTHONPATH=. python3 docker/generate_makefile.py docker/generated.mk\nmake: 'base-image' is up to date.\ndocker build \\\n--tag gcr.io/fuzzbench/worker \\\n--build-arg BUILDKIT_INLINE_CACHE=1 \\\n--cache-from gcr.io/fuzzbench/worker \\\n--file docker/worker/Dockerfile \\\n.\nbuildkit not supported by daemon\ndocker/generated.mk:115: recipe for target 'worker' failed\nmake: *** [worker] Error 1\n"}
> ERROR:root:Error conducting experiment. Extras: {'traceback': 'Traceback (most recent call last):\n File "/work/src/experiment/dispatcher.py", line 194, in main\n dispatcher_main()\n File "/work/src/experiment/dispatcher.py", line 148, in dispatcher_main\n trials = build_images_for_trials(experiment.fuzzers, experiment.benchmarks,\n File "/work/src/experiment/dispatcher.py", line 114, in build_images_for_trials\n builder.build_base_images()\n File "/work/src/experiment/build/builder.py", line 77, in build_base_images\n return buildlib.build_base_images()\n File "/work/src/experiment/build/local_build.py", line 38, in build_base_images\n return make([\'base-image\', \'worker\'])\n File "/work/src/experiment/build/local_build.py", line 33, in make\n return new_process.execute(command, cwd=utils.ROOT_DIR)\n File "/work/src/common/new_process.py", line 124, in execute\n raise subprocess.CalledProcessError(retcode, command)\nsubprocess.CalledProcessError: Command \'[\'make\', \'-j\', \'base-image\', \'worker\']\' returned non-zero exit status 2.\n'}
> Traceback (most recent call last):
> File "/work/src/experiment/dispatcher.py", line 212, in
> sys.exit(main())
> File "/work/src/experiment/dispatcher.py", line 197, in main
> raise error
> File "/work/src/experiment/dispatcher.py", line 194, in main
> dispatcher_main()
> File "/work/src/experiment/dispatcher.py", line 148, in dispatcher_main
> trials = build_images_for_trials(experiment.fuzzers, experiment.benchmarks,
> File "/work/src/experiment/dispatcher.py", line 114, in build_images_for_trials
> builder.build_base_images()
> File "/work/src/experiment/build/builder.py", line 77, in build_base_images
> return buildlib.build_base_images()
> File "/work/src/experiment/build/local_build.py", line 38, in build_base_images
> return make(['base-image', 'worker'])
> File "/work/src/experiment/build/local_build.py", line 33, in make
> return new_process.execute(command, cwd=utils.ROOT_DIR)
> File "/work/src/common/new_process.py", line 124, in execute
> raise subprocess.CalledProcessError(retcode, command)
> subprocess.CalledProcessError: Command '['make', '-j', 'base-image', 'worker']' returned non-zero exit status 2.
Any pointers to resolve this error?
Thanks.
Contributor guide
Assessment
This issue has not been assessed yet.