google / google/fuzzbench

Fix service account key issues

Open
#1,031 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.2k
Forks
302
PR merge metrics
No merged PRs in 30d

Description

Fuzzbench uses service account keys because when we didn't use them there was a horrible bug logging which is bad for 2 reasons:
1. It breaks experiments because we log everywhere.
2. It is really hard to debug when logging doesn't work.

We did this about a year ago.
This solution is imperfect however. Because service account key creation happens on the dispatcher, if it fails, we have no feedback at all. It fails not that infrequently because there is a limit of 10 service account keys.

I can think of a few solutions/mitigations to this problem.
1. Stop using service account keys. This solution is only viable if the above bug is fixed.
2. Store service account key somewhere and share between experiments. The problem with this approach is Fuzzbench exposes a lot to users and there is nowhere really to hide this, we could create another bucket though. This approach will probably solve the limit issue.
3. Make run_experiment.py pass service account key to the dispatcher. This has two advantages: 1. service account key validation can happen in run_experiment so we can get feedback 2. It allows us to avoid the limit on service account keys.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.