cockroachdb / cockroachdb/cockroach

release-26.3: roachtest: apt_problem failed

Open
#171,789 77 comments 0 reactions 0 assignees View on GitHub
branch-release-26.3 O-roachtest O-robot T-testeng X-infra-flake
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Note:** This build has runtime assertions enabled. If the same failure was hit in a run without assertions enabled, there should be a similar failure without this message. If there isn't one, then this failure is likely due to an assertion violation or (assertion) timeout.

roachtest.apt_problem [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/21392022?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/21392022?buildTab=artifacts#/django) on release-26.3 @ [8a9a8e3305daa0890b7b083c13cb572f911bd6cc](https://github.com/cockroachdb/cockroach/commits/8a9a8e3305daa0890b7b083c13cb572f911bd6cc):

Failed with:

```
test django failed: (django.go:70).func1: all attempts failed for install dependencies: full command output in run_164105.246996403_n1_sudo-aptget-qq-insta.log: error persisted after 3 attempts: TRANSIENT_ERROR(apt_problem): exit status 100
test artifacts and logs in: /artifacts/django/run_1
```
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21392022-1781763104-119-n1cpu16-0001 | 34.23.176.205 | 10.142.1.25 |

Parameters:
- arch=amd64
- cloud=gce
- coverageBuild=false
- cpu=16
- diskCount=0
- encrypted=false
- fs=ext4
- localSSD=true
- metamorphicLeases=default
- metamorphicWriteBuffering=true
- runtimeAssertionsBuild=true
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21392022/django/1781798385446/1781801076200)

See: [Datadog Logs](https://us5.datadoghq.com/logs?query=service%3Aroachtest+%40cluster%3Ateamcity-21392022-1781763104-119-n1cpu16&from_ts=1781798385446&to_ts=1781801076200&live=false&storage=flex_tier)

/cc @cockroachdb/test-eng

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*apt_problem.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-64925

Contributor guide

Open the contributing guide

Research direction

Start with the roachtest README, then inspect the artifacts under /artifacts/django/run_1 and the named run_164105.246996403_n1_sudo-aptget-qq-insta.log. Use the linked TeamCity output and the existing issue discussion to determine why dependency installation returns apt_problem; done means the django roachtest no longer fails for this cause.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, go, linux
Domain
cloud, devops, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.