cockroachdb / cockroachdb/cockroach

release-26.3.2-rc: roachtest: apt_problem failed

Open
#174,691 10 comments 0 reactions 0 assignees View on GitHub
branch-release-26.3.2-rc O-roachtest O-robot T-testeng X-infra-flake
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

roachtest.apt_problem [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGcePrivateBazel/21499576?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGcePrivateBazel/21499576?buildTab=artifacts#/django) on release-26.3.2-rc @ [6149d98a3244ba2a3e4417d9e363da83e213e1b7](https://github.com/cockroachdb/cockroach/commits/6149d98a3244ba2a3e4417d9e363da83e213e1b7):

Failed with:

```
test django failed: (django.go:70).func1: all attempts failed for install dependencies: full command output in run_184453.407343056_n1_sudo-aptget-qq-insta.log: error persisted after 3 attempts: TRANSIENT_ERROR(apt_problem): exit status 100
test artifacts and logs in: /artifacts/django/run_1
```
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21499576-1788544814-10-n1cpu16-0001 | | 10.91.100.194 |

Parameters:
- arch=amd64
- cloud=gce
- coverageBuild=false
- cpu=16
- diskCount=0
- encrypted=false
- metamorphicLeases=default
- metamorphicWriteBuffering=true
- runtimeAssertionsBuild=false
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21499576/django/1788545224974/1788547679512)

See: [Datadog Logs](https://us5.datadoghq.com/logs?query=service%3Aroachtest+%40cluster%3Ateamcity-21499576-1788544814-10-n1cpu16&from_ts=1788545224974&to_ts=1788547679512&live=false&storage=flex_tier)

Same failure on other branches

- #174466 roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
- #174341 basalt-26.3: roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake basalt-red branch-basalt-26.3]
- #173764 roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.4.15-rc]
- #173585 basalt-26.3: roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-basalt-26.3]
- #173563 release-26.2.6-rc: roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.2.6-rc]
- #173550 roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2.23-rc]
- #173460 release-26.3.1-rc: roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.3.1-rc]
- #172603 roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.1.8-rc]
- #172566 release-26.3.0-rc: roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.3.0-rc]
- #172511 roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2.22-rc]

/cc @cockroachlabs/test-eng

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*apt_problem.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-67911

Contributor guide

Open the contributing guide

Research direction

Start with pkg/cmd/roachtest/README.md and the django test failure reported at django.go:70. Inspect the referenced run_184453.407343056_n1_sudo-aptget-qq-insta.log and related artifacts, then compare the linked apt_problem failures on other branches. Done means the django roachtest no longer fails with this apt dependency error.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, go
Domain
infrastructure, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.