oxidecomputer / oxidecomputer/omicron

xde driver not reloaded after update

Open
#4,423 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

development
Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

tl;dr: it looks like a recent change updated xde, but the dev tooling / process does not actually reload the driver and that seems like a bug.

I went to deploy Omicron on maxwell (a lab PC). It had previously been set up on relatively old bits (probably like 6 weeks old), then rebooted, then was sitting for a few weeks.

Following the usual instructions, from the old git commit I ran omicron-package uninstall and destroy_virtual_hardware.sh. Then I moved to basically 63ca0f9d9603ee486241be1d541cdcc5101e9dc2 (the tip of "main"), ran install_prerequisites.sh, and proceeded with a standard setup. The omicron-package install step failed with:

Nov 02 23:20:08.738 WARN Deleting Omicron bootstrap VNIC, vnic_name: bootstrap0
Nov 02 23:20:08.757 WARN Deleting Omicron bootstrap etherstub, stub_name: bootstrap_stub0
Error: Failure interacting with the OPTE ioctl(2) interface: command ListPorts failed: BadApiVersion { user: 25, kernel: 23 }

Caused by:
    command ListPorts failed: BadApiVersion { user: 25, kernel: 23 }

I got some help in chat. I reran install_prerequisites (which I had already done). Here's the full output:

dap@maxwell ~/omicron $ . env.sh 
++++ dirname env.sh
+++ cd .
+++ echo /home/dap/omicron
++ OMICRON_WS=/home/dap/omicron
++ export PATH=/home/dap/omicron/out/cockroachdb/bin:/home/dap/.cargo/bin:/home/dap/bin:/opt/ooce/sbin:/opt/ooce/bin:/opt/local/sbin:/opt/local/bin:/usr/sbin:/usr/bin:/bin:/sbin
++ PATH=/home/dap/omicron/out/cockroachdb/bin:/home/dap/.cargo/bin:/home/dap/bin:/opt/ooce/sbin:/opt/ooce/bin:/opt/local/sbin:/opt/local/bin:/usr/sbin:/usr/bin:/bin:/sbin
++ export PATH=/home/dap/omicron/out/clickhouse:/home/dap/omicron/out/cockroachdb/bin:/home/dap/.cargo/bin:/home/dap/bin:/opt/ooce/sbin:/opt/ooce/bin:/opt/local/sbin:/opt/local/bin:/usr/sbin:/usr/bin:/bin:/sbin
++ PATH=/home/dap/omicron/out/clickhouse:/home/dap/omicron/out/cockroachdb/bin:/home/dap/.cargo/bin:/home/dap/bin:/opt/ooce/sbin:/opt/ooce/bin:/opt/local/sbin:/opt/local/bin:/usr/sbin:/usr/bin:/bin:/sbin
++ export PATH=/home/dap/omicron/out/dendrite-stub/bin:/home/dap/omicron/out/clickhouse:/home/dap/omicron/out/cockroachdb/bin:/home/dap/.cargo/bin:/home/dap/bin:/opt/ooce/sbin:/opt/ooce/bin:/opt/local/sbin:/opt/local/bin:/usr/sbin:/usr/bin:/bin:/sbin
++ PATH=/home/dap/omicron/out/dendrite-stub/bin:/home/dap/omicron/out/clickhouse:/home/dap/omicron/out/cockroachdb/bin:/home/dap/.cargo/bin:/home/dap/bin:/opt/ooce/sbin:/opt/ooce/bin:/opt/local/sbin:/opt/local/bin:/usr/sbin:/usr/bin:/bin:/sbin
++ export PATH=/home/dap/omicron/out/mgd/root/opt/oxide/mgd/bin:/home/dap/omicron/out/dendrite-stub/bin:/home/dap/omicron/out/clickhouse:/home/dap/omicron/out/cockroachdb/bin:/home/dap/.cargo/bin:/home/dap/bin:/opt/ooce/sbin:/opt/ooce/bin:/opt/local/sbin:/opt/local/bin:/usr/sbin:/usr/bin:/bin:/sbin
++ PATH=/home/dap/omicron/out/mgd/root/opt/oxide/mgd/bin:/home/dap/omicron/out/dendrite-stub/bin:/home/dap/omicron/out/clickhouse:/home/dap/omicron/out/cockroachdb/bin:/home/dap/.cargo/bin:/home/dap/bin:/opt/ooce/sbin:/opt/ooce/bin:/opt/local/sbin:/opt/local/bin:/usr/sbin:/usr/bin:/bin:/sbin
++ unset OMICRON_WS
++ set +o xtrace
dap@maxwell ~/omicron $ ./tools/install_prerequisites.sh -y
No updates necessary for this image.
No changes required.
No changes required.
MEDIATOR   VER. SRC. VERSION IMPL. SRC. IMPLEMENTATION
clang      system    15      system     
csh        system            system     illumos
ctags      system            system     illumos
gcc        vendor    12      vendor     
gcc        system    10      system     
gcc        system    7       system     
llvm       system    15      system     
mta        vendor            vendor     dma
openjdk    system    17      system     
openjdk    system    11      system     
openjdk    system    1.8     system     
openssl    vendor    3       vendor     
perl       system    5.36    system     
postgresql system    13      system     
python     vendor    3       vendor     
python     system    2       system     
python3    system    3.11    system     
ruby       system    3.0     system     
words      vendor            vendor     american-english
words      system            system     australian-english
words      system            system     british-english
words      system            system     canadian-english
words      system            system     french
words      system            system     italian
words      system            system     ngerman
words      system            system     ogerman
words      system            system     spanish
FMRI                                                                         IFO
pkg://helios-dev/developer/build-essential@11-2.0:20230623T110919Z           im-
pkg://helios-dev/developer/pkg-config@0.29.2-2.0:20230621T191911Z            im-
pkg://helios-dev/library/libxmlsec1@1.2.35-2.0:20230624T123441Z              im-
pkg://helios-dev/ooce/developer/clang-15@15.0.7-2.0:20231012T123315Z         im-
pkg://helios-dev/ooce/library/postgresql-13@13.11-2.0:20230623T123131Z       im-
pkg://helios-dev/package/pkg@0.5.11-2.0:20230621T202036Z                     i--
+ set -o errexit
+++ dirname ./tools/ci_download_cockroachdb
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron/tools
++ basename ./tools/ci_download_cockroachdb
+ ARG0=ci_download_cockroachdb
++ cat /home/dap/omicron/tools/cockroachdb_version
+ CIDL_VERSION=v22.1.9
+ source /home/dap/omicron/tools/cockroachdb_checksums
++ CIDL_MD5_DARWIN=2db972c254b4e3b599e12110520178b5
++ CIDL_MD5_LINUX=8c3170883e0a0be1a34b44090c067a8c
++ CIDL_MD5_ILLUMOS=d8999aff364e5d70f226e139fda724a3
+ TARGET_DIR=out
+ DOWNLOAD_DIR=out/downloads
+ DEST_DIR=./out/cockroachdb
+ CIDL_URL_COCKROACH=https://binaries.cockroachdb.com
+ CIDL_URL_ILLUMOS=https://illumos.org/downloads
+ main
+ [[ 0 != 0 ]]
+ CIDL_OS=solaris2.11
+ [[ 0 != 0 ]]
+ configure_os solaris2.11
+ echo 'current directory: /home/dap/omicron'
current directory: /home/dap/omicron
+ echo 'configuring based on OS: "solaris2.11"'
configuring based on OS: "solaris2.11"
+ case "$1" in
+ CIDL_BUILD=illumos
+ CIDL_SUFFIX=tar.gz
+ CIDL_MD5=d8999aff364e5d70f226e139fda724a3
+ CIDL_MD5FUNC=do_md5sum
+ CIDL_URL_BASE=https://illumos.org/downloads
+ CIDL_ASSEMBLE=do_assemble_illumos
+ TARBALL_DIRNAME=cockroach-v22.1.9.illumos
+ TARBALL_FILENAME=cockroach-v22.1.9.illumos.tar.gz
+ TARBALL_FILE=out/downloads/cockroach-v22.1.9.illumos.tar.gz
+ TARBALL_DIR=out/downloads/cockroach-v22.1.9.illumos
+ CIDL_URL=https://illumos.org/downloads/cockroach-v22.1.9.illumos.tar.gz
+ echo 'URL: https://illumos.org/downloads/cockroach-v22.1.9.illumos.tar.gz'
URL: https://illumos.org/downloads/cockroach-v22.1.9.illumos.tar.gz
+ echo 'Local file: out/downloads/cockroach-v22.1.9.illumos.tar.gz'
Local file: out/downloads/cockroach-v22.1.9.illumos.tar.gz
+ mkdir -p out/downloads
+ mkdir -p ./out/cockroachdb
+ local DO_DOWNLOAD=true
+ [[ -f out/downloads/cockroach-v22.1.9.illumos.tar.gz ]]
++ do_md5sum out/downloads/cockroach-v22.1.9.illumos.tar.gz
++ md5sum
++ awk '{print $1}'
+ calculated_md5=d8999aff364e5d70f226e139fda724a3
+ [[ d8999aff364e5d70f226e139fda724a3 == \d\8\9\9\9\a\f\f\3\6\4\e\5\d\7\0\f\2\2\6\e\1\3\9\f\d\a\7\2\4\a\3 ]]
+ DO_DOWNLOAD=false
+ '[' false == true ']'
+ do_untar out/downloads/cockroach-v22.1.9.illumos.tar.gz
+ tar xzf out/downloads/cockroach-v22.1.9.illumos.tar.gz -C out/downloads
+ do_assemble_illumos
+ rm -r ./out/cockroachdb
+ cp -r out/downloads/cockroach-v22.1.9 ./out/cockroachdb
+ ./out/cockroachdb/bin/cockroach version
Build Tag:        v22.1.9-dirty
Build Time:       2022/10/26 21:17:46
Distribution:     OSS
Platform:         illumos amd64 (x86_64-pc-solaris2.11)
Go Version:       go1.17.13
C Compiler:       gcc 10.3.0
Build Commit ID:  e438c2f89282e607e0e6ca1d38b2e0a622f94493
Build Type:       release
+ set -o errexit
+++ dirname ./tools/ci_download_clickhouse
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron/tools
++ basename ./tools/ci_download_clickhouse
+ ARG0=ci_download_clickhouse
+ TARGET_DIR=out
+ DOWNLOAD_DIR=out/downloads
+ DEST_DIR=./out/clickhouse
+ CIDL_VERSION=v22.8.9.24
+ source /home/dap/omicron/tools/clickhouse_checksums
++ CIDL_MD5_DARWIN=20603974a929926591fca70ff1df0e45
++ CIDL_MD5_LINUX=ea909519bd9d989fd5d090fd9bdd42f1
++ CIDL_MD5_ILLUMOS=7702939ce5b4b51846a1ba39f1392306
+ CIDL_URL_BASE=https://oxide-clickhouse-build.s3.us-west-2.amazonaws.com
+ main
+ [[ 0 != 0 ]]
+ CIDL_OS=solaris2.11
+ [[ 0 != 0 ]]
+ configure_os solaris2.11
+ echo 'current directory: /home/dap/omicron'
current directory: /home/dap/omicron
+ echo 'configuring based on OS: "solaris2.11"'
configuring based on OS: "solaris2.11"
+ CIDL_DASHREV=
+ case "$1" in
+ CIDL_PLATFORM=illumos
+ CIDL_MD5=7702939ce5b4b51846a1ba39f1392306
+ CIDL_MD5FUNC=do_md5sum
+ CIDL_DASHREV=-1
+ TARBALL_DIRNAME=clickhouse-v22.8.9.24
+ TARBALL_FILENAME=clickhouse-v22.8.9.24-1.illumos.tar.gz
+ TARBALL_FILE=out/downloads/clickhouse-v22.8.9.24-1.illumos.tar.gz
+ CIDL_URL=https://oxide-clickhouse-build.s3.us-west-2.amazonaws.com/clickhouse-v22.8.9.24-1.illumos.tar.gz
+ echo 'URL: https://oxide-clickhouse-build.s3.us-west-2.amazonaws.com/clickhouse-v22.8.9.24-1.illumos.tar.gz'
URL: https://oxide-clickhouse-build.s3.us-west-2.amazonaws.com/clickhouse-v22.8.9.24-1.illumos.tar.gz
+ echo 'Local file: out/downloads/clickhouse-v22.8.9.24-1.illumos.tar.gz'
Local file: out/downloads/clickhouse-v22.8.9.24-1.illumos.tar.gz
+ mkdir -p out/downloads
+ mkdir -p ./out/clickhouse
+ local DO_DOWNLOAD=true
+ [[ -f out/downloads/clickhouse-v22.8.9.24-1.illumos.tar.gz ]]
++ do_md5sum out/downloads/clickhouse-v22.8.9.24-1.illumos.tar.gz
++ md5sum
++ awk '{print $1}'
+ calculated_md5=7702939ce5b4b51846a1ba39f1392306
+ [[ 7702939ce5b4b51846a1ba39f1392306 == \7\7\0\2\9\3\9\c\e\5\b\4\b\5\1\8\4\6\a\1\b\a\3\9\f\1\3\9\2\3\0\6 ]]
+ DO_DOWNLOAD=false
+ '[' false == true ']'
+ do_untar out/downloads/clickhouse-v22.8.9.24-1.illumos.tar.gz ./out/clickhouse
+ mkdir -p ./out/clickhouse
+ tar xzf out/downloads/clickhouse-v22.8.9.24-1.illumos.tar.gz -C ./out/clickhouse
+ ./out/clickhouse/clickhouse server --version
ClickHouse server version 22.8.9.1.
+ set -o errexit
+++ dirname ./tools/ci_download_console
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron/tools
++ basename ./tools/ci_download_console
+ ARG0=ci_download_console
+ TARGET_DIR=out
+ DOWNLOAD_DIR=out/downloads
+ DEST_DIR=./out/console-assets
+ source /home/dap/omicron/tools/console_version
++ COMMIT=bd65b9da7019ad812dd056e7fc182df2cf4ec128
++ SHA2=e4d4f33996a6e89b972fac61737acb7f1dbd21943d1f6bef776d4ee9bcccd2b0
+ URL=https://dl.oxide.computer/releases/console/bd65b9da7019ad812dd056e7fc182df2cf4ec128.tar.gz
+ TARBALL_FILE=out/downloads/console.tar.gz
+ main
+ [[ 0 != 0 ]]
+ echo 'URL: https://dl.oxide.computer/releases/console/bd65b9da7019ad812dd056e7fc182df2cf4ec128.tar.gz'
URL: https://dl.oxide.computer/releases/console/bd65b9da7019ad812dd056e7fc182df2cf4ec128.tar.gz
+ echo 'Local file: out/downloads/console.tar.gz'
Local file: out/downloads/console.tar.gz
+ local DO_DOWNLOAD=true
+ [[ -f out/downloads/console.tar.gz ]]
++ do_sha256sum out/downloads/console.tar.gz
++ case "$OSTYPE" in
++ SHA=sha256sum
++ sha256sum
++ awk '{print $1}'
+ calculated_sha2=e4d4f33996a6e89b972fac61737acb7f1dbd21943d1f6bef776d4ee9bcccd2b0
+ [[ e4d4f33996a6e89b972fac61737acb7f1dbd21943d1f6bef776d4ee9bcccd2b0 == \e\4\d\4\f\3\3\9\9\6\a\6\e\8\9\b\9\7\2\f\a\c\6\1\7\3\7\a\c\b\7\f\1\d\b\d\2\1\9\4\3\d\1\f\6\b\e\f\7\7\6\d\4\e\e\9\b\c\c\c\d\2\b\0 ]]
+ DO_DOWNLOAD=false
+ mkdir -p out/downloads
+ '[' false == true ']'
+ rm -rf ./out/console-assets
+ mkdir -p ./out/console-assets
+ do_untar out/downloads/console.tar.gz ./out/console-assets
+ mkdir -p ./out/console-assets
+ tar xzf out/downloads/console.tar.gz -C ./out/console-assets
+ set -o errexit
+++ dirname ./tools/ci_download_maghemite_openapi
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron/tools
++ basename ./tools/ci_download_maghemite_openapi
+ ARG0=ci_download_maghemite_openapi
+ TARGET_DIR=out
+ DOWNLOAD_DIR=out/downloads
+ source /home/dap/omicron/tools/maghemite_ddm_openapi_version
++ COMMIT=d7169a61fd8833b3a1e6f46d897ca3295b2a28b6
++ SHA2=9737906555a60911636532f00f1dc2866dc7cd6553beb106e9e57beabad41cdf
+ URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/maghemite/openapi/d7169a61fd8833b3a1e6f46d897ca3295b2a28b6/ddm-admin.json
+ LOCAL_FILE=out/downloads/ddm-admin-d7169a61fd8833b3a1e6f46d897ca3295b2a28b6.json
+ main
+ [[ 0 != 0 ]]
+ echo 'URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/maghemite/openapi/d7169a61fd8833b3a1e6f46d897ca3295b2a28b6/ddm-admin.json'
URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/maghemite/openapi/d7169a61fd8833b3a1e6f46d897ca3295b2a28b6/ddm-admin.json
+ echo 'Local file: out/downloads/ddm-admin-d7169a61fd8833b3a1e6f46d897ca3295b2a28b6.json'
Local file: out/downloads/ddm-admin-d7169a61fd8833b3a1e6f46d897ca3295b2a28b6.json
+ local DO_DOWNLOAD=true
+ [[ -f out/downloads/ddm-admin-d7169a61fd8833b3a1e6f46d897ca3295b2a28b6.json ]]
++ do_sha256sum out/downloads/ddm-admin-d7169a61fd8833b3a1e6f46d897ca3295b2a28b6.json
++ case "$OSTYPE" in
++ SHA=sha256sum
++ sha256sum
++ awk '{print $1}'
+ calculated_sha2=9737906555a60911636532f00f1dc2866dc7cd6553beb106e9e57beabad41cdf
+ [[ 9737906555a60911636532f00f1dc2866dc7cd6553beb106e9e57beabad41cdf == \9\7\3\7\9\0\6\5\5\5\a\6\0\9\1\1\6\3\6\5\3\2\f\0\0\f\1\d\c\2\8\6\6\d\c\7\c\d\6\5\5\3\b\e\b\1\0\6\e\9\e\5\7\b\e\a\b\a\d\4\1\c\d\f ]]
+ DO_DOWNLOAD=false
+ mkdir -p out/downloads
+ '[' false == true ']'
+ source /home/dap/omicron/tools/maghemite_mg_openapi_version
++ COMMIT=d7169a61fd8833b3a1e6f46d897ca3295b2a28b6
++ SHA2=d0f7611e5ecd049b0f83bcfa843942401f155a0be36d9a2dfd73b8341d5f816e
+ URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/maghemite/openapi/d7169a61fd8833b3a1e6f46d897ca3295b2a28b6/mg-admin.json
+ LOCAL_FILE=out/downloads/mg-admin-d7169a61fd8833b3a1e6f46d897ca3295b2a28b6.json
+ main
+ [[ 0 != 0 ]]
+ echo 'URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/maghemite/openapi/d7169a61fd8833b3a1e6f46d897ca3295b2a28b6/mg-admin.json'
URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/maghemite/openapi/d7169a61fd8833b3a1e6f46d897ca3295b2a28b6/mg-admin.json
+ echo 'Local file: out/downloads/mg-admin-d7169a61fd8833b3a1e6f46d897ca3295b2a28b6.json'
Local file: out/downloads/mg-admin-d7169a61fd8833b3a1e6f46d897ca3295b2a28b6.json
+ local DO_DOWNLOAD=true
+ [[ -f out/downloads/mg-admin-d7169a61fd8833b3a1e6f46d897ca3295b2a28b6.json ]]
++ do_sha256sum out/downloads/mg-admin-d7169a61fd8833b3a1e6f46d897ca3295b2a28b6.json
++ case "$OSTYPE" in
++ SHA=sha256sum
++ sha256sum
++ awk '{print $1}'
+ calculated_sha2=d0f7611e5ecd049b0f83bcfa843942401f155a0be36d9a2dfd73b8341d5f816e
+ [[ d0f7611e5ecd049b0f83bcfa843942401f155a0be36d9a2dfd73b8341d5f816e == \d\0\f\7\6\1\1\e\5\e\c\d\0\4\9\b\0\f\8\3\b\c\f\a\8\4\3\9\4\2\4\0\1\f\1\5\5\a\0\b\e\3\6\d\9\a\2\d\f\d\7\3\b\8\3\4\1\d\5\f\8\1\6\e ]]
+ DO_DOWNLOAD=false
+ mkdir -p out/downloads
+ '[' false == true ']'
+ set -o errexit
+++ dirname ./tools/ci_download_dendrite_openapi
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron/tools
++ basename ./tools/ci_download_dendrite_openapi
+ ARG0=ci_download_dendrite_openapi
+ TARGET_DIR=out
+ DOWNLOAD_DIR=out/downloads
+ source /home/dap/omicron/tools/dendrite_openapi_version
++ COMMIT=343e3a572cc02efe3f8b68f9affd008623a33966
++ SHA2=544ab42ccc7942d8ece9cdc80cd85d002bcf9d5646a291322bf2f79087ab6df0
+ URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/dendrite/openapi/343e3a572cc02efe3f8b68f9affd008623a33966/dpd.json
+ LOCAL_FILE=out/downloads/dpd-343e3a572cc02efe3f8b68f9affd008623a33966.json
+ main
+ [[ 0 != 0 ]]
+ echo 'URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/dendrite/openapi/343e3a572cc02efe3f8b68f9affd008623a33966/dpd.json'
URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/dendrite/openapi/343e3a572cc02efe3f8b68f9affd008623a33966/dpd.json
+ echo 'Local file: out/downloads/dpd-343e3a572cc02efe3f8b68f9affd008623a33966.json'
Local file: out/downloads/dpd-343e3a572cc02efe3f8b68f9affd008623a33966.json
+ local DO_DOWNLOAD=true
+ [[ -f out/downloads/dpd-343e3a572cc02efe3f8b68f9affd008623a33966.json ]]
++ do_sha256sum out/downloads/dpd-343e3a572cc02efe3f8b68f9affd008623a33966.json
++ case "$OSTYPE" in
++ SHA=sha256sum
++ sha256sum
++ awk '{print $1}'
+ calculated_sha2=544ab42ccc7942d8ece9cdc80cd85d002bcf9d5646a291322bf2f79087ab6df0
+ [[ 544ab42ccc7942d8ece9cdc80cd85d002bcf9d5646a291322bf2f79087ab6df0 == \5\4\4\a\b\4\2\c\c\c\7\9\4\2\d\8\e\c\e\9\c\d\c\8\0\c\d\8\5\d\0\0\2\b\c\f\9\d\5\6\4\6\a\2\9\1\3\2\2\b\f\2\f\7\9\0\8\7\a\b\6\d\f\0 ]]
+ DO_DOWNLOAD=false
+ mkdir -p out/downloads
+ '[' false == true ']'
+ set -o errexit
+++ dirname ./tools/ci_download_dendrite_stub
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron/tools
++ basename ./tools/ci_download_dendrite_stub
+ ARG0=ci_download_dendrite_stub
+ source /home/dap/omicron/tools/dendrite_stub_checksums
++ CIDL_SHA256_ILLUMOS=0808f331741e02d55e199847579dfd01f3658b21c7122cef8c3f9279f43dbab0
++ CIDL_SHA256_LINUX_DPD=3e276dd553dd7cdb75c8ad023c2cd29b91485fafb94f27097a745b2b7ef5ecea
++ CIDL_SHA256_LINUX_SWADM=645faf8a93bcae9814b2f116bccd66a54763332b56220e93b66316c853ce13d2
+ source /home/dap/omicron/tools/dendrite_openapi_version
++ COMMIT=343e3a572cc02efe3f8b68f9affd008623a33966
++ SHA2=544ab42ccc7942d8ece9cdc80cd85d002bcf9d5646a291322bf2f79087ab6df0
+ TARGET_DIR=out
+ DOWNLOAD_DIR=out/downloads
+ DEST_DIR=./out/dendrite-stub
+ BIN_DIR=./out/dendrite-stub/root/opt/oxide/dendrite/bin
+ ARTIFACT_URL=https://buildomat.eng.oxide.computer/public/file
+ REPO=oxidecomputer/dendrite
+ PACKAGE_BASE_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/dendrite/image/343e3a572cc02efe3f8b68f9affd008623a33966
+ main
+ [[ 0 != 0 ]]
+ CIDL_OS=solaris2.11
+ [[ 0 != 0 ]]
+ configure_os solaris2.11
+ echo 'current directory: /home/dap/omicron'
current directory: /home/dap/omicron
+ echo 'configuring based on OS: "solaris2.11"'
configuring based on OS: "solaris2.11"
+ case "$1" in
+ SET_BINARIES=
+ CIDL_SHA256=0808f331741e02d55e199847579dfd01f3658b21c7122cef8c3f9279f43dbab0
+ CIDL_SHA256FUNC=do_sha256sum
+ TARBALL_FILENAME=dendrite-stub.tar.gz
+ PACKAGE_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/dendrite/image/343e3a572cc02efe3f8b68f9affd008623a33966/dendrite-stub.tar.gz
+ TARBALL_FILE=out/downloads/dendrite-stub.tar.gz
+ echo 'URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/dendrite/image/343e3a572cc02efe3f8b68f9affd008623a33966/dendrite-stub.tar.gz'
URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/dendrite/image/343e3a572cc02efe3f8b68f9affd008623a33966/dendrite-stub.tar.gz
+ echo 'Local file: out/downloads/dendrite-stub.tar.gz'
Local file: out/downloads/dendrite-stub.tar.gz
+ mkdir -p out/downloads
+ mkdir -p ./out/dendrite-stub
+ fetch_and_verify
+ local DO_DOWNLOAD=true
+ [[ -f out/downloads/dendrite-stub.tar.gz ]]
++ do_sha256sum out/downloads/dendrite-stub.tar.gz
++ sha256sum
++ awk '{print $1}'
+ calculated_sha256=0808f331741e02d55e199847579dfd01f3658b21c7122cef8c3f9279f43dbab0
+ [[ 0808f331741e02d55e199847579dfd01f3658b21c7122cef8c3f9279f43dbab0 == \0\8\0\8\f\3\3\1\7\4\1\e\0\2\d\5\5\e\1\9\9\8\4\7\5\7\9\d\f\d\0\1\f\3\6\5\8\b\2\1\c\7\1\2\2\c\e\f\8\c\3\f\9\2\7\9\f\4\3\d\b\a\b\0 ]]
+ DO_DOWNLOAD=false
+ '[' false == true ']'
+ do_untar out/downloads/dendrite-stub.tar.gz
+ tar xzf out/downloads/dendrite-stub.tar.gz -C out/downloads
+ do_assemble
+ rm -r ./out/dendrite-stub
+ mkdir ./out/dendrite-stub
+ cp -r out/downloads/root ./out/dendrite-stub/root
+ ln -s /home/dap/omicron/out/dendrite-stub/root/opt/oxide/dendrite/bin/ /home/dap/omicron/out/dendrite-stub/bin
+ set -o errexit
+++ dirname ./tools/ci_download_maghemite_mgd
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron/tools
++ basename ./tools/ci_download_maghemite_mgd
+ ARG0=ci_download_maghemite_mgd
+ source /home/dap/omicron/tools/maghemite_mgd_checksums
++ CIDL_SHA256=452dfb3491e1b6d4df6be1cb689921f59623aed082e47606a78c0f44d918f66a
++ MGD_LINUX_SHA256=d4c48eb6374c0cc7812b7af2c0ac92acdcbc91b7718a9ce64d069da00ae5ae73
+ source /home/dap/omicron/tools/maghemite_mg_openapi_version
++ COMMIT=d7169a61fd8833b3a1e6f46d897ca3295b2a28b6
++ SHA2=d0f7611e5ecd049b0f83bcfa843942401f155a0be36d9a2dfd73b8341d5f816e
+ TARGET_DIR=out
+ DOWNLOAD_DIR=out/downloads
+ DEST_DIR=./out/mgd
+ BIN_DIR=./out/mgd/root/opt/oxide/mgd/bin
+ ARTIFACT_URL=https://buildomat.eng.oxide.computer/public/file
+ REPO=oxidecomputer/maghemite
+ PACKAGE_BASE_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/maghemite/image/d7169a61fd8833b3a1e6f46d897ca3295b2a28b6
+ main
+ [[ 0 != 0 ]]
+ CIDL_OS=solaris2.11
+ [[ 0 != 0 ]]
+ configure_os solaris2.11
+ echo 'current directory: /home/dap/omicron'
current directory: /home/dap/omicron
+ echo 'configuring based on OS: "solaris2.11"'
configuring based on OS: "solaris2.11"
+ case "$1" in
+ SET_BINARIES=
+ CIDL_SHA256FUNC=do_sha256sum
+ TARBALL_FILENAME=mgd.tar.gz
+ PACKAGE_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/maghemite/image/d7169a61fd8833b3a1e6f46d897ca3295b2a28b6/mgd.tar.gz
+ TARBALL_FILE=out/downloads/mgd.tar.gz
+ echo 'URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/maghemite/image/d7169a61fd8833b3a1e6f46d897ca3295b2a28b6/mgd.tar.gz'
URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/maghemite/image/d7169a61fd8833b3a1e6f46d897ca3295b2a28b6/mgd.tar.gz
+ echo 'Local file: out/downloads/mgd.tar.gz'
Local file: out/downloads/mgd.tar.gz
+ mkdir -p out/downloads
+ mkdir -p ./out/mgd
+ fetch_and_verify
+ local DO_DOWNLOAD=true
+ [[ -f out/downloads/mgd.tar.gz ]]
++ do_sha256sum out/downloads/mgd.tar.gz
++ sha256sum
++ awk '{print $1}'
+ calculated_sha256=452dfb3491e1b6d4df6be1cb689921f59623aed082e47606a78c0f44d918f66a
+ [[ 452dfb3491e1b6d4df6be1cb689921f59623aed082e47606a78c0f44d918f66a == \4\5\2\d\f\b\3\4\9\1\e\1\b\6\d\4\d\f\6\b\e\1\c\b\6\8\9\9\2\1\f\5\9\6\2\3\a\e\d\0\8\2\e\4\7\6\0\6\a\7\8\c\0\f\4\4\d\9\1\8\f\6\6\a ]]
+ DO_DOWNLOAD=false
+ '[' false == true ']'
+ do_untar out/downloads/mgd.tar.gz
+ tar xzf out/downloads/mgd.tar.gz -C out/downloads
+ do_assemble
+ rm -r ./out/mgd
+ mkdir ./out/mgd
+ cp -r out/downloads/root ./out/mgd/root
+ set -o errexit
+++ dirname ./tools/ci_download_transceiver_control
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron/tools
++ basename ./tools/ci_download_transceiver_control
+ ARG0=ci_download_transceiver_control
+ source /home/dap/omicron/tools/transceiver_control_version
++ COMMIT=8813cf996f37a6c43e7b83d6084f88c5f0f0430a
++ CIDL_SHA256_ILLUMOS=388f9e03089e8bc6d47fd21f9c83554a6cdc626eb928374fcff945a881e1f7e1
+ TARGET_DIR=out
+ DOWNLOAD_DIR=out/downloads
+ DEST_DIR=./out/transceiver-control
+ BIN_DIR=/opt/oxide/bin
+ BINARY=xcvradm
+ ARTIFACT_URL=https://buildomat.eng.oxide.computer/public/file
+ REPO=oxidecomputer/transceiver-control
+ PACKAGE_BASE_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/transceiver-control/bins/8813cf996f37a6c43e7b83d6084f88c5f0f0430a
+ main
+ [[ 0 != 0 ]]
+ CIDL_OS=solaris2.11
+ [[ 0 != 0 ]]
+ configure_os solaris2.11
+ echo 'current directory: /home/dap/omicron'
current directory: /home/dap/omicron
+ echo 'configuring based on OS: "solaris2.11"'
configuring based on OS: "solaris2.11"
+ case "$1" in
+ SET_BINARIES=
+ CIDL_SHA256=388f9e03089e8bc6d47fd21f9c83554a6cdc626eb928374fcff945a881e1f7e1
+ CIDL_SHA256FUNC=do_sha256sum
+ ARCHIVE_FILENAME=xcvradm.gz
+ PACKAGE_URL=https://buildomat.eng.oxide.computer/public/file/oxidecomputer/transceiver-control/bins/8813cf996f37a6c43e7b83d6084f88c5f0f0430a/xcvradm.gz
+ ARCHIVE_FILE=out/downloads/xcvradm.gz
+ echo 'URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/transceiver-control/bins/8813cf996f37a6c43e7b83d6084f88c5f0f0430a/xcvradm.gz'
URL: https://buildomat.eng.oxide.computer/public/file/oxidecomputer/transceiver-control/bins/8813cf996f37a6c43e7b83d6084f88c5f0f0430a/xcvradm.gz
+ echo 'Local file: out/downloads/xcvradm.gz'
Local file: out/downloads/xcvradm.gz
+ mkdir -p out/downloads
+ mkdir -p ./out/transceiver-control
+ fetch_and_verify
+ local DO_DOWNLOAD=true
+ [[ -f out/downloads/xcvradm.gz ]]
++ do_sha256sum out/downloads/xcvradm.gz
++ sha256sum
++ awk '{print $1}'
+ calculated_sha256=388f9e03089e8bc6d47fd21f9c83554a6cdc626eb928374fcff945a881e1f7e1
+ [[ 388f9e03089e8bc6d47fd21f9c83554a6cdc626eb928374fcff945a881e1f7e1 == \3\8\8\f\9\e\0\3\0\8\9\e\8\b\c\6\d\4\7\f\d\2\1\f\9\c\8\3\5\5\4\a\6\c\d\c\6\2\6\e\b\9\2\8\3\7\4\f\c\f\f\9\4\5\a\8\8\1\e\1\f\7\e\1 ]]
+ DO_DOWNLOAD=false
+ '[' false == true ']'
+ do_unpack out/downloads/xcvradm.gz
+ mkdir -p out/downloads/root//opt/oxide/bin
+ gzip -dc out/downloads/xcvradm.gz
+ chmod +x out/downloads/root//opt/oxide/bin/xcvradm
+ do_assemble
+ rm -r ./out/transceiver-control
+ mkdir ./out/transceiver-control
+ cp -r out/downloads/root ./out/transceiver-control/root
All builder prerequisites installed successfully, and PATH looks valid
No updates necessary for this image.
FMRI                                                                         IFO
pkg://helios-dev/developer/pkg-config@0.29.2-2.0:20230621T191911Z            im-
pkg://helios-dev/library/libxmlsec1@1.2.35-2.0:20230624T123441Z              im-
pkg://helios-dev/ooce/library/postgresql-13@13.11-2.0:20230623T123131Z       im-
pkg://helios-dev/package/pkg@0.5.11-2.0:20230621T202036Z                     i--
pkg://helios-dev/service/network/chrony@4.3-2.0:20230804T102734Z             im-
pkg://helios-dev/system/zones/brand/omicron1/tools@1.0.16:20230601T213844Z   im-
+ MARKER=/etc/opt/oxide/NO_INSTALL
+ [[ -f /etc/opt/oxide/NO_INSTALL ]]
++ uname
+ [[ SunOS != \S\u\n\O\S ]]
+++ dirname ./tools/install_opte.sh
++ cd ./tools
++ pwd
+ SOURCE_DIR=/home/dap/omicron/tools
+ cd /home/dap/omicron/tools/..
+ OMICRON_TOP=/home/dap/omicron
+ RC=0
+ pkg info -lq driver/network/opte
+ [[ 0 -eq 0 ]]
++ pkg info -l driver/network/opte
++ grep Version
++ tr -s ' '
++ cut -d ' ' -f 3
++ cut -d . -f 2
+ OPTE_VERSION=25
+ [[ 25 -lt 21 ]]
+ RC=0
+ man -l mac_getinfo
mac_getinfo(9f) -M /usr/share/man
+ [[ 0 -ne 0 ]]
++ cat /home/dap/omicron/tools/opte_version
+ OPTE_VERSION=0.25.183
+ OMICRON_FROZEN_PKG_COMMENT=OMICRON-PINNED-PACKAGE
++ pkg freeze
++ grep driver/network/opte
+ PKG_FROZEN='driver/network/opte 0.25.183                    02 Nov 2023 23:05:59 UTC OMICRON-PINNED-PACKAGE'
++ echo 'driver/network/opte 0.25.183                    02 Nov 2023 23:05:59 UTC OMICRON-PINNED-PACKAGE'
++ awk '{ print $(NF) }'
+ FROZEN_COMMENT=OMICRON-PINNED-PACKAGE
+ '[' OMICRON-PINNED-PACKAGE '!=' OMICRON-PINNED-PACKAGE ']'
+ pfexec pkg unfreeze driver/network/opte
driver/network/opte was unfrozen.
+ RC=0
+ pfexec pkg install -v pkg://helios-dev/driver/network/opte@0.25.183
No updates necessary for this image.
+ RC=4
+ [[ 4 -eq 0 ]]
+ [[ 4 -eq 4 ]]
+ echo 'Correct xde driver already installed'
Correct xde driver already installed
+ RC=0
+ pfexec pkg freeze -c OMICRON-PINNED-PACKAGE driver/network/opte@0.25.183
driver/network/opte was frozen at 0.25.183
+ [[ 0 -ne 0 ]]
+ RC=0
+ which opteadm
+ RC=1
+ [[ 1 -ne 0 ]]
+ echo 'The `opteadm` administration tool is not on your path.'
The `opteadm` administration tool is not on your path.
+ echo 'You may add "/opt/oxide/opte/bin" to your path to access it.'
You may add "/opt/oxide/opte/bin" to your path to access it.
fetching npuzone
Ensuring oxidecomputer/softnpu/image/npuzone in out/npuzone
  (commit: c1c42398c82b0220c8b5fa3bfba9c7a3bcaa0943)
Getting hash for npuzone
  (hash: 4e3cd20ca95b90f85396e965a68dac08e26c0454be09c388b91cf679eea6d90e)
npuzone already exists with correct hash
All runner prerequisites installed successfully

At that point I tried to uninstall again:

dap@maxwell ~/omicron $ pfexec ./target/release/omicron-package uninstall
Nov 02 23:55:59.962 DEBG target[active]: Target({"image": "standard", "machine": "non-gimlet", "rack-topology": "multi-sled", "switch": "softnpu"})
Nov 02 23:55:59.964 INFO Removing all Omicron zones
Nov 02 23:55:59.970 INFO Uninstalling all packages
Nov 02 23:56:00.059 INFO Removing networking resources
Nov 02 23:56:00.078 WARN Deleting Omicron underlay VNIC, vnic_name: underlay0
Nov 02 23:56:00.088 WARN Deleting Omicron underlay etherstub, stub_name: underlay_stub0
Nov 02 23:56:00.093 WARN Deleting Omicron bootstrap VNIC, vnic_name: bootstrap0
Nov 02 23:56:00.107 WARN Deleting Omicron bootstrap etherstub, stub_name: bootstrap_stub0
Error: Failure interacting with the OPTE ioctl(2) interface: command ListPorts failed: BadApiVersion { user: 25, kernel: 23 }

Caused by:
    command ListPorts failed: BadApiVersion { user: 25, kernel: 23 }

In case it's helpful:

$ /opt/oxide/opte/bin/opteadm -V
opteadm 0.25.183

$ pkg list driver/network/opte
NAME (PUBLISHER)                                  VERSION                    IFO
driver/network/opte                               0.25.183                   if-

$ pkg verify opte
$

After running update_drv xde, I was able to teardown and deploy successfully. Some part of the tooling should do this for you (or else the instructions should be updated to say that you have to do it).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the omicron-package install entry point and tools/install_prerequisites.sh, then trace how the xde/OPTE driver is handled after an update. Use the reported BadApiVersion mismatch as the reproduction signal; done means the updated driver is loaded before installation uses the OPTE ioctl interface.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
operating-systems, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.