mozilla / mozilla/bugbug

Bring the hackbot images much closer to e.g. Firefox test machines

Open
#6,774 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
570
Forks
351
Avg merge
2d 13h
Merged PRs (30d)
65

Description

https://searchfox.org/firefox-main/source/taskcluster/scripts/tester/test-linux.sh is useful configuration, https://searchfox.org/firefox-main/source/taskcluster/docker/recipes/ubuntu2404-test-system-setup-base.sh is a list of packages.

It feels like there would be some value in moving from a Debian python image to something closer to what a Firefox user would use.

This would make scenario like: https://hackbot.moz.tools/runs/1e26c389-3efa-4088-9e30-6c44cf4eb0ab, better.

Relevant excerpts from the LLM trace ``` pid:4307 /workspace/firefox/obj-artifact/dist/bin/xpcshell: error while loading shared libraries: libgtk-3.so.0: cannot open shared object file: No such file or directory ```
[agent→tool] Bash
{
  "command": "ldconfig -p 2>/dev/null | grep -c gtk; ls /usr/lib/x86_64-linux-gnu/ 2>/dev/null | grep -i \"gtk\\|gdk\" | head; echo \"---\"; id; sudo -n true 2>&1 | head -2",
  "description": "Check for GTK libraries and sudo"
}
--- turn 118 ---
[agent→tool] Bash
{
  "command": "cd /workspace/firefox/obj-artifact/dist/bin && LD_LIBRARY_PATH=. ldd xpcshell 2>&1 | grep \"not found\" | sort -u; echo \"=== libxul ===\"; LD_LIBRARY_PATH=. ldd libxul.so 2>&1 | grep \"not found\" | sort -u",
  "description": "List missing shared libraries"
}
  [tool←ok]
	libXcomposite.so.1 => not found
	libXdamage.so.1 => not found
	libatk-1.0.so.0 => not found
	libgdk-3.so.0 => not found
	libgtk-3.so.0 => not found
=== libxul ===
	libXcomposite.so.1 => not found
	libXdamage.so.1 => not found
	libatk-1.0.so.0 => not found
	libgdk-3.so.0 => not found
	libgtk-3.so.0 => not found

--- turn 119 ---
[agent:thinking]


--- turn 120 ---
[agent→tool] Bash
{
  "command": "mkdir -p /tmp/gtklibs && cd /tmp/gtklibs && apt-get download libgtk-3-0t64 libgdk-pixbuf-2.0-0 libatk1.0-0t64 libxcomposite1 libxdamage1 2>&1 | tail -10; ls",
  "description": "Try downloading gtk debs without root"
}
  [tool←ok]
E: Unable to locate package libgtk-3-0t64
E: Unable to locate package libatk1.0-0t64
E: Couldn't find any package by glob 'libatk1.0-0t64'
E: Couldn't find any package by regex 'libatk1.0-0t64'
E: Unable to locate package libxcomposite1
E: Unable to locate package libxdamage1
E: Can't find a source to download version '2.42.12+dfsg-4+deb13u1' of 'libgdk-pixbuf-2.0-0:amd64'
Shell cwd was reset to /workspace/firefox

--- turn 121 ---
[agent→tool] Bash
{
  "command": "cat /etc/os-release | head -3; apt-get update 2>&1 | tail -5",
  "description": "Check distro and try apt update"
}
  [tool←ok]
PRETTY_NAME="Debian GNU/Linux 13 (trixie)"
NAME="Debian GNU/Linux"
VERSION_ID="13"
Reading package lists...
E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied)
[system:thinking_tokens] {"type": "system", "subtype": "thinking_tokens", "estimated_tokens": 100, "estimated_tokens_delta": 100, "uuid": "14a31134-e8db-4972-866f-3641a4868f0f", "session_id": "715022aa-9a42-4c4d-bff2-71b60de8828a"}
[system:thinking_tokens] {"type": "system", "subtype": "thinking_tokens", "estimated_tokens": 200, "estimated_tokens_delta": 100, "uuid": "7bfd1516-2a25-4328-a024-714b8f2ff256", "session_id": "715022aa-9a42-4c4d-bff2-71b60de8828a"}
[system:thinking_tokens] {"type": "system", "subtype": "thinking_tokens", "estimated_tokens": 300, "estimated_tokens_delta": 100, "uuid": "08fdc813-b4ea-419e-bce2-d1689d83ce04", "session_id": "715022aa-9a42-4c4d-bff2-71b60de8828a"}
[system:thinking_tokens] {"type": "system", "subtype": "thinking_tokens", "estimated_tokens": 400, "estimated_tokens_delta": 100, "uuid": "33768b4e-377f-4ced-a079-d386201d81e7", "session_id": "715022aa-9a42-4c4d-bff2-71b60de8828a"}
[system:thinking_tokens] {"type": "system", "subtype": "thinking_tokens", "estimated_tokens": 550, "estimated_tokens_delta": 150, "uuid": "e5ef7c1a-4111-4c51-858b-991e77a7bb57", "session_id": "715022aa-9a42-4c4d-bff2-71b60de8828a"}

--- turn 122 ---
[agent:thinking]


--- turn 123 ---
[agent→tool] Bash
{
  "command": "cd /tmp/gtklibs && timeout 300 curl -sO https://deb.debian.org/debian/dists/trixie/main/binary-amd64/Packages.gz && gunzip -f Packages.gz && ls -la Packages && grep -c \"^Package:\" Packages",
  "description": "Download Debian package index",
  "timeout": 300000
}
  [tool←ok]
-rw-r--r-- 1 agent agent 56561288 Sep  3 10:59 Packages
68755
Shell cwd was reset to /workspace/firefox

--- turn 124 ---
[agent→tool] Bash
{
  "command": "cd /tmp/gtklibs && python3 - <<'EOF'\nimport re\npkgs={}\ncur={}\nfor block in open('Packages',encoding='utf-8',errors='replace').read().split('\\n\\n'):\n    if not block.strip(): continue\n    d={}\n    for line in block.split('\\n'):\n        if line.startswith(' '): continue\n        if ':' in line:\n            k,v=line.split(':',1); d[k]=v.strip()\n    if 'Package' in d: pkgs[d['Package']]=d\nimport json\nwant=['libgtk-3-0t64','libatk1.0-0t64','libatk-bridge2.0-0t64','libxcomposite1','libxdamage1','libepoxy0','libcloudproviders0','libxkbcommon0','libxinerama1','libxrandr2','libxcursor1','libxi6','libxfixes3','libcairo-gobject2','libpangoft2-1.0-0','libpangocairo-1.0-0','libwayland-cursor0','libwayland-egl1','libcolord2','libgtk-3-common']\nfor w in want:\n    if w in pkgs:\n        print(pkgs[w]['Filename'])\n    else:\n        print(\"MISSING\",w)\nEOF",
  "description": "Resolve deb filenames from index"
}
  [tool←ok]
pool/main/g/gtk+3.0/libgtk-3-0t64_3.24.49-3_amd64.deb
pool/main/a/at-spi2-core/libatk1.0-0t64_2.56.2-1+deb13u1_amd64.deb
pool/main/a/at-spi2-core/libatk-bridge2.0-0t64_2.56.2-1+deb13u1_amd64.deb
pool/main/libx/libxcomposite/libxcomposite1_0.4.6-1_amd64.deb
pool/main/libx/libxdamage/libxdamage1_1.1.6-1+b2_amd64.deb
pool/main/libe/libepoxy/libepoxy0_1.5.10-2_amd64.deb
pool/main/libc/libcloudproviders/libcloudproviders0_0.3.6-2_amd64.deb
pool/main/libx/libxkbcommon/libxkbcommon0_1.7.0-2_amd64.deb
pool/main/libx/libxinerama/libxinerama1_1.1.4-3+b4_amd64.deb
pool/main/libx/libxrandr/libxrandr2_1.5.4-1+b3_amd64.deb
pool/main/libx/libxcursor/libxcursor1_1.2.3-1_amd64.deb
pool/main/libx/libxi/libxi6_1.8.2-1_amd64.deb
pool/main/libx/libxfixes/libxfixes3_6.0.0-2+b4_amd64.deb
pool/main/c/cairo/libcairo-gobject2_1.18.4-1+b1_amd64.deb
pool/main/p/pango1.0/libpangoft2-1.0-0_1.56.3-1_amd64.deb
pool/main/p/pango1.0/libpangocairo-1.0-0_1.56.3-1_amd64.deb
pool/main/w/wayland/libwayland-cursor0_1.23.1-3_amd64.deb
pool/main/w/wayland/libwayland-egl1_1.23.1-3_amd64.deb
pool/main/c/colord/libcolord2_1.4.7-3_amd64.deb
pool/main/g/gtk+3.0/libgtk-3-common_3.24.49-3_all.deb
Shell cwd was reset to /workspace/firefox

--- turn 125 ---
[agent→tool] Bash
{
  "command": "cd /tmp/gtklibs && mkdir -p root && for f in \\\npool/main/g/gtk+3.0/libgtk-3-0t64_3.24.49-3_amd64.deb \\\npool/main/a/at-spi2-core/libatk1.0-0t64_2.56.2-1+deb13u1_amd64.deb \\\npool/main/a/at-spi2-core/libatk-bridge2.0-0t64_2.56.2-1+deb13u1_amd64.deb \\\npool/main/libx/libxcomposite/libxcomposite1_0.4.6-1_amd64.deb \\\npool/main/libx/libxdamage/libxdamage1_1.1.6-1+b2_amd64.deb \\\npool/main/libe/libepoxy/libepoxy0_1.5.10-2_amd64.deb \\\npool/main/libc/libcloudproviders/libcloudproviders0_0.3.6-2_amd64.deb \\\npool/main/libx/libxkbcommon/libxkbcommon0_1.7.0-2_amd64.deb \\\npool/main/libx/libxinerama/libxinerama1_1.1.4-3+b4_amd64.deb \\\npool/main/libx/libxrandr/libxrandr2_1.5.4-1+b3_amd64.deb \\\npool/main/libx/libxcursor/libxcursor1_1.2.3-1_amd64.deb \\\npool/main/libx/libxi/libxi6_1.8.2-1_amd64.deb \\\npool/main/libx/libxfixes/libxfixes3_6.0.0-2+b4_amd64.deb \\\npool/main/c/cairo/libcairo-gobject2_1.18.4-1+b1_amd64.deb \\\npool/main/p/pango1.0/libpangoft2-1.0-0_1.56.3-1_amd64.deb \\\npool/main/p/pango1.0/libpangocairo-1.0-0_1.56.3-1_amd64.deb \\\npool/main/w/wayland/libwayland-cursor0_1.23.1-3_amd64.deb \\\npool/main/w/wayland/libwayland-egl1_1.23.1-3_amd64.deb \\\npool/main/c/colord/libcolord2_1.4.7-3_amd64.deb \\\npool/main/g/gtk+3.0/libgtk-3-common_3.24.49-3_all.deb ; do\n  n=$(basename $f); curl -sfO \"https://deb.debian.org/debian/$f\" || echo \"FAILED $f\"; dpkg-deb -x \"$n\" root 2>/dev/null || { ar x \"$n\" && tar xf data.tar.* -C root; }\ndone; find root -name \"*.so*\" | head -40",
  "description": "Download and extract gtk runtime libs",
  "timeout": 300000
}
  [tool←ok]
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules/im-am-et.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules/im-broadway.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules/im-cedilla.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules/im-cyrillic-translit.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules/im-inuktitut.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules/im-ipa.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules/im-multipress.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules/im-thai.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules/im-ti-er.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules/im-ti-et.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules/im-viqr.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules/im-wayland.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/immodules/im-xim.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/printbackends/libprintbackend-cups.so
root/usr/lib/x86_64-linux-gnu/gtk-3.0/3.0.0/printbackends/libprintbackend-file.so
root/usr/lib/x86_64-linux-gnu/libgdk-3.so.0.2417.32
root/usr/lib/x86_64-linux-gnu/libgtk-3.so.0.2417.32
root/usr/lib/x86_64-linux-gnu/libgdk-3.so.0
root/usr/lib/x86_64-linux-gnu/libgtk-3.so.0
root/usr/lib/x86_64-linux-gnu/libatk-1.0.so.0.25611.1
root/usr/lib/x86_64-linux-gnu/libatk-1.0.so.0
root/usr/lib/x86_64-linux-gnu/libatk-bridge-2.0.so.0.0.0
root/usr/lib/x86_64-linux-gnu/libatk-bridge-2.0.so.0
root/usr/lib/x86_64-linux-gnu/libXcomposite.so.1.0.0
root/usr/lib/x86_64-linux-gnu/libXcomposite.so.1
root/usr/lib/x86_64-linux-gnu/libXdamage.so.1.1.0
root/usr/lib/x86_64-linux-gnu/libXdamage.so.1
root/usr/lib/x86_64-linux-gnu/libepoxy.so.0.0.0
root/usr/lib/x86_64-linux-gnu/libepoxy.so.0
root/usr/lib/x86_64-linux-gnu/libcloudproviders.so.0.3.6
root/usr/lib/x86_64-linux-gnu/libcloudproviders.so.0
root/usr/lib/x86_64-linux-gnu/libxkbcommon.so.0.0.0
root/usr/lib/x86_64-linux-gnu/libxkbcommon.so.0
root/usr/lib/x86_64-linux-gnu/libXinerama.so.1.0.0
root/usr/lib/x86_64-linux-gnu/libXinerama.so.1
root/usr/lib/x86_64-linux-gnu/libXrandr.so.2.2.0
root/usr/lib/x86_64-linux-gnu/libXrandr.so.2
root/usr/lib/x86_64-linux-gnu/libXcursor.so.1.0.2
root/usr/lib/x86_64-linux-gnu/libXcursor.so.1
root/usr/lib/x86_64-linux-gnu/libXi.so.6.1.0
Shell cwd was reset to /workspace/firefox

--- turn 126 ---
[agent→tool] Bash
{
  "command": "cd /workspace/firefox/obj-artifact/dist/bin && LD_LIBRARY_PATH=/tmp/gtklibs/root/usr/lib/x86_64-linux-gnu:. ldd xpcshell 2>&1 | grep \"not found\" | sort -u; echo \"rc done\"",
  "description": "Recheck missing libs with extracted path"
}
  [tool←ok]
	libatspi.so.0 => not found
rc done

--- turn 127 ---
[agent→tool] Bash
{
  "command": "cd /tmp/gtklibs && f=$(python3 -c \"\nimport sys\nfor block in open('Packages',encoding='utf-8',errors='replace').read().split('\\n\\n'):\n    d={}\n    for line in block.split('\\n'):\n        if line.startswith(' ') or ':' not in line: continue\n        k,v=line.split(':',1); d[k]=v.strip()\n    if d.get('Package')=='libatspi2.0-0t64': print(d['Filename']); break\n\") && echo \"$f\" && curl -sfO \"https://deb.debian.org/debian/$f\" && dpkg-deb -x \"$(basename $f)\" root && LD_LIBRARY_PATH=/tmp/gtklibs/root/usr/lib/x86_64-linux-gnu:. ldd xpcshell 2>&1 | grep \"not found\" | sort -u; echo OK",
  "description": "Fetch libatspi and recheck"
}
  [tool←ok]
pool/main/a/at-spi2-core/libatspi2.0-0t64_2.56.2-1+deb13u1_amd64.deb
OK
Shell cwd was reset to /workspace/firefox

--- turn 128 ---
[agent→tool] Bash
{
  "command": "cd /workspace/firefox/obj-artifact/dist/bin && LD_LIBRARY_PATH=/tmp/gtklibs/root/usr/lib/x86_64-linux-gnu:. ldd xpcshell 2>&1 | grep \"not found\" | sort -u; echo \"=== xul ===\"; LD_LIBRARY_PATH=/tmp/gtklibs/root/usr/lib/x86_64-linux-gnu:. ldd libxul.so 2>&1 | grep \"not found\" | sort -u; echo DONE",
  "description": "Verify all libs resolve"
}
  [tool←ok]
=== xul ===
DONE

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the current Debian Python image configuration with Firefox's taskcluster/scripts/tester/test-linux.sh and taskcluster/docker/recipes/ubuntu2404-test-system-setup-base.sh. Use the linked Hackbot run and xpcshell's missing-library output to identify the needed runtime environment; done means Firefox scenarios can run without missing GTK and related shared libraries.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.