luckyframework / luckyframework/lucky_flow
Issue with Flapping Tests
Nobody has claimed this yet.
- Dominant language
- Crystal
- Stars
- 55
- Forks
- 9
- PR merge metrics
- No merged PRs in 30d
Description
I'm currently leveraging LuckyFlow to test the entirety of https://luckycasts.com
I've never had an issue in my development environment, but in GitHub Actions I continue to get random tests that will fail with content like this:

If I re-run the job, everything is fine and the test passes without any changes.
Here's that spec code:
it "allows admin users to delete topics" do
flow = TopicManagementFlow.new
topic = TopicBox.create
user = UserBox.create &.is_admin(true)
flow.visit Admin::Topics::Edit.with(topic.id), as: user
flow.click("@delete-topic-button")
flow.accept_alert
flow.should have_current_path(Admin::Topics::Index)
flow.el("body", text: topic.label).should_not be_on_page
end
Here's the relevant GitHub workflow definition parts:
name: Lucky App CI
on:
push:
branches:
- main
pull_request:
jobs:
RunSpecs:
runs-on: ubuntu-latest
container:
image: crystallang/crystal:0.35.1
env:
LUCKY_ENV: test
DB_HOST: postgres
services:
postgres:
image: postgres:12-alpine
env:
POSTGRES_PASSWORD: postgres
ports:
- 5432:5432
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v2
- name: Install PostgreSQL client
run: |
apt-get update
apt-get -yqq install libpq-dev postgresql-client
- name: Install browser
run: apt-get -yqq install chromium-browser
- uses: actions/setup-node@v2.1.4
with:
node-version: "12.x"
- name: "Install yarn"
run: npm install -g yarn
- name: Get yarn cache directory path
id: yarn-cache-dir-path
run: echo "::set-output name=dir::$(yarn cache dir)"
- name: Set up Yarn cache
uses: actions/cache@v2
id: yarn-cache
with:
path: |
${{ steps.yarn-cache-dir-path.outputs.dir }}
**/node_modules
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Set up Crystal cache
uses: actions/cache@v2
id: crystal-cache
with:
path: |
~/.cache/crystal
bin/ameba
lib
key: ${{ runner.os }}-crystal-${{ hashFiles('**/shard.lock') }}
restore-keys: |
${{ runner.os }}-crystal-
- name: Install shards
if: steps.crystal-cache.outputs.cache-hit != 'true'
run: shards check || shards install
- name: Install yarn packages
if: steps.yarn-cache.outputs.cache-hit != 'true'
run: yarn install --frozen-lockfile --no-progress
- name: Compiling assets
run: yarn prod
- name: Run tests
run: crystal spec
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided TopicManagementFlow spec and the GitHub Actions workflow, then reproduce the intermittent failure by running crystal spec in the stated container and browser setup. Compare failing and successful runs to isolate the source of the flakiness; done means the cause is understood and the workflow passes reliably without reruns.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- crystal, github-actions
- Domain
- ci-cd, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100