openedx / openedx/openedx-platform

[Flaky test] `openedx/core/djangoapps/user_authn/views/tests/test_logistration.py::LoginAndRegistrationTest::test_login_and_registration_form_ratelimited`

Open
#35,724 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

flaky-test
Dominant language
Python
Stars
8.2k
Forks
4.4k
Avg merge
6d 18h
Merged PRs (30d)
42

Description

This test fails intermittently and has been deleted according to the Flaky Test Process in https://github.com/openedx/edx-platform/pull/32459.

See docs on how to address flaky tests for why this should be fixed and how to go about it.

Failure output:

=================================== FAILURES ===================================
____ LoginAndRegistrationTest.test_login_and_registration_form_ratelimited _____

self = <openedx.core.djangoapps.user_authn.views.tests.test_logistration.LoginAndRegistrationTest testMethod=test_login_and_registration_form_ratelimited>

    def test_login_and_registration_form_ratelimited(self):
        """
        Test that rate limiting for logistration enpoints works as expected.
        """
        login_url = reverse('signin_user')
        for _ in range(5):
            response = self.client.get(login_url)
            assert response.status_code == 200
    
        # then the rate limiter should kick in and give a HttpForbidden response
        response = self.client.get(login_url)
>       assert response.status_code == 429
E       assert 200 == 429
E        +  where 200 = <HttpResponse status_code=200, "text/html; charset=utf-8">.status_code

openedx/core/djangoapps/user_authn/views/tests/test_logistration.py:132: AssertionError

(Previously filed as https://2u-internal.atlassian.net/browse/CR-5847.)

Notes copied from deletion PR:

  • [idea] It is unclear whether it would pass if it were tried one more time, or given a little more sleep, but maybe the test could be written so if the 429 isn't seen when expected, it does a very small sleep and tries one last time. Or, maybe there is a real bug in the rate limiting code and real failures to rate limit might occur at times?
    • If you choose the workaround in place of looking to see if there is a real flaky bug, then you should comment next to the workaround that if it ever fails, we should really look into whether this is a bug that is not self-healing with our security code.
  • [idea] Rerunning the test many times in a tight loop locally may help duplicate?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with openedx/core/djangoapps/user_authn/views/tests/test_logistration.py and the Flaky Test Process linked in the issue. Run the named test repeatedly and review the recorded 200-versus-429 failure, then determine whether the test or rate-limiting behavior needs correction. Done means the rate-limit behavior is reliably verified without intermittent failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, python
Domain
backend, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.