codecombat / codecombat/codecombat

Should same code submissions yield same results?

Open
#4,747 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
8.6k
Forks
4.2k
Avg merge
1d 4h
Merged PRs (30d)
39

Description

Should same code submissions by same people yield same game results?

Currently, I'm observing a strange phenomena. This is my loss in ladder:
this_is_my_loss2

But I'm winning against the very same opponent!
submission2

Both games happened after my last submission, and his:
submitted_long_ago2

In last 200 games that were automatically played, my submission won 4 games out of 5 against this opponent. (Won't attach a super-lenghty screenshot, but it's true.)

Here's a link I constructed to watch the game between these two submissions: https://codecombat.com/play/spectate/zero-sum?session-one=5849db3f457d1e24009c0db8&session-two=5515a210d792f354050c9339 Sometimes it says that an infinite loop is detected. Another time it loads whole game through to the end (if you re-load the page). The loss happened about 3 times in the last 3 days.

I understand that, between all the problems that codecombat does or can experience, some tiny glitch may not be the biggest thing happening. But maybe this might hint to some underlying bug that would be useful to fix and that could affect more players? There's definitely something non-deterministic going on, and it subjectively feels that my submission is relatively predictable and should not lose at all (it does in fact win against any other opponent now, and wins this in most cases too).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by replaying the two submissions through the linked spectate sessions and checking whether reloads change the outcome or trigger the reported infinite-loop message. Compare repeated games involving the same submissions; the issue is resolved when their results are deterministic and the inconsistent execution behavior no longer occurs.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
game-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.