Baritone sometimes gets stuck on opposing ladders

## Some information
Operating system: Windows 10 2004 build 19041.572 (64-bit)
Java version: Oracle HotSpot JDK 17.0.2
Minecraft version: 1.18.2 Fabric
Baritone version: Null version (Meteor client packages Baritone incorrectly; Meteor 0.4.9)
Other mods (if used): N/A

## Exception, error or logs
N/A

## How to reproduce
Try to path down this shaft:


Baritone will get stuck going back and forth and `#stop` will not cancel anything. You have to run `#stop` then break a ladder and then run `#stop` again before Baritone gets stuck, and then it'll be cancelled

## Modified settings
No modified settings

## Final checklist
- [x] I know how to properly use check boxes
- [x] I have included the version of Minecraft I'm running, baritone's version and forge mods (if used).
- [x] I have included logs, exceptions and / or steps to reproduce the issue.
- [x] I have not used any OwO's or UwU's in this issue.