Baritone not knowing were to go and wandering around randomly

https://ibb.co/H77mB4v

There is an image of the issue, i am loading a schematic and baritone keeps wandering around (to the green blocks u can see it wants to go to) and once it reaches them just turns around as there is nothing wrong with those blocks and then gets stuck in an infinite loop of walking around.

Note i have temporarily excluded the carpets in the map art i am making so i can build the foundations first, but it is clear that baritone isn't working as intended as you can see the translucent blocks to the left in the image (Unfinished parts of the schematic that need to be built) and despite being next to it, decides that it isn't worth its time and goes of to wander around.