Reasoning Execution precedence, Trajectory drowning, Non imperative select tool, non imperative file system order
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.4k
- PR merge metrics
- PR metrics pending
Description
- Copilot Chat Extension Version: Latest, insider
- VS Code Version: Insider
- OS Version:
- Feature (e.g. agent/edit/ask mode):
- Selected model (e.g. GPT 4.1, Claude 3.7 Sonnet):
- Logs:
The trajectory precedence for multiple files, seems flawed. It is hard to say what is the exactly precedence that we are getting. I carefully built considering https://code.visualstudio.com/docs/agent-customization/custom-agents and all whetevers the agents found.
If there is some config in specific I'm missing, please let me know.
**If a change results in a config parameter, please tag me back during pull requesting, if it is not much inconvenience, so the automated ontology can rebuild and warn me**
How not to drown? - I run like 10M characters only, but I'm certain how the things should look, hence I'm forwarding you pictures of before I tweaked the fileSystem so you can understand what to expect.
https://github.com/MuraraAllan/codebridge-harness/tree/alphaHarness << If you need an environment with this issue happening, just run this in the copilot non-local.
1 - Meta reasoning output - It is straight forward that, copilot on or off, basically. And in this case : Copilot is ON! and the prompt chain of precedence is heritage from the fileSystem ( Current Main of the github repo. )
2 - Meta reasoning output
Those pictures are before I added the .agents/skills/codebridge-harness - which is currently the chosen precedence harness file.
And it shouldnt, because as we see in Picture 5, I'm attaching files, and even a pre-prompt hook to ensure that I don't have to relay on the decision of which part of the agents topology the models must act from.
Basically they should Reason using the agent file,
Picture 3 - The entry worker. It should be the one picked, because it is the one that is prompted when editing (Picture 4).
Picture 4 - I'm editing one file, but the called file is a SKILL, and not the agent one. ( Rule of precedence Flaw 1.)
Picture 5 - Prompt agents and files have precedence accordingly to the link I provided. But they don't! this is not the output of my meta-agentical topology.
Picture 6 - The actual file that is being loading, which locks out the meta-reasoning :
Picture 7 - ACTUAL meta prompting. First chain precedence ( The chosen agent in the picture 4 selection of edition ) which SHOULD have been choosen insteda of picture 6.
Picture 8 The selection of the incorrect skill ( https://github.com/MuraraAllan/codebridge-harness/blob/alphaHarness/.agents/skills/codebridge-harness/SKILL.md )
Picture 9 - Running the prompt after fixing some wrong files ( Addendum 1 ). Are we almost there? Yes! Ok so, this is loading the user-context-instructions ( Finally I sware it is almost done )
Figure 10 - BUT, this is not the main AGENT still. It is not the .agent.md nor it is the entry-point. The entry point has skept in favor of .agents/skills/SKILL.md (Which is an agent SKILL, not an .agent nor a precedence file ).
Figure 99 - Do we have this stuff ?
Figure 100 - Yes
But we are not there yet! The Next moment shows more drowning.
Figure A : The meta-prompt is clearly stating to USE THIS PROMPT before anything.
There is not as kill, there is the agent itself, the compositional stuff that should proccess it. The thing whose thoughts will derive the next steps, so nothing aside from the first picture is correct.
The first picture depicts what it was before the choices in the chain took out of precedence.
I will resume the submission later on, as this already provides the following mistakes :
Skills are taking precedence over prompt files.
Skills are taking precedence over local agents folder.
Skills are taking precedence over copilot-instructions.
There is indeed some files broken in this branch I suggested you used, which may be helping in confuse even more the context of my report.
So let's keep it straight as to :
IF I select (fig 4) a agent it should or should not take precedence ? Why is that a skills hidden layers, We can see in figure A ( Logs ) that the agent self-choice of not enforcing the prompt skill / agent, resulted in a poor / flawed decision as Figure B depictes, violating the chain of precedence of both :
The manual choice in the IDE and the prompt order itself. Which is, really, scarry.
Figure 1 and 2 clearly state how the models react. I will push the fixes that put it back to work, and then I can add any other necessary material to clarify .
Figure B - the decision towards an skill instead of an agent / sub-agent ( I do have sub agents enabled )
Addemdum 1 - Some changes, files changed so the Figure 10 ALMOST works.
Steps to Reproduce:
1. Create an agent, or use the codebase present at my github plugin.
2. Select the agent, as long as there are two files as mentioned.
3. set the environment to copilot.
4. Select how not to drvn - churces playing on youtube
5. watch the agents drawing
plugin repository if necessary : https://github.com/MuraraAllan/codebridge-harness/tree/alphaHarness
Contributor guide
Assessment
This issue has not been assessed yet.