microsoft / microsoft/vscode

Reasoning Execution precedence, Trajectory drowning, Non imperative select tool, non imperative file system order

Open
#334,055 0 comments 0 reactions 1 assignee Claimed by @pwang347 View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

- Copilot Chat Extension Version: Latest, insider
- VS Code Version: Insider
- OS Version:
- Feature (e.g. agent/edit/ask mode):
- Selected model (e.g. GPT 4.1, Claude 3.7 Sonnet):
- Logs:

The trajectory precedence for multiple files, seems flawed. It is hard to say what is the exactly precedence that we are getting. I carefully built considering https://code.visualstudio.com/docs/agent-customization/custom-agents and all whetevers the agents found.
If there is some config in specific I'm missing, please let me know.

**If a change results in a config parameter, please tag me back during pull requesting, if it is not much inconvenience, so the automated ontology can rebuild and warn me**

How not to drown? - I run like 10M characters only, but I'm certain how the things should look, hence I'm forwarding you pictures of before I tweaked the fileSystem so you can understand what to expect.

https://github.com/MuraraAllan/codebridge-harness/tree/alphaHarness << If you need an environment with this issue happening, just run this in the copilot non-local.

1 - Meta reasoning output - It is straight forward that, copilot on or off, basically. And in this case : Copilot is ON! and the prompt chain of precedence is heritage from the fileSystem ( Current Main of the github repo. )
Image
2 - Meta reasoning output
Image

Those pictures are before I added the .agents/skills/codebridge-harness - which is currently the chosen precedence harness file.

And it shouldnt, because as we see in Picture 5, I'm attaching files, and even a pre-prompt hook to ensure that I don't have to relay on the decision of which part of the agents topology the models must act from.

Basically they should Reason using the agent file,

Picture 3 - The entry worker. It should be the one picked, because it is the one that is prompted when editing (Picture 4).

Image

Picture 4 - I'm editing one file, but the called file is a SKILL, and not the agent one. ( Rule of precedence Flaw 1.)
Image

Picture 5 - Prompt agents and files have precedence accordingly to the link I provided. But they don't! this is not the output of my meta-agentical topology.

Image

Picture 6 - The actual file that is being loading, which locks out the meta-reasoning :

Image

Picture 7 - ACTUAL meta prompting. First chain precedence ( The chosen agent in the picture 4 selection of edition ) which SHOULD have been choosen insteda of picture 6.

Image

Picture 8 The selection of the incorrect skill ( https://github.com/MuraraAllan/codebridge-harness/blob/alphaHarness/.agents/skills/codebridge-harness/SKILL.md )

Image

Picture 9 - Running the prompt after fixing some wrong files ( Addendum 1 ). Are we almost there? Yes! Ok so, this is loading the user-context-instructions ( Finally I sware it is almost done )

Image

Figure 10 - BUT, this is not the main AGENT still. It is not the .agent.md nor it is the entry-point. The entry point has skept in favor of .agents/skills/SKILL.md (Which is an agent SKILL, not an .agent nor a precedence file ).

Image

Figure 99 - Do we have this stuff ?
Image
Figure 100 - Yes
Image

But we are not there yet! The Next moment shows more drowning.

Figure A : The meta-prompt is clearly stating to USE THIS PROMPT before anything.
There is not as kill, there is the agent itself, the compositional stuff that should proccess it. The thing whose thoughts will derive the next steps, so nothing aside from the first picture is correct.
The first picture depicts what it was before the choices in the chain took out of precedence.

Image

I will resume the submission later on, as this already provides the following mistakes :

Skills are taking precedence over prompt files.
Skills are taking precedence over local agents folder.
Skills are taking precedence over copilot-instructions.

There is indeed some files broken in this branch I suggested you used, which may be helping in confuse even more the context of my report.
So let's keep it straight as to :

IF I select (fig 4) a agent it should or should not take precedence ? Why is that a skills hidden layers, We can see in figure A ( Logs ) that the agent self-choice of not enforcing the prompt skill / agent, resulted in a poor / flawed decision as Figure B depictes, violating the chain of precedence of both :
The manual choice in the IDE and the prompt order itself. Which is, really, scarry.

Figure 1 and 2 clearly state how the models react. I will push the fixes that put it back to work, and then I can add any other necessary material to clarify .

Figure B - the decision towards an skill instead of an agent / sub-agent ( I do have sub agents enabled )
Image

Addemdum 1 - Some changes, files changed so the Figure 10 ALMOST works.

Image

Steps to Reproduce:

1. Create an agent, or use the codebase present at my github plugin.
2. Select the agent, as long as there are two files as mentioned.
3. set the environment to copilot.
4. Select how not to drvn - churces playing on youtube
5. watch the agents drawing

plugin repository if necessary : https://github.com/MuraraAllan/codebridge-harness/tree/alphaHarness

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.