[mixed debug engines]: how to synchronize access between multiple debug engines

Hello,

I've also posted this question on "Visual Studio Integrate" forum [1],
but this might be the more appropriate forum to discuss and raise this question.
Also, since the VS IDE hang is sporadic (see problem below), I'm concerned
that suggestion solution (launch with /log) might not address/reveal the problem.

Would very much appreciate any help :-).

Problem

I am working on a custom debug engine that debugs applications on some custom proprietary runtime.

My goal is to be able to debug native applications and proprietary code at the same time,

But it appears that getting the SDM to synchronize the paused/running state across all debug engine is a non-trivial task.

I get random VS IDE hangs and I need to kill Visual Studio and start all over again.

Details

This custom debug engine [1] is a fork of MIEngine, and is a gdb and gdbserver based solution

for debugging our applications running inside the proprietary runtime.

Scenario

1. User launches a native (host) application
2. Our VSPackage (MyVSPackage1) intercepts the IDebugLoadCompleteEvent2 event
3. on IDebugEventCallback2.Event callback (by calling AdviseDebugEventCallback).
4. When we receive an IDebugLoadCompleteEvent2, we launch gdb and gdbserver
   (we call LaunchDebugTargets4 by calling ExecuteCmd(MIDebugPackage.MIDebugLaunch))

Now, at that point in time we have 2 debug engines running in parallel.

Whenever a break event happened, e.g. you stopped in a breakpoint in your native C++ application,

VS calls CauseBreak(), and we wish to pause the debuggee on our custom debug engine.

The implementation at MIEngine GitHub repo doesn't wait for acknowledgement

that the debuggee stopped, and it is executed asynchronously.

The problem with that is that the state of our debuggee is not synchronized with Visual Studio,

and it causes lots of errors and issues.

To alleviate that, we force the Main UI thread (I know it's horrible and that's why I'm here)

to wait until we also stop/continue, something like that:

public int CauseBreak()
{

        _debuggedProcess.SDMSyncEvent.Reset();

        _pollThread.RunOperation(() => _debuggedProcess.CmdBreak());

        // Blocking the VS IDE main thread for 10 seconds,
        // as this is the only way to keep the SDM in sync
        // with our debug engine.
        //
        bool stopped = true;
        if (_debuggedProcess.ProcessState != ProcessState.Stopped)
        {
            stopped = _debuggedProcess.SDMSyncEvent.WaitOne(SYNC_EVENTS_TIMEOUT);
        }

}
and the same is done in AD7Engine.cs, Continue().

It fixes the original problem of being out-of-sync with VS, but creates another problem, VS IDE sporadically hangs

while debugging, and it didn't happen before this fix was introduced.

So my questions to you:

1. Is the above the correct way to synchronize between two debug engines?

 Or there is some SDM wizardry to make the debug engines co-exist peacefully?

2. Why would the main UI thread hang? I provided a timeout of 10 seconds,

I would expect the message pump thread or whatever to gracefully recover.

Thanks,
Ofir

[1] https://social.msdn.microsoft.com/Forums/sqlserver/en-US/b7ddffc7-5baa-4d3d-a013-dee18139e763/mixed-debug-engines-how-to-synchronize-access-between-multiple-debug-engines?forum=vsx

Start with AD7Engine.cs and the CauseBreak() and Continue() paths described in the issue, then trace the MIEngine LaunchDebugTargets4 and ExecuteCmd entry points. Reproduce the mixed native and proprietary-engine scenario and inspect the asynchronous stop/continue behavior. Done means the synchronization approach and the cause of the intermittent Visual Studio hang are established.