Support for attaching to a parent window
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 9.7k
- Forks
- 565
- PR merge metrics
- No merged PRs in 30d
Description
This is a tracking issue for supporting passing in a `parent` handle when creating a window. This functionality will enable users to use Druid as a child window for platform-native windows *not* created by Druid.
### Progress
See rest of issue for context.
General idea for implementation, might not be 100% accurate. Will update this section if needed.
- Platform-independent
- [ ] Figure out what type to use for the window handle in the user-facing API (not `c_void *`)
- [ ] User-facing API: Add an `Option` field in `WindowDesc`, and create a public function for the user to pass the window handle to. `WindowDesc::build_native` should essentially end up passing the field through to the platform-specific `WindowBuilder`.
- Windows
- [ ] Add an `Option` field in `WindowBuilder`, and create a function for `WindowDesc::build_native` to call to set it.
- [ ] When creating the window (`create_window` in `WindowBuilder::build`), pass the value if the new field is `Some`, otherwise pass `0` if the new field is `None`.
- Mac OS
- [ ] Add an `Option` field in `WindowBuilder`, and create a function for `WindowDesc::build_native` to call to set it.
- [ ] When creating the window (`WindowBuilder::build`), attach to the given parent window if the new field is `Some`, otherwise create a normal root window if the new field is `None`. See Raph's [`vst` branch](https://github.com/xi-editor/druid/tree/vst) for ideas.
- X11
- [ ] Create an entire X11 backend (see #475, at least the "bare minimum" bullet point)
- [ ] Add an `Option` field in `WindowBuilder`, and create a function for `WindowDesc::build_native` to call to set it.
- [ ] When creating the window, attach to the given window if the new field is `Some`, otherwise create a normal root window if the new field is `None`. See [this code](https://github.com/crsaracco/vst2-gui-prototypes/blob/master/linux-opengl-vst/src/editor/window/mod.rs#L112) for ideas.
- GTK
- [ ] (Is there an equivalent for GTK? Implement if so.)
### Background / Motivation
This functionality has been desired by many in the [RustAudio](https://github.com/RustAudio) group for a while now: it is a requirement for creating GUIs in the [VST](https://en.wikipedia.org/wiki/Virtual_Studio_Technology) plugin format, one of the most popular plugin formats in modern Digital Audio Workstations (DAWs). Currently, no GUI crate in the Rust ecosystem fully meets all of the requirements to create GUIs for VST plugins, so this will be a step towards making Druid the first one.
*(A "plugin", here, is basically a third-party musical "device" that the user can load into the DAW to synthesize or alter sounds. Plugins typically have a GUI that enables the user to change parameters for this device so that they can tweak the sounds/effects as they see fit, among other features.)*
In DAWs following the VST specification, the DAW is responsible for handling the lifecycle of all plugin GUI windows: creation, opening, closing, etc. When the user loads a plugin into the DAW, the DAW probes the plugin to figure out if it supports having a GUI window. If so, the DAW creates the window for the plugin, then passes a handle to that window (the "parent window handle") to the plugin so that the plugin can attach to that window.
This parent handle is always a handle to a platform-native window (`HWND` on Windows, `NSView` on Mac OS, or an X11 window ID on X11 systems), which can be used in the platform-native window creation function to "attach" to the parent window. (examples: [Windows](https://github.com/crsaracco/vst2-gui-prototypes/blob/master/windows-opengl-vst/src/editor/window/win32_window.rs#L23-L36), [Mac OS](https://github.com/rust-dsp/rtb-rs/blob/master/src/platform/cocoa/mod.rs#L48-L79), [Linux/X11](https://github.com/crsaracco/vst2-gui-prototypes/blob/master/linux-opengl-vst/src/editor/window/mod.rs#L112-L114))
Therefore, the ideal workflow from a VST developer's perspective would be:
1. DAW loads the plugin, opens a window, and passes a handle to that window to the plugin.
2. The plugin code takes that parent handle and passes it to Druid via Druid's app creation API; Druid then handles the rest of the window creation/attaching process.
3. The plugin code now has access to a window that it can draw on, using standard Druid API calls. The GUI portion of the plugin now functions as a typical Druid application.
### Discussion on impact to Druid's current project roadmap scope
Before I get too much farther into the weeds here, I should mention: while I would love to see *some* crate within the Rust ecosystem handle our use case, I still consider this to be pretty low priority as far as Druid is concerned. I know there's [a lot of other stuff on your plate for the near-future](https://github.com/xi-editor/druid/issues/434), and I would like to disrupt that roadmap/timeline as little as possible.
I'm not specifically requesting the Druid core developers to implement this feature for us; some of us in the RustAudio group are willing to do the grunt work. That said, we might have some specific design questions as they relate to the architecture/vision of the crate, PRs take time to review, and all code in a codebase has some sort of maintenance cost.
This feature might be the first of a few for implementing VST support. I'm not sure what else will be needed afterwards, but I feel like parent window handle support is a self-contained enough feature to not be *too* disruptive, while allowing us to prove that Druid is a viable option for us, and also allowing us to dig in and identify future requirements.
If you feel like *any* aspect of us implementing VST support is too disruptive, please let us know and we can hold off. :) Similarly, if you feel like some aspect is completely out-of-scope for Druid as a whole, please let us know that as well.
### Implementation details / proposal
Essentially, we have a handle to a parent window, and we want to pass that handle all the way down to `druid-shell`'s platform-specific window creation functions ([Windows](https://github.com/xi-editor/druid/blob/master/druid-shell/src/platform/windows/window.rs#L1044-L1072), [Mac OS](https://github.com/xi-editor/druid/blob/master/druid-shell/src/platform/mac/window.rs#L132-L187)).
In [`vst-rs`](https://github.com/RustAudio/vst-rs), the API currently [gives you a `parent: *mut c_void`](https://github.com/RustAudio/vst-rs/blob/c1e29953a80946c47987180bec8b3c26c494941a/src/editor.rs#L32). I don't find this to be a particularly good public API for use in Druid, so I propose using something like `rust-windowing`'s [`raw-window-handle`](https://github.com/rust-windowing/raw-window-handle) instead to pass the parent handle around. Alternatively, we could make our own thing if `raw-window-handle` doesn't suit our needs, or if we want to reduce our dependencies. Either way, it would be up to the developer to transform this `*mut c_void` (or whatever they have, for other use cases) into the correct handle type.
I think the best place to expose this API to users would probably be in [`WindowDesc`](https://docs.rs/druid/0.4.0/druid/struct.WindowDesc.html). We could make the function something like:
```rust
pub fn parent(self, parent: RawWindowHandle) -> Self
```
If the user does not call this function, Druid should operate as normal (creating its own window instead of attaching to a parent window).
I think the rest of the implementation is basically just plumbing some data around, but let me know if there's some roadblocks I didn't foresee.
**NOTE:** X11 support needed to get VSTs working, see #475 for the tracking issue + context.
### Unresolved issues and questions
1. [DAWs expect to be the main controller of its threads](https://github.com/crsaracco/vst2-gui-research/blob/master/vst2-gui-background.md#events-and-threads); for example, they expect to be able to call something like `window.open()` and have the control given back to the DAW when that function is done. Therefore, handling events inside a runloop, which never gives back control to the DAW, is problematic. It looks like we might be able to get around this by making an `Application` manually instead of [launching an `AppLauncher`](https://github.com/xi-editor/druid/blob/master/druid/src/app.rs#L95-L113), and just not creating that `RunLoop`. Would there be any issue doing it this way? Would events still be handled? (here's where my knowledge of how GUIs work drops off.)
2. I'm not sure that it's possible to do *any* of this with GTK. More research needed (anyone familiar with it to know if this would work?).
Update:
> In GTK + X11 (i.e. not all gtk platforms) there is: GtkSocket, and GtkPlug which implement the XEmbed spec (caveat, not exactly familiar with any of these APIs/specifications).
Not sure if there's an equivalent for GTK on other platforms.
### Miscellaneous resources and links
- The Rust implementation of VST 2.4 that we are specifically targeting is [RustAudio's `vst-rs` crate](https://github.com/RustAudio/vst-rs), but this feature should work with anything that has a similar use case.
- Raph prototyped VST support in druid for the Mac OS platform about a year ago in his [`vst` branch](https://github.com/xi-editor/druid/tree/vst). While a lot has changed in the crate since then, it still might be a good reference (especially for Mac OS implementation specifically).
- I have [created a few prototypes in the past](https://github.com/crsaracco/vst2-gui-prototypes) as a proof-of-concept that platform-native windowing is viable in Rust for VST use cases. This repo might be a useful reference for implementing this feature in Druid. [`rtb-rs`](https://github.com/rust-dsp/rtb-rs) might also be a good reference.
- I (along with some others in RustAudio) have written up a few pages of background research on the VST GUI problem space in [`vst2-gui-research`](https://github.com/crsaracco/vst2-gui-research).
- @wrl's [`rutabaga`](https://github.com/wrl/rutabaga) GUI toolkit (in C) is being used in a [production VST](https://lhiaudio.com/) (which he also made), so it might prove to be a valuable resource as well.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with druid/src/app.rs and the platform-specific window creation entry points in druid-shell/src/platform/windows/window.rs and druid-shell/src/platform/mac/window.rs, then review #475 for X11 scope. The work is complete only when a parent-window handle API is defined and passed through the relevant platform backends, with the unresolved GTK and event-loop questions settled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- desktop
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100