General Plugin / VST Roadmap

One of our biggest pain points at the moment is administrating our repositories. One reason for this is that we have a nebulous idea of what goals we are trying to achieve.

To help us keep on the same page, we need a roadmap. This roadmap will help us decide on what we want to implement before it is actually coded. Because we are a small group, I don’t currently think we should assign due dates or anything similar.

Once this roadmap is finalized, we can create a GitHub project and work from there! In the management topic we outlined a simple method of approving PRs that should work nicely.

Because programming is not my strong suit (especially in the context of the vst crate) I will be taking a more PM-oriented role and making sure that PRs and issues are resolved. That means this roadmap will lack specific details. Please feel free to expand on what is required in the thread below.

To get started, this topic is for the discussion and finalization of our 0.1.0 release of vst and the 0.1.0 release of vst-sys. (We started at 0.0.1 for vst, though IMO versions using semver should start at 0.1.0.

VST Roadmap

  • Split out a vst-sys crate from the current vst crate
  • Use a workspace for this in the current rust-vst repo containing the vst and vst-sys members
  • Separate all abstraction to the vst crate, and have the vst crate depend on vst-sys for everything else.

Questions: What specific steps are needed to split vst into two crates?

General Plugin Libary

The greater discussion for this can be found here. Here’s a TL;DR:

  • The general plugin library will target arbitrary backends
  • Plugins are implemented with traits specific to a backend

For example, imagine the following pseudocode that creates a VST and LV2 plugin.

struct MyPlugin {};

// Basically what the `vst` crate already has
impl Plugin for MyPlugin {
    fn process() { random_data }

// New stuff to target an arbitrary backend
impl Vst2 for MyPlugin {
    fn some_vst2_specific_thing() {}

// New stuff to target an arbitrary backend
impl Lv2 for MyPlugin {
    fn some_lv2_specific_thing() {}

This wrapper would not offer any more abstraction besides creating plugins with universal code. In the future, we may want to investigate yet another crate similar in scope and functionality to JUCE.

While it seems like a lot right now, I think it would be a good idea to develop this “General Plugin” library in tandem with the vst-sys crate, so we can make sure we’re designing the API in a maintainable and ergonomic way.

Question: So, what do we need to know when starting work on that? It seems like a fairly straightforwards project, but there’s a lot I’m not familiar with such as threading, etc.

Near-term roadmap

There are currently two major changes proposed for the crate, and I’m not sure which order they should land in (it probably doesn’t matter what the order is actually):

  1. @askeksa’s thread safety changes
  2. split the current crate into vst-sys and vst

No matter which order we choose, I think these two things are good targets for the next two versions of the crate (0.1.0 and 0.2.0 ?). Here are my thoughts on both:

Thread safety changes

This seems like a pretty easy milestone to me: “just” land PR #65.

In the past there was some contention around if this would break people’s workflows in making VSTs, but I think the vst-sys / vst split will resolve that. I’m thinking the thread safety changes will reside mostly in the vst portion of the crate split, so if a developer decides that they want to take a different route for thread safety, they are free to build their own second layer on top of vst-sys (see below).

In addition to “just” landing #65, we might want to include (formalized) documentation on the change, in the form of a .md file outlining how someone might 1) build a VST from scratch, 2) build a VST given a synth they already have in a separate crate, and 3) migrate their pre-#65 VST to the post-#65 VST format. @askeksa made a pretty good tutorial/example here, for reference.

That’s entirely optional IMO, but nice-to-have. We could also decide to defer that sort of documentation to some tutorials as well.

Split the crate into vst-sys and vst

The basic idea is to have vst-sys be a super simple crate to help you work with the VST 2.4 API, and have vst (or other higher-level crates like the API-independent project) be the thing that actually makes your plugins.

What this would entail, AFAIK (feel free to correct me), is to move all of the simple stuff (like enums, structs, definitions/constants, maybe some of the functions, maybe even some of the traits – all of the stuff that is pretty much defined by the VST 2.4 standard) into the vst-sys crate and delete them from the vst crate, and then have vst depend on vst-sys for all of that simpler stuff.

vst-sys would then be useful for anyone wanting to implement any form of VST 2.4 they want, and vst would turn into one specific implementation of VST plugins in Rust (with its own opinions on how to generate the plugin, how to ensure thread safety, etc).

Ideally, this milestone would actually be 100% transparent to the users of the vst crate, so I guess it technically wouldn’t necessitate bumping the semver number up, but I’d argue for doing it anyway since it’s a large enough backend change.

Also, since @askeksa’s changes are mostly focused around thread safety, I think that means that it should be pretty contained in the vst portion of the split. Thus, I think we should be able to do these two milestones in either order.

This, I’m not too sure about (beyond vague ideas of what it should roughly look like). I wanted to look into this a few weekends ago and try to rough-draft a split myself, but instead I ended up focusing on GUI work. :slight_smile: Maybe we should just hack an idea together to be able to discuss what needs to be changed from there, though.

Next steps

  • After the above two milestones are done, I’d argue that we should push to get the vst-sys crate into some form of 1.0.0 version, so that the crates that depend on it can be fairly sure that the API won’t change and you can start to develop your higher-level crates a little bit more confidently. I’m thinking the 1.0.0 version looks scarier than it actually is (you just need to make sure all of your definitions match what they actually should be from the VST 2.4 API), but I could be wrong.
  • On the other hand, work being done on both vst and the “general plugin library” might give us more confidence that our vst-sys / vst split is sane, so ¯\_(ツ)_/¯.
  • We also probably want to start getting some testing into both vst-sys (wherever that makes sense, being mostly definitions from the VST 2.4 protocol) and vst, as discussed here.

Also, I forgot to mention that we should probably be considering the host side of VST as well during the first two milestones.

I’m not sure the best way to go about doing that, though, considering nobody’s really programming hosts with the crate yet. Maybe we should hash out a proof-of-concept host?

Do we want to keep the vst crate with the ability to generate both plugins and hosts, or do we want to defer host creation to later? If the latter, maybe we could name the upper-level crates vst-plugin and vst-host, or something. There was discussion about doing this previously, but I wasn’t too sure how feasible this would be. Maybe the vst-sys crate makes that point moot though?

Another thing I was thinking of: since we’re considering doing both VST 2.4 and VST 3.* in the API-independent crate, maybe we should name our crates vst2-{sys,plugin,host} and vst3-{sys,plugin,host} to reflect that?

Excellent stuff! Thanks for the detailed run-down.

As for @askeksa’s branch, is there anything preventing us from merging that now? We’ve been pretty slow on approving that stuff (as an understatement) and it looks as though it’s ready besides some niceties like tutorials. If it’s good to go, I’d say just merge ASAP and get the ball rolling.