Rust Audio

Design an API-Independent plugin interface in Rust

Howdy y’all. A few of us have voiced interest in developing a rust-based wrapper that can target any plugin type as a back-end. I’m here to get the ball rolling on the discussion in the forum so that others can give their input.

Things to consider:

  • Thread safety
  • Avoiding the ‘lowest-common-denominator’ while trying to satisfy all capabilities across the different plugin types.
  • Modeling off of previous attempts at open standards
  • Setting up a bibliography of resources and learning materials
  • Avoiding the pitfalls of a ‘design-by-committee’ hodgepodge (I’m looking at you, C++)

I’m looking forward to contributing a lot to this effort, but like most of us, I’m busy with work/school/life. If we can pace ourselves and make sure not to get burnt out I think we could come up with something nice!

2 Likes

I would like to have this sort of thing, but I also don’t want to lose the VST-specific crate. Would this just be a sort of wrapper on top of rust-vst et. al.?

For posterity and future readers/contributers, here’s a list of things to care about:

APIs people care about

APIs people maybe care about

APIs for inspiration


Reference materials:

1 Like

I’m going to be a little blunt about this. Even though I’m very unhappy with Steinberg about how they’ve handled VST2, VST2 is nothing special and we don’t lose much if we deprecate it. Honestly, it’d be better to do so sooner rather than later and move to something API-independent. We certainly shouldn’t be championing it as our standard API.

Steinberg has shown their colours as an unreliable steward of their technologies, and I don’t see them backpedalling on VST2.

2 Likes

Agree’d on VST2 not being the future, but it might be a good idea to support it for the folks who have signed the agreement.

It would be cool to see if an open plugin API could be extracted from the project similar to LV2, or maybe taking LV2 and making it cross-platform (I know it’s an idealistic pie-in-the-sky dream but it’s something I’m keeping in the back of my mind).

One thing that I meant to mention was the issue of licensing and making sure the project has a good license. Also, we need to make sure we’re respecting the terms of any of the plugin-vendor’s licenses. I’m sure we could do something similar to JUCE or Dplug in that regard.

With regards to these APIs, I’m going to rearrange these a little bit.

Pro Audio APIs

I think it makes sense to approach these from a cost/benefit standpoint. As such, I’ve reordered them roughly in order of the benefit we get from implementation.

  • VST2
    Widest market penetration. good place to start.
    Should assume bringing a product to release will be fraught with issues in light of the Steinberg licensing situation.

  • AUv2
    not going anywhere, covers most of the remaining ground that VST2 leaves exposed.
    docs/headers are in the AudioToolbox framework of the MacOS SDK (non-redistributable but easy enough to find online).
    I’m currently porting my product to AUv2 and can handle implementing a backend for the resulting Rust framework.

  • VST3
    We’ll need to bite the bullet on this at some juncture. an acquaintance of mine implemented VST3 support in his home-rolled framework in a little under a week, so may not be a huge issue.

  • LV2
    In my opinion, this is important from a philosophical point of view. LV2 is the only standard on this list which is open and not controlled by a company (save for LADSPA, though LV2 is literally “LADSPA V2”). I personally would prioritise this above VST3, but that’s mostly because I’m angry at Steinberg.

  • AAX
    Important, but the licensing/NDA situation is nothing I really want to deal with. I think it should be a “someday” priority unless somebody comes onboard who needs it bad for a project.

  • AUv3
    Works on iOS. People struggle to get plugins working on Mac. AUv2 isn’t going anywhere, so until folks want to deploy on iOS I think we should backburner this.

Unity and wWise seem interesting but I don’t think they should be high priority. That said, looking at JUCE’s Unity code, seems straightforward enough.

LADSPA is indeed obsolete at this point. No MIDI input, no custom editors, no extension mechanism.

3 Likes

I guess I should have worded this differently.

I personally want to be able to use some underlying crate (in this case, the VST2 crate we’ve already built up), if for some reason I decide that I want more control than the “lowest common denominator” API is giving me, if that happens to be an issue.

On top of that, I’d rather only deal with VST2 development in one repo, rather than have to update two separate ones.

Yeah this is a good point. Ideally we’d be able to split up the VST crate into a vst-sys crate and a higher-level one that would eventually be replaced by the API-Independent Trait library

FYI here is the GMPI final draft

1 Like

Yeah I think this is the right strategy. I certainly wasn’t suggesting that we implement the higher-level API-independent layer on top of the Plugin trait that already exists – we’d be implementing that on top of the raw vst-sys constants and such.

1 Like

I agree that promoting an open standard would be a good hedging strategy in case any company wants to get nasty with their IP à la Steinberg.

In that case, agree with everything except “replacing” the higher-level rust-vst crate. It would just be another option, IMO.

It’s just going to bit rot. Why not deprecate it?

If we get the same featureset with a backend-agnostic library that builds on top of vst-sys, maintaining both that new library and a VST specific library would be difficult to maintain (and probably pretty redundant) IMO.

What exactly is in the vst crate now that would be a candidate to remove from a -sys crate, and move into an agnostic library?

This is a two-parter. First, “what’s in the vst create now that would be a candidate to remove from a -sys crate” is, in my opinion, anything beyond the pluginterface ABI/API. So, Plugin trait, helper methods – all of that, no place in the -sys crate.

The second part, “what can be moved into an agnostic library” is the trickier one. It really depends on what API surface we decide is appropriate. It will in all likelihood resemble the VST crate but won’t match it 1:1. For example, the way that VST2 models parameters is pretty raw, and the higher-level crate should have something higher-level. So, at the very least, get_parameter and set_parameter would need to take different signatures, and it’s likely the internal logic would be different as well.

Likewise for process_events(). Events in other frameworks are delivered in different ways (AUv2, for example, sends one event per function call, and the plugin buffers internally and then iterates along the buffer in process()). Even in VST2, the events are timestamped relative to the next process() call, and process_events() shouldn’t do much more than buffer them.

These are just a couple of examples which are API-specific and really just cause problems when the plugins have to handle them.

1 Like

Alright, that makes sense.

If we go in that direction, we could execute in a few different ways from a management perspective:

  1. repurpose the vst repo to vst-sys, and then create a new repo for our “Generic” plugin code
  2. Make vst read-only after creating two new crates (vst-sys and our-cool-plugin-thing)

#1 has the advantage of keeping our existing social metrics like stars & visibility.
#2 has the advantage of possibly being more clear to the end-user.

It shouldn’t matter too much, though. And this is of course assuming we end up going down this road.

edit:

#3 - we could also split the rust-vst repo into two workspaces - vst and vst-sys. We could deprecate vst in favor of whatever solution we end up with, and use the vst-sys part of the workspace for active development.

Arguments for AAX/Wwise/Unity:

  • ProTools is still the industry standard, plugins need to target AAX to support it. Bigger deal for effects and mixing/mastering than instruments (and I think a lot of us edge towards synth dev)
  • Some of the other rust-audio business is creeping on Amethyst, it would be nice to keep game devs in our thoughts when developing a plugin architecture. There could be a good early adopter userbase there.

Arguments against:

  • There’s enough trouble making a sane wrapper around just AU and VST, let alone the game engine APIs.
  • Licensing/DRM situation is not ideal
  • AAX is about as closed as a 3rd party extension SDK can be, and Avid killed off TDM and RTAS like a race horse with a broken leg. If you’re troubled by long term API stability/viability, AAX might be concerning.
1 Like

I was thinking this as well. Creating a target for an audio system used in the rust gamedev scene is a really cool idea that I imagine would help both our ecosystems.

As for AAX, I think - while it is nice - that the cons outweigh the pros in terms of Licensing/DRM stuff. It’s an achievable long term goal, but as of today I don’t know if it should be prioritized in any way. I haven’t checked unity, but if they require licensing it’s out of scope IMO for now

Agreed. Though I think it shouldn’t be our highest priority.

There are definitely benefits to AAX, and my prioritisation of it above is mostly around the (unfounded, mind you) assumption that there will initially be very little overlap of “people developing audio plugins in Rust” and “people who use Pro-Tools”. Once the boulder is rolling a bit, we should definitely make AAX a priority, but I’d put it off to post-critical-mass for the time being.

1 Like

It’s more “people who buy plugins use pro tools” than “people who write Rust use pro tools”

2 Likes