Rust Audio

Giving an overview about LV2

Since most people in this forum don’t seem to too familiar with LV2, I will first write some words about
LV2 itself. This is only a rough summary of how LV2 is built; If you want to have more information,
you should check out the official LV2 book or my effort to translate
it to Rust
.

First of all, to get things straight: LV2 is platform-agnostic. There is almost nothing in the specification that limits the platforms it can run on; The “almost” will be covered later.

Next, what is LV2? Technically speaking, LV2 is a set of RDF vocabularies together with some C-Headers. These vocabularies can be used to express information about plugins and the headers standardize the way hosts can interact with plugins.

The core specification is relatively thin: It handles plugin detection, names, translations, Audio with f32 frames, parameters of type f32 and hooks to pass optional features into the plugin and function pointers out for extensions. LV2’s genral theme of expressing information is to put as much information in the configuration files and as little information in the shared object.

The next specification of note is Atom, basically LV2’s own type system. All types of data can be expressed with it, like integers, floats, vectors (comparable to Rust slices), tuples (different atoms stuck together), even objects! MIDI messages are just a different atom type, but due to historical reasons, it has its own specification. Implementing these specifications was quite hard since I had to take a deep dive into unsafe Rust and it’s type system, but it was definitely worth it! This is where I pick up that “almost”: My implementation relies on the fact that all types can be aligned to 64-bit without the need for padding. That’s it!

I haven’t looked too much into the UI specification, but from what I’ve seen from it and what I’ve heard about VST’s GUI system, it must be heaven: First of all, UIs are separate objects from plugins and the specification suggests to keep them in different shared objects. Then, a UI has to specify which UI framework it supports, the core specification gives you a choice of multiple versions of GTK and QT, as well as X11, Cocoa and Windows (+1 for platform support!) When the host opens a UI, it creates a window for the supported framework and hands it to the UI. From there, it’s framework-specific and not something LV2 cares about.

These are the most interesting parts of LV2! I glossed over a lot, like the worker or state specifications, or the whole URID thing, but that’s not important to have a basic understanding about LV2.

3 Likes

Yep. VST2 hosts create the window “for” you, and you have to attach to it given a “parent” pointer to that window. That’s issue #1 for Rust crates, since most don’t provide that. Although I’m interested in this:

Qt and GTK support are the killer features, it would seem: VST2 seems like it supports all platforms in theory, but you’re stuck with the baseline platform-specific window system (X11, Cocoa, Windows, maybe even Wayland?, etc), so if you want a cross-platform plugin, you have to support each platform yourself. I’m doing some work on my weekends to help kick that off :slight_smile: I’ll have a writeup on my design thoughts pretty soon.

However that does make me wonder: how do you use the (Qt, GTK) window that the host gives you in LV2? I guess it’s created for you, and then it’s moved to the GUI thread entirely so you have full ownership of it? If so, I think it would be significantly easier to give it an OpenGL context and handle events. :slight_smile:

I actually didn’t know. :grinning: I looked in the sampler example in the book and it looks like I gave you wrong information: It looks like the plugin creates a GTK widget and passes a pointer to that widget to the host (via return-by-reference). You can see that in the instantiate function in the sampler_ui.c block. I’m really sorry about that!

Anyways. This would implies that the plugin owns the widgets. Another good question: Which lifetime should we enforce on the widget? 'static? The lifetime of the UI? As you can see, I haven’t looked at the UI part in such detail! :grin:

Unfortunately, GTK/Qt support requires a pretty serious hack with (significant special casing) to reliably get working. For better or worse, the best option is to keep depending only on the baseline OS/window-system is still the best approach.

It’s, uh… its own special setup. To speak in LV2 terms, VST only supports CocoaUI, WindowsUI, and X11UI, and which to use is implicitly specified based on which platform the plugin is running on.

LV2 seems to work a bit like AUv2 during instantiation. Namely, rather than the host passing in a pointer to the parent window (though in LV2 I understand this is an optional method), the plugin creates the widget and returns a pointer to the host.

Are you viewing this from a plugin’s side or a host’s side? When you’re writing a host and want to support all of these frameworks - pfew, that’s a hard task indeed! However, the way understand it is that the plugin provides multiple UIs for different frameworks and the host opens the one it supports. Also, Qt5 is quite platform-independent, isn’t it? One could, theoretically, go ahead and only support Qt5. Then you would technically limit yourself to certain hosts/plugins, what you are always doing, and not to certain platforms. Anyways, I see your problem: Supporting all frameworks is complicated!

Sorry if I got that wrong! All I’ve heard is that you have to create an OpenGL context, which works differently on every platform, and from there, you have to build a UI framework on your own. This sounded quite nasty to me and therefore, LV2’s approach looked way better! Or to be precise: It looked better for someone who wants to write a library for plugin creation, because it means less work! :smile:

That… seems unrealistic. UI development is time-consuming enough without having to provide the same experience using several different toolkits (for example, a plugin providing both Qt5 and GTK interfaces doesn’t seem realistic to me).

The issue is mostly how the event loops mesh and whether the toolkits have global state. I don’t know enough about Qt5 to know if it’s possible to build cross-platform plugin UIs with it. I’ve heard people in the industry talk about having heard about plugin UIs developed with Qt, but I’ve never talked with a plugin dev who’s actually shipped one.

I have absolutely talked to a plugin dev who ships a standalone app using Qt but then uses a bespoke in-house toolkit for the plugin version to sidestep linkage/globals issues.

It sounds like LV2 might let the plugin create the entire window, not need you to attach to something the host made? In which case, LV2 might be a little (a lot) more lenient about event loops, etc.

From a very quick glance, it does look like LV2 has explicit support for Qt and GTK, so I assume someone had to have created at least a mini example while developing it originally. Never know, though.

This is indeed not the case: the plugins ship their UI with the toolkit they want, and it’s the host’s job to handle it. The above-linked suil library (made by one of the maintainers of LV2, and used by Ardour for instance) is to be used by hosts to embed pretty much anything into their own toolkit (mostly using X11’s XEmbed protocol spec).

It is to note that the host may also chose to not load a plugin’s UI as a fallback, either because it does not support the plugin’s toolkit (which may be something else than what the LV2 UI spec lists), or by user’s choice (some plugins’ UIs are so bad I just prefer just having three sliders).

The host is also free to generate its own UI instead of the plugin’s one, using what the plugin describes for its ports and controls.
For instance, here are UIs for Calf’s Delay plugin, the left one is from the plugin, and the right one is generated by Ardour.

LV2 plugins actually only create a widget/container, not a full window (although the spec does indicate that a “UI” may not be a widget, but pretty much anything you like. After all, as far as LV2 is concerned, a UI is just a pointer with an URI to identify what it is, though most hosts expect it to be something embeddable).

For instance, you can notice in my screenshots that the host adds a few buttons on top, even when using the plugin’s UI.

The LV2 UI spec does, however, provide the guarantee that the plugin’s UI is ran in the same thread as the host’s UI, and that the framework (and its event loop) is initialized before the plugin’s UI is.
In that sense, instantiating and LV2 plugin’s UI is nothing more than a glorified function call that returns the pre-configured widget to be added to the rest of the window, which is a very common UI building pattern that literally every UI framework in existence supports.

Now for framework usage in plugins, a quick ripgrep in my plugins folder showed me that Qt is used by some plugins, though not that much. (synthv1 is an example). The majority uses GTK2 or GTK3 (the Calf plugin suite uses GTK2 for instance), and a few others (like ZynAddSubFX/Zyn-Fusion create an X11 frame and use it to put an OpenGL context for rendering.
This is biased, being my own plugin library of course, but it shows there is quite a bit of variety in the toolkits and methods used.

This is indeed not the case: the plugins ship their UI with the toolkit they want, and it’s the host’s job to handle it.

Wait… so if I write an LV2 host I need to be able to handle whatever framework any plugin wants, or fallback to default UI? Plugins can’t just be completely independent of the host?

So whatever 3rd party code is running knows it’s running in the same address space as both my host and any other plugin loaded by the host?

What kind of encapsulation/sandboxing strategies do hosts use for LV2? Do they just not care?

If, as a host, you want to support the majority of LV2 plugins out there, then you need to handle embedding GTK and Qt. (Which is not that bad to implement, considering they both support embedding and getting embedded into X11 through XEmbed)

They can be independent somewhat, if they only target the “native” platform UIs (X11, Cocoa, Windows APIs).
While using (all of!) these directly is so low level it is pretty much unthinkable for most plugin authors, I think however this is the right fit for Rust, since we are quite used to build our own abstractions on top of those low level primitives.

I messed up my explanation here, sorry about that. ^^’
Plugin UIs don’t have to run in the same address space as the host’s UIs nor as the “processing” address space, their only expectation is to be ran in the same thread (and address space) as the main loop of their chosen toolkit, but it doesn’t have to be the one the host uses for its own UIs, even though some hosts implement it in the same thread as the UI one, namely DAWs like Ardour because they (rightfully) don’t care, considering running on a typical Linux/X11 userland means having access to everything anyways.
Although we could probably implement a sandboxed host by leveraging stuff like SELinux/AppArmor (at least on Linux, dunno about the other OSes) to separate audio processing and UI. But that’s quite a stretch for now.

The LV2 UI spec explicitly specifies that UIs may be ran in a separate process, or even on a separate machine than the one processing the audio, hence the need for serialization and IPC/RPC (although LV2 pretty much leaves how RPC is implemented to the host, the UI just has to send messages to the plugin).

Building cross-platform plugin UIs is one of the features Qt5 is sold for! (source) You almost can’t get more cross-platformy than Qt :wink:

To me, it looks like both of you are right: When you look at, for example, GTK+, you see that the specification demands that the widget is a GtkWidget. And when you then look up the object hierarchy of GTK, you see that a Window is just a widget! The same is true for QMainWindow, which is just a QWidget. LV2 doesn’t care what kind of widget you’re creating and therefore, you can also straight-up create a window. However, I don’t know if that plays well with most host, but technically, the hosts are interpreting the standard wrong, not the plugins.

Concluding from all these host-related points, it looks like creating LV2-compatible plugins is rather easy, but creating LV2-compatible host is a very serious task! We should definitely not mix these to projects up!

Sorry, looks like I got that wrong!

It’s true that we should support the “native” platforms, just because we can, but I don’t think that we should exclusively stick to them. The thing is that creating proper UI frameworks is a serious task and there is a reason why there are only a handful of good ones out there. If we wouldn’t support Qt and GTK and wait for a good, rusty UI framework to arise, nobody will use the rest of our crate for a very long time since plugins without a proper UI are almost (at least for musicians) useless. That would be a shame!

1 Like