Rust Audio

The ultimate audio software?

Do not concider my self a developer yet. Atleast not a Rust developer. I made some stuff in SynthMaker and FlowStone: Here you find my VSTs. I have a big idea and no clue if I ever are going to be able to realize it :slight_smile:

What I have in mind is multiplatform software with modular libraries that is written from ground up with Rust. Audio editor, DAW, Live audio mixer with plugins, graphical plugin deveopment for all people and more all in one. Because from what I have learned the strengths of Rust suites this very good. I do not want to take use of existing plugin formats etc. But first of all I need to learn and get Rust under my skin.

So for now: Learn Rust and keep thinking abot how the stucture og the whole thing could look like and ofcourse find a solution GUI and IOs to work cross platform. A modular approach is key I think. Write as little code as possible to do as much as possible in an inuitive and logical way. I guess Rust Audio is the right place to start.

All ideas and poiners in the right direction are very welcome :slight_smile:

It’s a very ambitious goal. I would suggest, look into existing open source DAWs and
get a feeling what it is about. Most prominent examples are:
LMMS (C++ with Qt), Ardour (C++ with Gtk) and the relatively new Zrythm (C with Gtk?).

The main features a DAW should bring to the table are:

  • A mixer with audio routing options
  • Plugin host (VST2, VST3, LV2, LADSPA, …)
  • Parameter Automation of plugins and of all (important) knobs inside the DAW
  • Audio recording
  • Timeline sequencing (laying out the Song structure, automation, …)
  • MIDI recording and sequencing (piano roll, plugging in hardware synthesizers, …)

There are more advanced things you would want from a DAW:

  • Time stretch
  • (Clip based) live performances
  • Track bouncing
  • Integrated or bundled plugins and sound library:
    • Synthesizer
    • Drum-Synthesizer
    • Sampler
    • Audio FX: Equalizer, Reverb, Compressor, Limiter, …

The current state of open source audio software has progressed a lot
in the past 10 years, but it has taken quite a lot of work of so many developers
to get to the current (C/C++) ecosystem.

I would start out with writing a few plugins in Rust, using either rust-lv2 or rust-vst.
To test plugins I found Carla (a plugin host) to be quite helpful.

And use them in existing DAWs. You can write things like a piano roll sequencer
as plugin and use it from other DAWs. Or even an audio timeline with placeable samples
could be put into a plugin.

2 Likes

We have a very similar dream :wink:

1 Like

Thanks :slight_smile:

I must say Zrythm looks very promizing in some way. Seems solid and the graphics is quite nice in some way. He says it is intuitive… well, it is not. As with all DAW the UI is very cluttered and busy and little homogenic. I have worked on and played with them all allmost. My filosofy is keep it stupid simple, beautiful and inspiring. That applies to both the GUI and the workflow. I should probably study some open source DAWs to get a feeling on how they do things. But I am dead serious when I say I want the software to be buildt from ground up and a new proprietary plugin format first of all. This is because og the modualar approach I have in mind. Support for others like VST, LV2, LADDSPA etc could come at a later point but it is far from my goal. If Zrythm is made with GTK then it is worth looking into for the GUI. A cross platform GUI solution seem like one of the biggest challenges to just get started.

Yes, Zrythm is still immature. I had my problems with it too. But so I had when I was trying
out Ableton Live or Ardour or even LMMS. Every DAW has it’s ways and you need to figure
things out for a few hours.

You want a new Plugin format? And a proprietary one? Why?

LV2 is a good format, a bit static with the TTL files, but it fits the bill of open plugin APIs.

2 Likes

I’m trying to do something similar, although much smaller in scope. I would say you need a really good grounding in digital signal processing in addition to the programming skills.

1 Like

I myself am new to systems programming. I’ve wanted to make a music tracker like Reaper and Renoise for many years. After months of research I’ve realized that the Rust ecosystem is still too young to support a pure Rust project of this scope. C++ frameworks like JUCE are just so far ahead. Certain parts (like the tracker core) could be written in Rust, but if you want an attractive and modular UI, a C++ frontend would be the way to go, at least right now. It’s a little frustrating, but understandable and not anyone’s fault. The Rust team does things a very specific way so that Rust is a rock-solid language.

Take for example Conrod. Conrod is a hybrid retained-mode/immediate mode GUI framework, and immediate-mode doesn’t work so well for things like VST GUIs.

I would say Radium --> source code is a good place to start studying. For UI it uses Qt, which has at least one Rust library. Zrythm is cool but still quite young, whereas Radium has been in development for years.

This is all very opinionated, and it sounds like you want a full fledged DAW like Ardour whereas I want to build a tracker. Keep your dreams alive!

2 Likes

WeirdConstructor, yes a new pluginformat much like Reason Rack Extensions. I really like the homogenic look and framing. Also take a look at https://gigperformer.com/ You see what i mean? Also interesting is https://audiostrom.com/ Live Professor, but the GUI is a mess. I dont want that mess of floating windows and cluttered space that annoys the eyes. Another point is that Id like the plugins to be built up by sub modules as the rest of the system i have in mind. Except here plugins could also be easily made in app with drag n drop of modules with and without gui (available dsp coding), bit n bolts for GUI and DSP and so. Proprietary or not. It is not the most important part. Of course I have a dream to make some money as I really dont have much of that. But then again easier to get help and collaborate with others to do magic :slight_smile:

scalarwaves, I really see your point. The GUI part seems hard. Will definetly check out Radium. Strange I did not know about it. As I am norwegian and I know about Notam. But have not wisited their website for years. Maybe I can get in touch with the developer and get some help. Makes it easier as we both speak the same language too. I really appreciated that pointer. Yes I want a hybrid DAW, Live mixer, modular type system with integrated development possibilities. Thanks. :slight_smile:

I’m sorry, but I think I have to get some of you down to earth a bit: :confused:

First of all, don’t reinvent the wheel: Have you had the pleasure to work with something professional like Ableton Live yet? This piece of software is in continuous development for over 19 years now and was created by people who know their craft pretty well. And compared to the competition, Ableton Live is rather young: Logic Pro is 27 years old, and Pro Tools and Steinberg Cubase are even 31 years old! All of these DAWs are under constant development and are extremely specialized and optimized, both regarding pure rendering performance and ergonomics. They already excel at everything you might want a DAW to do and the bar for new DAWs is damn high. Reinventing the wheel isn’t going to get you anywhere, you’ll just do what all the other people did in the 2000s. Try to do something new that hasn’t been done (properly) before!

Another thing is that “It’s written in Rust” is not a feature for the user of an application. I love to create libraries and frameworks in Rust, but when the user is only using an application, the language it was written in shouldn’t be important. If a user asks you why they should use your software and your only answer is “It’s written in Rust” you’ve done something wrong. Try to offer something new that hasn’t been done (properly) before, even if it involves not using Rust! However, “It’s Open Source” is a feature because it means that the users are independent of the original authors, but that’s a story for another time.

The last thing is that you seem to underestimate the power of platforms. Why do you think everyone uses VST? Not necessarily because it’s the best technology out there, but because almost all hosts and plugins support it. Building up a new platform without a way to transition to it usually doesn’t work since no one wants to use a platform with nothing on it. Try to work together with other technology to become relevant. Rust-VST and Rust-LV2 wouldn’t have caught on if no host would support them.

Okay, rant over, sorry for that! :sweat_smile: Maybe I should say now what I want to do. You’ve probably figured by now that I consider the DAW business as more or less sorted out. However, one thing that isn’t sorted out to me is a proper translation of conceptually modular synthesizers to software solutions. There are quite a lot of approaches and things I’ve tried out, for example VCV rack, Carla and Reaktor, but they all fail for the same reason: They limit their possibilities by clinging to outdated metaphors.

Take, for example, VCV Rack: Its goal is to exactly replicate modular Eurorack synthesizers. These are basically suitcases filled with modules that can be connected with audio cables and control each other by applying voltages to these cables. This concept is pretty neat, but clinging to firmly to it leads to unnecessary restrictions: For example, modules in VCV Rack can not be folded together, hidden, or abstracted into new modules, because you can’t do that with hardware modules. The connections can also only carry single-voiced audio data, no events, or other formatted data. Another odd detail that really annoys me is that VCV Rack doesn’t integrate with any other software/DAW since it considers itself as a DAW too (Source: https://vcvrack.com/manual/FAQ), which it straight-up isn’t. However, it is extensible via its own plugin standard, which is better than nothing.

Reaktor does a better job: It’s a visual programming environment where you arrange modules in a graph to describe a higher-level synthesizer. Modules can be grouped and abstracted into macros, and you can also build some neat UIs to use your completed synth in another application. It also has so-called “core modules” which use a Petri net instead of a graph and are compiled on-the-fly for some extra performance. Native Instruments use it to build almost all of their synths and I’ve also put it to some good use and created some nice sounds with it, but it’s hard to describe any concepts that aren’t exactly DSP programs. For example, it can only work directly with simple data types like floats or ints, and you can not directly interact with the operating system, create new UI elements, or run tasks in the background. Lastly, it’s a closed system, which means that you cannot extend it with “proper” libraries to work around the issues I’ve described above. All in all, I’m okay to work with Reaktor for now, but it’s far from perfect.

Another piece of software that always stands out to me is Carla. It describes itself as a “fully-featured modular audio plugin host” (Source: https://kx.studio/Applications:Carla), which fits quite nicely: It supports many plugin standards and lets you arrange instances of these plugin in a graph, a bit like Reaktor. It’s also available as an LV2 or VST plugin and therefore integrates into other applications. However, its editing and arranging capabilities aren’t as powerful as those of Reaktor and it doesn’t support the unique features of LV2, like Atom ports for the exchange of arbitrary data between plugins. I guess that the goal behind Carla is to provide a virtual studio with a few dozens of big sound processors, not hundreds of small synth modules. It also focuses too much on supporting everything, at least for my taste: It can host plugins of almost all relevant standards and operates with a lot of audio drivers. This requires a lot of constant maintenance and development power that isn’t spent on actual features. Also, telling from which features are considered “experimental”, its architecture isn’t really made for the things that Reaktor can do.

Concluding, I basically want an open-source version of Reaktor that hosts LV2 plugins instead of having built-in cells. This would mean that I could just jam together synthesizers to create the sounds I want and use them in my Ableton projects, but if I want something that can not be expressed as a graph or want to have more performance, I could implement it as a native LV2 plugin node. I’m explicitly mentioning LV2 here because it’s an established standard that can be easily extended and that I know it quite well by now. It would be stupid for me not to use it.

Notice how I didn’t mention Rust until this point. To be honest, I don’t necessarily need this software to be written in Rust. I just want this software to make music, and if someone else happens to write it in C++ or even C, I’m fine with that. The language an application is written in isn’t a feature and it might also be better to not use Rust for the frontend / UI because other, more mature technologies have better tools available. I’m maybe thinking about QtQuick backed by a static C++ or Rust library, but I haven’t made my mind up on that.

That’s my vision. Sadly, my time and energy for it have diminished quite a lot in the last months. I almost quit Rust-LV2 entirely, but since activity here and on Github has sparked up again lately, I might come back again. My next step would be to write a product vision that precisely describes what I want to do and can be used as a reference for further discussion and development. What do you think?

Hi, let me cheer you all up a little bit :slight_smile:

First, there’s no denying what @Janonard says about the “competition”: the amount of work that has been put in the real big DAW’s is enormous. When I started programming, I believed that if you were smart enough, you could do in a couple of days what others would cost weeks or even months. Now I’ve learned that while this is true to some extent, there are also tasks in the development process that just cost time, no matter what. So then, how to proceed?

The first thing is: collaborate! Join forces! Modern development collaboration platforms like Github are better than they’ve ever been; it’s a great time for collaboration. Also Rust is a very good language for that since it invites (or maybe forces?) to write simple, readable code and it allows to write doc-tests etc. If there is a piece of Open Source software (in Rust) that you like, except for some aspects, it’s far less work to just contribute to that (or fork it) than to first build something similar and only then add the things you like or fix the things you don’t like. Also, because Rust invites to split software into many crates, chances are that you can re-use a lot of what has been developed by others.

The second thing: work process-oriented, not goal oriented. Programming can be fun (ok, some frustrations from time to time). Make sure you have something you can use quite soon in the process (and not only after 31 years, because then you’re going to give up). So start by building a small application, with limited features that you can use and then add features, rather than trying to build one monolithic application that is only finished after 31 years. Above all: have fun!

Will this help to build the next Ableton Live? No, of course not. But you may have fun building something that others enjoy to use.

I could give my two cents.

While it will be a monuments task to create the “blender” of audio production to compete with other DAWs, that doesn’t mean current commercial software doesn’t leave anything to be desired.

My main wish is to have the DAW with the best user interface (which is why I am mostly focused on GUI stuff). I don’t really care about the quality of built-in plugins as I’m just going to use third party ones anyway.

I dream of a DAW with a piano roll that feels as good to use as FL Studio, the ability to seamlessly edit MIDI and automation between multiple instruments, DAW-wide snapping of MIDI to the key of your song, a built-in multiple source spectrum-analyzer, modular plugin racks (somewhere in-between FL Studio Patcher an Bitwig Studio, I feel Bitwig is a bit too modular to the point where it’s unintuitive), a seamless “sound design” mode to go along with the default timeline and clip launcher mode, automatic sorting of samples based on timbre (like one plugin I can’t remember the name of), LV2 and Linux support, an intelligent mixer that shows the balance between tracks and not just their volumes individually, FL-like drum sequencer with integrated velocity, pitch, swing, and automation controls, and the list goes on. Oh yeah, and a big one for me is user-customizable themes and skins. I like to change the mood of my software to match the mood of the current track I’m working on.

There is a huge advantage when it comes to choosing Rust as your programming language. Like @PieterPenninckx said, its modular crate system makes it so easy to reuse code that other people have contributed. You should always play to your strengths (like mine with GUI), and let others with expertise in other areas help out. I would never be able to figure out the things on my own that wrl and the others are doing in the Discord chat, and I am very thankful. I would love to collaborate with someone who is better at DSP to help me out when I eventually start my DAW project.

There is an argument to be made where the end user might actually care if the software is written in Rust, and that is stability. I feel like the bane of other open source projects I follow like Krita is how many months they need to spend on just fixing bugs and crashes alone. I want to like ZRythm, but it is one of the most unstable pieces of software I ever used. It just crashes left and right.

Rust by design makes it far easier to create maintainable, readable code that is less likely to crash because of its lifetime checks and forced pattern matching. I imagine anyone who creates open source software with it can afford to spend a lot less time testing and fixing bugs, and thus can have an edge over the competition. This is a big reason I originally considered Rust in the first place.

I wouldn’t bother creating a new plugin standard until there is a Rust DAW that has gained popularity. If we can create the “blender” of audio production, I imagine there will be a desire to create completely-Rusty plugins for that specific DAW. With projects like baseplug, it could be as simple as adding an extra target to your compiler.

Fun fact: Ardour contains nearly no DSP code; the only DSP code is for the meters (I think it was mentioned in this interview).

@BillyDM, the example of blender you give is an interesting one: if you look at the history of blender, you see that a lot of resources were used to create and maintain blender. Just as an example: the blender institute currently employs 24 people, that’s about 20% of the number of people on this forum.

So my question is: how do you motivate people (including yourself!) to donate resources (time or money) over an extended period of time to a big project that has just begun? You need to assure somehow that the project will not fail and that others will also invest in it and keep investing in it. Explaining how great the end product would be often has the opposite effect: every feature added to the “wish list” increases the amount of resources needed and hence decreases the confidence that enough resources will be found to eventually get to that goal.

I think we can create the “blender of audio” in Rust, but just not in in the short or medium term; it will take much longer. In the meanwhile, we can build the technical foundations, the know-how and the experience and keep ourselves motivated by working on smaller projects.

Krita is another interesting case. In addition to sponsors, they also have yearly “stretch goals” with their donations. At least they used to, I’m not sure they do that anymore.

I agree it is important to build up experience by working on the smaller projects first. Our most important focus right now is to create a working full-stack audio plugin. We’re close, but there are still a few hurdles.

A roadmap could help keep us motivated. I imagine it is currently something like this:

  • Create some system to communicate parameter changes to and from baseplug to baseview.
  • Integrate parameter systems into GUI libraries such as iced_audio.
  • Add animation support to iced_audio whenever it is officially added to iced.
  • Finish basic window and keyboard input handling in baseview for Linux, Windows, and Mac.
  • Add character input system for baseview.
  • Create working MIDI IO in baseplug.
  • Create a few simple example VST plugins.
  • Add LV2 support to baseplug.
  • Finish VST3 support in baseplug.
  • Add AU support to baseplug.
  • Remove nightly dependencies in baseplug.
  • Polish baseplug and baseview.
  • Write documentation and possibly a website for full-stack audio plugin development in Rust.
  • Get attention from other developers and sponsors.
  • See where to go from here.

We may also be at the point where we can assign tasks to certain people, instead of having each person work on whatever.