A new DJ application in Rust

Hello,
After working on Mixxx for 6.5 years, I am exploring the possibility of starting fresh with a new DJ application written in Rust. So far I have no code to show, only a placeholder repository with a roadmap. I am posting here to ask if anyone is interested in collaborating on this and how we can collaborate on the ecosystem across projects.

Mixxx was started 20 years ago by someone who still doesn’t consider himself a DJ and many bad technical decisions were made early on. That was before DJ controllers existed. Although the controller mapping system’s flexibility is great, it is also quite clunky because the design of the core application has evolved inconsistently (and I’ve been wanting to rewrite it since I first started using Mixxx :stuck_out_tongue: ).

Now, Mixxx is at a crossroads where the technical debt standing in the way of upgrading from Qt5 to Qt6 is enormous. Mixxx’s GUI rendering is very complicated, inefficient, and revolves around a deprecated API which was removed in Qt6. There were efforts to move to Qt’s newer API which were unsuccessful despite months of work. The only feasible path forward for Mixxx is to rewrite the entire GUI from scratch using QML instead of QWidgets. Progress has been made towards that, but there would still be a long way to go. Lots of Mixxx is coupled to the GUI, including the entire music library database system and Auto DJ feature, so those would have to be completely rewritten for Qt6. Although I have resolved compile errors with Qt6, finding boundaries to cut out code coupled to the legacy GUI so the application could link with Qt6 is a major challenge. Realistically I would anticipate it would take 1 - 2 years for Mixxx to make a release with Qt6.

So, I am now considering to invest those years into a new application built on a better foundation in Rust instead of continuing to rewrite Mixxx system by system. I have been exploring the existing Rust audio ecosystem and it looks like there are some nice libraries available to get started. In particular, creek and Symphonia look very useful. dasp looks like a great foundation which I hope gets widely adopted so we can build an ecosystem of easily interoperable crates. cpal looks useful as well, although it seems immature. I do want this application to be cross platform eventually, but for now I want to focus on Linux (unless some other early contributor wants to help with supporting another OS). If it turns out that cpal does not meet my needs, I might directly use the JACK bindings for now.

I think there is a lot of opportunity for collaboration with the RustyDAW project (ping @BillyDM). I don’t know if a DJ application would benefit from the complexity of rusty-daw-audio-graph because DJ applications have relatively fixed signal paths with typically only an effects system that is modifiable by the user at runtime. I think it could be really interesting to collaborate on rusty-daw-timeline, particularly the MusicalTime struct described in the Meadowlark design document. I am doubtful about the proposal in that design document to use MusicalTime as the basis for timing throughout the application. For a DJ application, the musical time data would come from automatic analysis which likely won’t be accurate in many cases. For a DAW, the musical timeline should not need to exist. For example, a musical timeline is useless if you’re editing a recording of speech. Instead, I suggest to use clock time (seconds) as the basis of timing throughout these applications.

As for the GUI, I plan to use SixtyFPS. While there are a handful of other cross platform Rust GUI projects, I think SixtyFPS is the most promising for a few reasons, despite its present immaturity. First and perhaps most importantly, it has two developers working on it full time who have a lot of experience from developing Qt (one of them was a lead engineer for QML). A cross platform GUI library is an enormous task and I doubt hobbyist projects will reach the maturity needed for something as complex as a DJ application. Also, I don’t want to create the GUI with Rust code; the declarative domain-specific language of SixtyFPS looks much more appealing to design and maintain IMO.

Let me know if you’re interested in collaborating. :smiley: The first steps will be simply playing an audio file with creek and showing a slider for the playhead. Take a look at the roadmap for further plans.

Do you have any suggestions for the application’s name?

9 Likes

Sounds interesting to me. I might not have that much time to contribute besides job and daddy stuff but that surely sounds interesting to me

I’m a systems programmer, currently learning Rust, and DJ in a previous life. I’d be interested in contributing.

2 Likes

Hello, lead developer of Meadowlark here!

This sounds like a very neat project! The goal of the RustyDAW project (which Meadowlark is a part of) is to be as flexible as possible to allow anyone to create a DAW with whatever workflow they wish. A Mixxx-like application would be a great addition to this project!

You’ve already noted one of RustyDAW’s project creek. Two other relevant (still WIP) repos we have are rusty-daw-audio-graph and rusty-daw-io.

The rusty-daw-audio-graph crate is meant to be a flexible and powerful audio graph system (the backbone of any DAW project). You are able to define whatever custom nodes you want. So far mono and stereo audio routing seems to be working (although it’s not thoroughly tested yet). We still need to add more features such as MIDI routing and multithreaded scheduling.

The rusty-daw-io crate is meant to be a replacement for cpal for the same reasons you mentioned. It’s focused on making it easy to create a powerful settings GUI for the end user to configure their audio devices. So far I think only Jack is implemented.

The rusty-daw-timeline repo is meant for a more traditional timeline-style approach. It’s sole purpose is to create a TimelineTrackNode which acts like a single node in the audio graph. However you will be able to define whatever custom audio graph nodes you want, including one with its own custom playback system for live DJ workflows.

We are also planning on creating a suite of DSP/plugins that can be used with the RustyDAW project. I’ve recently drafted a design document for that here: DSP Design Document.

GUI in rust is definitely a challenge. For Meadowlark specifically we are working closely with the developer of VIZIA for this project. It’s still very experimental, but one of the goals is to be optimized for large and complex GUIs.

If you’re interested we are most active in the Meadowlark Discord Server.

1 Like

I saw this and I was wondering what led you to the conclusion it would be better to start a new cross platform audio I/O library from scratch than fix the problems with cpal. I see that cpal is relatively old, started in 2015, and Rust has grown a lot since then.

Yeah, cpal just has some very strange design decisions such as returning a list of all possible final configurations instead of just returning a list of the available configurations per-device (which is what an end user configuring settings in a GUI application would expect).

Also some audio APIs like Jack and CoreAudio also have the benefit of syncing MIDI along with duplex audio to achieve the lowest-latency possible, and we want to take advantage of that.

That is an interesting point. AFAIK none of the C or C++ cross platform audio I/O libraries take advantage of this.

Another serious problem with all the C and C++ cross platform audio I/O libraries is that none of them support hotplugging devices! This is a serious issue for live performance software. Having to restart the application because a USB cable gets unplugged is really bad! If we could overcome this in rusty-daw-io that would be great.

Also, PortAudio (which Mixxx uses) is clunky to use with JACK. There is no API to create ports without using them immediately, which makes an awkward user experience together with JACK session management applications. PortAudio also doesn’t have an API for the application to name ports, so everything is just in_# and out_#.

Hmm, I’m unsure how hotplugging would even work. Of course if anyone knows how to do that than I’ll love to have you contribute!

rusty-daw-io lacks documentation and looks even less mature than cpal, so for now I think I’ll use the PipeWire Rust bindings. I am still interested in collaborating on a cross platform audio I/O library, but I don’t want to get too concerned with that now before I have anything working at all. Using the PipeWire API will be a good learning exercise so I could contribute a backend to a cross platform library. I want to design cross platform abstractions that allow applications to take full advantage of PipeWire’s features with graceful fallbacks for audio APIs that do not provide them. I don’t want to end up in a situation like the PortAudio JACK backend where an API stabilizes that prevents applications from behaving as users expect with the backend API.

Cool, sounds good! If you ever want to contribute to rusty-daw-io just let me know!

After lots of reflection and discussion, I’ve decided to change the direction of this project a bit. The world doesn’t need another software just like Mixxx, Serato, Traktor, or Rekordbox. There’s been hardly any groundbreaking innovation in DJ software in years. djay and Virtual DJ have recently integrated source separation, but it doesn’t sound very good (yet), so I’m not very interested in it. Apart from that, DJ software is largely stuck in the 2/4 turntables and a mixer paradigm, both in terms of UI design and workflow. I want to do something novel that opens new creative possibilities.

In my opinion, much of the skill of DJing is pressing play at the right time, so I want to design this new application to facilitate that. I added intro and outro section markers to Mixxx towards that end, but this can be taken much farther. I started a discussion brainstorming ideas to do this in Mixxx years ago, but now I am thinking it would be better to start a whole new application.

Instead of separate decks, I want to create an interface more like a DAW where all the tracks are together on one big timeline. That will allow the user to plan out the precise alignment of tracks well in advance of when they get played to the audience. However, DAWs are designed for offline editing, going back and forth between different times to make fine adjustments. I want software designed for improvisational live performance. I want a DAW-like timeline where tracks are like decks in conventional DJ software and can be mapped to real world DJ controllers that are commercially available.

I have been exploring other software and the closest that another software comes to what I’m thinking about is this new application Blueberry. I really like the idea of a headphones playhead decoupled from the main playhead that goes to the audience and I want to implement that. However, there are lots of odd aspects of Blueberry’s design that I don’t like, plus it is written in C++. Blueberry’s website says “Most DJ equipment is confusing and expensive. That’s why we made blueberry different.” It is designed to not be used with DJ gear. While it’s great that it can be, I have the opposite perspective. I want to design this new application to take full advantage of my NI Traktor Kontrol S4 Mk3 from the beginning and be usable with other common, commercially available DJ gear too. At the same time, I want it to be usable as a casual music player on my PinePhone so I can annotate tracks as I’m listening to them.

So I think there is more room for collaboration with a more conventional DAW project than I originally thought. If we collaborate on a common library for a managing an audio timeline, we could design it so that projects could be exchanged between these applications easily. So you could perform a mix live, record all the parameter changes as automation data rather than simply audio, and then edit that in Meadowlark later.

One big difference from a conventional DAW is that each track must have its own musical time grid. Synchronizing these musical time grids is really useful, but it must not be necessary like it is in Blueberry. That would limit the application’s usefulness to tracks that have a constant tempo that the software can analyze accurately and are close in tempo. Want to mix in a jazz track in Blueberry? Too bad, it gets timestretched. Want to manually beatmatch with an old school 2 turntables and a mixer setup with timecode vinyl? Too bad, Blueberry doesn’t work that way.

This timeline-centered design could eventually be expanded to add a clip launching feature with an interface similar to Bitwig’s horizontal clip launching layout where the clip grid is shown side-by-side with the timeline (rather than Ableton’s design where clips are in columns that take most of the screen). A major drawback of the clip launching approach is that it requires a lot of advance preparation. Ableton isn’t designed so you can just show up at a gig without planning anything and play a set the way you can with Mixxx.

One application that could combine both a traditional DJ approach of mixing complete tracks with clip launching could open a lot of novel creative possibilities. Native Instruments has tried to do this with Traktor’s remix decks and their not-really-open Stems format and, in my opinion, has largely failed (meanwhile neglecting core functionality and technical debt for a decade). Remix decks take a lot of preparation, but what if you could just load a project from Meadowlark or another DAW and easily put it into a clip launching interface? Stems claims to be an open format but actually requires a proprietary library and signing an NDA to view the specification. Moreover, Stems requires cooperation from producers and publishers, which just isn’t happening on a substantial scale, and I don’t think it ever will.

In conclusion, I’m excited to explore the creative possibilities that open up without needing to maintain 20 years of legacy code.

1 Like

I’m not 100% committed to this name yet, but I think I want to name this new application Moiré, which is a “an interference pattern produced by overlaying similar but slightly offset templates”. I’m imagining a logo with a moiré pattern made from waveforms.

PipeWire does too:

https://docs.pipewire.org/page_midi.html

After reading the Pipewire documentation and discussion in the Pipewire Matrix room, I now know that the Pipewire Rust bindings are not yet suitable for this project. The Rust bindings have a Rust wrapper for the simple pw_stream API which isn’t suitable for applications with multiple independent inputs and outputs. An application can create multiple pw_stream instances, but this creates separate nodes in the Pipewire processing graph and requires the application to handle a separate callback for each pw_stream, which introduces the issue of how to synchronize the buffers. By contrast the pw_filter API works more like a JACK client where the application gets one callback and can use an arbitrary number of independent ports which are all mono 32 bit floats. However, the Pipewire Rust bindings don’t have a Rust wrapper for pw_filter yet. So, at least for now, I’ll use the JACK Rust bindings.

I deleted the placeholder repository and moved to Be.ing/moire: Moiré is a musical live performance application with a DAW-like timeline interface - moire - Codeberg.org

I setup a Zulip chat server to discuss Moiré. Come say hi if you’re interested. :slight_smile:

At the point the code plays a file path specified on the command line to the JACK system output and shows the elapsed playback time in a GUI.

Amazing! I started something similar circa one year ago (I wanted an alternative to remix decks and a dj software that use a new paradigm).

Cause other things to do and lack of time I abandoned it. But I definitely have time to contribute to another project.

1 Like

I’m making some progress. I figured out how to use dynamically sized Vecs in the audio thread without heap (de)allocation in the audio thread, so I can freely add decks from the GUI at runtime and each deck can have an arbitrary number of clips. I’m using Vec::with_capacity to preallocate a container in the GUI thread, sending that and a new item over a channel with rtrb, then in the audio thread transferring ownership of the old elements into the new container. I’m using basedrop to deallocate the old Vecs in a dedicated thread. assert_no_alloc has also been really helpful for telling me when I have messed up. :slight_smile:

I implemented click and drag seeking: Imgur: The magic of the Internet

(Sorry for the imgur link. Discourse won’t let me upload a video even though it’s only 1.2 MB.)