Rust Audio

Playing back a midi with midly: how to organise the ecosystem

Hi all,

In this issue of the midly crate, GitHub user Dimev asks how to use midly to play back a midi file. This requires several steps:

a.1. Parsing the midi file. This is what midly already does.
a.2. Converting the delta times into “absolute” times
a.3. Merging all the tracks
a.4. Looping over the midi events and actually producing sounds or sending the midi events to a midi device.

Step a.4. (sending the midi events) depends heavily on what system you’re working with (Linux/Windows/Mac OS/…) and there are already crates for that purpose (e.g. jack, alsa, …). So what remains are steps a.2. and a.3. As it happens, I’ve already implemented steps a.2. and a.3. in rsynth (“cutting edge” version, not in the version released on crates.io). Now I don’t think rsynth is the appropriate place. midly maintainer negamartin doesn’t seem to be eager to include this functionality either.

So how to proceed with this? Any ideas? The most logical thing currently seems to implement a.2. and a.3. in a separate crate, maybe with the name midly-player. Now there’s also the converse problem of recording midi. For this you would need the converse:

b.1. Capturing midi events from a midi device (can be done with existing crates)
b.2. Converting absolute time into relative time.
b.3. Split tracks (optionally)
b.4. Save the tracks to a file (can be done with midly)

So then we can also create a crate that does a.2. and a.3., maybe with the name midly-recorder. Now, converting times would be a shared functionality between midly-player and midly-recorder, so we can split that off in a separate crate, let’s call it time-conversion.

So then a typical dependency tree would look like the following for a crate that plays midi files

(your crate that plays midi files)
 ├── midly-player
 │      ├── time-conversion
 │      └── midly
 ├── alsa or jack or ...
 └── ... (other dependencies)

A typical dependency tree would look like the following for a crate that records midi files

(your crate that records midi files)
 ├── midly-recorder
 │      ├── time-conversion
 │      └── midly
 ├── alsa or jack or ...
 └── ... (other dependencies)

Any thoughts, opinions or suggestions?

I’d say the straightforward way to group all functionality together would be to make a simple-midi crate of sorts, which integrates MIDI encoding/decoding from crates like midly with MIDI playback from crates like midir that already abstract from the underlying OS, ideally with flexibility with regard to the MIDI input and output (eg. a trait for producers/consumers of MIDI messages, and implement this trait for midir streams and a type representing a running MIDI file).

For example, the playback case would be handled by a MIDI-file-producer linked to a midir-output-consumer, while the recording case would be handled by a midir-input-producer linked to a MIDI-file-consumer. These common cases should have an easy shortcut function in this theoretical library.

I agree that rsynth seems out-of-scope. I believe that the scope of such a library should only be MIDI, with the concept of audio samples and sample rates being out-of-scope.

And in the subject of naming, it would be incredibly nice to get the midi crate name for such a broad-scope high level crate. But it doesn’t really matter.

Hi @negamartin , thanks for joining the forum and the discussion!

I had some use cases in mind that I didn’t mention and that guided me to idea that I sketched in my previous post. These use cases are the following:

  • An application that changes the delta-times and the tempi so that the midi sounds the same, but the bars/beats are aligned (think of it as a “smart quantize” that shifts the bars instead of the notes) (this is something I would like to write, but let’s see)
  • An application that plays the midi as audio with some software synthesizer.

For these use cases, you only need features a.2, a.3, b.2 and b.3, but an extra dependency on midir would be useless for these use cases.

Of course, we can “stack” things. The dependency tree for an application that plays midi files would look as follows (assuming we can claim the midi crate):

(your crate that plays midi files)
 ├── midi
 │      ├── midly-player
 │      │   ├── time-conversion
 │      │   └── midly
 │      └── midir
 └── ... (other dependencies)

The dependency tree for the first use case would look as follows:

("smart quantize" crate)
 ├── midly-player
 │   ├── time-conversion
 │   └── midly
 ├── midly-recorder
 │   ├── time-conversion (duplicate)
 │   └── midly (duplicate)
 └── ... (other dependencies)

The second use case would have a dependency tree that looks like the following:

("timidity clone")
 ├── midly-player
 │      ├── time-conversion
 │      └── midly
 ├── alsa or jack or ...
 └── ... (other dependencies)

What do you think about this idea?

Hi all, I’ve published the crate that I had named time-conversion in the previous example as timestamp-stretcher. See the announcement on this forum. I’m thinking of taking midly-player and midly-recorder together, see the repository to have an idea what it would look like. I’m currently thinking about naming it “midi-reader-writer”, but that’s perhaps too general since it’s only compatible with midly. @negamartin, what do you think about publishing it as midly-reader-writer (may be a little confusing since it’s not by the same author as midly).