Stem Separation Explained: How AI Splits a Song into Stems

What a stem is (and what it isn't)

In a studio, a stem is a submix: all the drum tracks bounced to one stereo file, all the vocals to another. Separation software borrows the word for the same idea. The standard four stems (vocals, drums, bass and "other") come from the research benchmark most models are trained and scored on, MUSDB18, which is why nearly every tool from a DAW to the stem splitter offers exactly those four. A stem is not a multitrack: you get the drums as one file, not the kick, snare and overheads as separate ones.

Why you can't just un-mix a song

Mixing adds waveforms together, and addition loses information. Two instruments playing the same note at the same time produce one combined wave with no label saying which part came from where. So separation cannot be an inverse of mixing; there is nothing to invert. It has to be an estimate: given this mixture, what is the most plausible set of parts that would add up to it? That is a learning problem, and it is why the field only took off when deep learning did.

How the models do it

The first widely used model, Deezer's Spleeter (2019), worked on the spectrogram: it looked at a picture of the song's frequencies over time and learned a mask for each stem, a map of which pixels belong to the voice, which to the drums. It was fast and much better than anything before it, and it left the characteristic "underwater" artifacts of a mask that cannot decide.

Meta's Demucs went the other way and worked directly on the waveform, which lets a model repair phase, the thing spectrogram masks cannot touch. Demucs v3 combined the two views in a hybrid network, and Demucs v4 (2022) added transformer layers on top; that version, Hybrid Transformer Demucs, scores around 9 dB of signal-to-distortion ratio on the standard benchmark, against roughly 6 dB for Spleeter. It is the model the browser separator on this site runs. Since then, the leading vocal-separation results have come from transformer models that work on mel-scaled bands of the spectrogram, such as Mel-Band RoFormer, which is the kind of model behind the HD tier.

What the SDR numbers mean for your ears

Signal-to-distortion ratio is a logarithmic score of how close an estimated stem is to the real one. A 3 dB gain is a large, obvious improvement; the jump from Spleeter to Demucs v4 is roughly that. In listening terms, the failure modes at each level are different. Weak models leave bleed: drums audible in the vocal, voice ghosting in the instrumental. Mid-level models leave phasiness and a slightly hollow tone on sustained notes. Good models are mostly caught out by content the training data did not cover, such as heavily processed vocals, unusual instruments, or a live recording with a crowd.

Four stems or six?

The four-stem models lump guitars, keys, strings and synths into "other". A six-stem variant of Demucs also separates guitar and piano. Every added class makes the problem harder, because the model has more ways to be wrong, and the six-stem versions score a little lower on the shared stems. For a practice track (play along without the guitar part) or a remix, six stems are worth it; for a karaoke track or an acapella, four are cleaner. On this site the browser tier runs the four-stem model and Pro adds six.

Why "other" is a catch-all

Because the benchmark defined it that way, and because it is honest. The model is confident about voice, drums and bass, three sources with distinctive character. Everything else shares too much: a distorted guitar and a synth pad overlap in pitch, texture and stereo position. Rather than guess, the model puts what remains after the confident three into one stem. When you hear a synth in the drums stem or a brass stab in the bass, that is the boundary of what the model learned to recognise.

Running it in a browser

A separation model is just a large set of numbers and a lot of multiplication. Modern browsers can do that multiplication on the graphics card through WebGPU, and fall back to multi-threaded WebAssembly on the CPU where WebGPU is missing. The model file for the browser separator is about 170 MB, downloaded once and cached; after that a song takes one to three minutes on a laptop GPU and the audio never leaves the machine. The practical limits are memory and time, which is why the browser tier caps files at 50 MB or 10 minutes and why Safari and Firefox, on the CPU path, take longer.

What an HD model changes

Larger models separate better and cost more to run, and the largest ones are not practical in a browser at all. The HD tier on this site sends the file to a server GPU, runs a bigger vocal model and the six-stem Demucs, and returns the results as WAV or FLAC; the upload is deleted about an hour later. The audible difference shows up on exactly the songs where the browser model struggles: dense mixes, long reverbs, low-bitrate sources. On a clean pop master, the two are close.

Formats: what to feed the model and what to export

Give the model the least-compressed file you have. Lossy codecs like MP3 discard high-frequency detail on purpose, and that is precisely where the model tells cymbals from sibilance. Export stems as WAV or FLAC, not MP3, until the final mix: each lossy encode adds artifacts, and they compound on an exposed stem in a way they never would on a full mix.