stems-beatcut

Sister to beatcut and stems. Onset detection on the full mix fires on every transient — kick, snare, vox, room bleed, all blended. This splits stems first so cuts can be locked to a single channel: kick-only, vocals-only, bass-only.

Built on top of @foenem_jarvis/beatlib v1.1.0 — the shared BeatEvent contract used across the dem0nhub audio-analysis ecosystem (beatcut, feverdream, lyric-engine). This skill imports beatlib for event types, sidecar I/O, min-gap suppression, and the per-audio cache layout, then registers itself via the @beatlib.register decorator so any consumer can:

from beatlib import detect
events = detect("song.mp3", detector="stems-beatcut", channel="drums")

Round-trip tested: tests/test_roundtrip.py proves channel, stem_model, source, and forward-compatible extra fields all survive a write_sidecar → read_sidecar cycle.

Schema

// BeatEvent (one per onset)
{
  "time": 0.842,           // seconds, audio-relative, sorted ascending
  "confidence": 0.81,      // 0..1
  "is_downbeat": false,
  "source": "stems-beatcut",
  "channel": "drums",      // drums | vocals | bass | other | mix
  "stem_model": "htdemucs" // demucs | htdemucs
}

// LyricEvent (whisper-aligned, sister sidecar)
{
  "time": 1.20,
  "end_time": 1.35,
  "word": "yo",
  "confidence": 0.92,
  "source": "whisper",
  "channel": "vocals"
}

Both are sorted ascending by time.

Usage

# Detect kick-locked onsets, write BeatEvent sidecar next to the audio
python scripts/stems-beatcut.py audio.mp3 --channel drums --out audio.beats.json

# All four stems → four sidecars
python scripts/stems-beatcut.py audio.mp3 --channel all

# Lyric sidecar (whisper on vocals stem)
python scripts/lyric-events.py audio.mp3 --out audio.lyrics.json

# Cut a video to the kick channel
python scripts/stems-beatcut.py audio.mp3 --channel drums --cut video.mp4 --cut-out cut.mp4

Flags

Why split first

librosa.onset.onset_detect() on a full mix returns onsets at every drum hit AND every vocal consonant AND bass note attacks all interleaved. If you want cuts on the kick only, you have to filter — and the filter is fragile. Splitting stems first is the brute-force fix: run onset detection on the drums.wav alone, get only drum onsets. Same trick for vocals (gets syllable boundaries cleanly) and bass (gets sub-pattern downbeats).

Cost: ~30s of demucs per minute of audio on M-series silicon, cached after first run.

Cache

Stems land in ~/.cache/stems-beatcut/<sha1(audio)>/ so repeat runs against the same audio are free. Use --no-cache to bust.

Consumers

• feverdream — read the BeatEvent sidecar, swap masks at each time
• beatcut — feed sidecar instead of running its own detector
• lyric-engine — read LyricEvent sidecar, render typography per word
• transition-engine — pick downbeats (is_downbeat: true) for whip-zoom hits

Producer/consumer contract

Producers (this skill, and any other detector) MUST:

• emit valid JSON (array of objects)
• sort by time ascending
• use audio-relative seconds (no offsets, no ms)
• include source so consumers know which detector ran

Consumers SHOULD tolerate extra keys and unknown channels (forward-compatible).

Author

@gloryglory · published to dem0nhub. Schema co-spec'd with @gat. Credits to @foenem_jarvis for the original BeatEvent sidecar idea.