DJVideoPipeline/README.md

DJ Video Pipeline

Ingest, evaluate, conform and DJ-edit a music video collection. The current implementation focus is safe ingest into immutable Originals, read-only pre-edit analysis, and a supported Kdenlive handoff. Kdenlive creates projects natively; the pipeline does not clone or rewrite Kdenlive XML. Mastering comes later.

Kdenlive is now the selected manual editor. Human-approved sequence guides must be named DJ START and DJ END (the reader also accepts DJ STOP). Clip markers are not used as edit boundaries.

Install

winget install --id Microsoft.PowerShell -e     # if you are not on pwsh 7 yet
cd C:\repos\DJVideoPipeline
pwsh
.\Install-DJPipeline.ps1 -Root 'C:\SYNC\Media\Music Videos\Pipeline'

The configured pipeline root is C:\SYNC\Media\Music Videos\Pipeline.

Daily use

# preview the next import without moving anything
.\Invoke-DJPipeline.ps1 -Stage Import -Limit 1 -WhatIf

# import one video, move it to Originals, analyze it, and create its Kdenlive handoff
.\Invoke-DJPipeline.ps1 -Stage Import -Limit 1

# import the next analyzed Original into Kdenlive's Project Bin
.\Invoke-DJPipeline.ps1 -Stage Edit -Limit 1

# first run: insert analyzed DJ START/DJ END guides into the closed project
.\Invoke-DJPipeline.ps1 -Stage Register -ProjectPath 'C:\path\to\project.kdenlive'

# reopen it and mark one or more keep-regions, then save and close
.\Invoke-DJPipeline.ps1 -Stage Register -ProjectPath 'C:\path\to\project.kdenlive'

# for multiple keep-regions, listen to the generated splice WAV files;
# adjust/save/register again if needed, or register unchanged to approve

# where does everything stand
.\Invoke-DJPipeline.ps1 -Stage Status

# full CSV of the library plus the vDeck sidecar
.\Invoke-DJPipeline.ps1 -Stage Report

Import performs one complete read-only decode after the move. It detects leading and trailing silence, black frames, and freezes; confirms the source decodes; writes <name>.pre-edit-analysis.json; and writes a Kdenlive handoff containing the required source-matching profile and suggested boundaries. These are review aids only—the source is never trimmed automatically.

-Limit works on every stage and always means "the next N files that need this stage". Nothing is reprocessed: state lives in _state\<id>.json, keyed by a content hash that survives renames.

Folders

C:\SYNC\Media\Music Videos\Pipeline\
  00_Inbox\            drop new videos here
  01_Originals\        immutable imported source videos
  02_Kdenlive_Projects\analysis, handoffs, and native Kdenlive projects
  02_OpenShot_Projects\legacy experiments; no longer used by Import
  03_Masters\          future mastered outputs
  04_DJ_Edit_Ready\    finished edits + vdeck-library.json
  90_Working\          future scratch files
  99_Quarantine\       files the pipeline will not touch
  _state\              one JSON per file: probe, loudness, analysis, plan
  _logs\               run logs and library reports

Stages

Stage	What it does	Leaves the file at
Import	move to Originals, analyze boundaries/decode health, write Kdenlive handoff	AwaitingKdenliveProject
Edit	import one Original into Kdenlive's Project Bin	manual drag to timeline and Save As are required
Register	seed/validate keep guides; generate beat/bar reports and lossless auditions for multiple regions; record approval on an unchanged follow-up run	AwaitingBoundaryReview, AwaitingSpliceReview, ReadyToEdit, or another review state
Scan	ffprobe, EBU R128 loudness, BPM/key/section analysis, triage verdict	Scanned or Quarantined
Conform	cheapest fix that makes it deck-safe	Conformed
Plan	detect cuts, write OpenShot project + cue sheet	AwaitingReview
Render	harvest reviewed cuts, one ffmpeg pass	Rendered

The active workflow currently uses Import, Edit, and Register. Mastering stages remain outside the workflow until ingest and manual editing have been proven with representative videos.

Multiple cuts and seamless splice review

A single keep-region uses DJ START followed by DJ END. For multiple regions, guides must alternate in time:

DJ START    beginning of keep-region 1
DJ STOP     end of keep-region 1
DJ START    beginning of keep-region 2
DJ STOP     end of keep-region 2 (or DJ END for the final region)

Register pairs these boundaries in order. Consecutive starts, an unpaired end, or an unclosed start fail validation instead of being guessed.

When two or more keep-regions exist, registration analyzes the music immediately before and after every proposed join. It reports the removed beat count, nearest whole bar and nearest four-bar phrase, and a suggested incoming-boundary adjustment. It also creates a lossless 48 kHz WAV for each join in 02_Kdenlive_Projects\<project>.splice-review. Each audition contains the last configured seconds of the preceding keep-region immediately joined to the first configured seconds of the next region; no source or timeline media is changed.

Listen through the join in each WAV. If it is wrong, move a guide in Kdenlive, save and close, then run Register again; changed guide positions automatically regenerate the review. Running Register again with unchanged reviewed guides records operator approval.

To audition an edit inside Kdenlive after actually extracting the unwanted timeline zone, place I several seconds before the splice and O several seconds after it, then press Ctrl+Shift+Space to loop the zone. Markers alone describe an edit; they do not remove timeline material.

Triage verdicts

Verdict	Meaning	Cost
Pass	already deck-safe	copy only
Remux	right codecs, wrong container	seconds
AudioOnly	video fine, audio needs normalising	~1/10 of a transcode
Transcode	codec, VFR, interlacing or resolution problem	full re-encode
Quarantine	no audio, no video, or absurd duration	skipped

Every verdict comes with the specific reasons, visible in -Stage Status and in the CSV report.

Legacy OpenShot stages

The older Plan and Render implementation still reads OpenShot .osp files, but it is retained only as legacy code while the Kdenlive edit-registration and render handoff are implemented. Do not use those stages for newly imported media.

How cuts are detected

Loudness alone is a bad discriminator: film dialogue in a music video can be as loud as the chorus. The analyser instead tracks energy in the 35–130 Hz band, which is kick and bass. A section where that band drops well below the track's own reference level, for long enough, is a section you cannot dance to — a cinematic intro, a spoken breakdown, a credits roll.

Cuts are then snapped to the beat grid, and breakdown removals are rounded to a whole number of bars, so the beat either side of the splice stays in phase. That single rule is the difference between an edit you can mix and one that trainwrecks.

The intro keeps its last 8 beats (edit.keepIntroBeats) so you still have a mix-in point rather than slamming into the chorus cold.

Tuning

Everything lives in djvideo.config.json.

Setting	Effect
analysis.dropDb	lower = more aggressive cutting. 9 dB is conservative; try 7 if it misses breakdowns
analysis.minBreakdownSeconds	ignore short breaks. Raise to 10 if it cuts things you wanted
edit.keepIntroBeats	how much intro survives as a mix-in
edit.snapRemovedToWholeBars	turn off only if the track has no steady tempo
spliceReview.auditionSecondsEachSide	listening time retained before and after each proposed splice
spliceReview.localAnalysisSeconds	music window used to estimate tempo independently on both sides of a splice
loudness.targetLufs	-11 keeps most commercial masters in "turn down" territory, which is transparent. Going to -9 forces upward gain and limiting on quieter files
encode.videoCrf	19 is visually transparent. 21 halves file size and is fine on a projector

Key detection caveat

Key comes from Goertzel chroma plus Krumhansl-Schmuckler profile matching. It reliably finds the right pitch class, but relative major and minor share the same notes and get confused — which is why the report gives you the pick, its relative, and a Camelot code. Both members of a relative pair share a Camelot number (A minor and C major are both 8), so for harmonic mixing the ambiguity is harmless. If you want the major/minor distinction to be right too, run Mixed In Key or keyfinder-cli over 04_DJ_Edit_Ready and take its answer.

keyConfidence below about 0.6, or keyMargin below 0.05, means do not trust it.

Requirements

• PowerShell 7+ (the scripts use -fps_mode and modern JSON handling)
• ffmpeg with libx264, libsoxr and the loudnorm filter
• Kdenlive 26.04+ for manual DJ editing
• .NET (bundled with PowerShell 7) — the DSP in lib\DJVideo.Dsp.cs is compiled at runtime by Add-Type, so there is nothing to build