Dick Show Index

A complete pipeline to download, transcribe, analyze, and search all episodes of The Dick Show podcast.

What This Does

Downloads all episodes from the official RSS feed (Libsyn)
Transcribes audio locally using Whisper (faster-whisper, CPU/GPU)
Analyzes each episode with an LLM (via opencode) to extract:
- Talking points with timestamps
- Arguments, ideas, and notable quotes
- Transgression Index (TI) 0-10 scoring
- Speaker attribution and content warnings
Fact-checks analyses against a verified research dossier
Indexes everything in SQLite FTS5 for instant search
Exports a master JSON (data/dickshow_analysis.json) for web apps

Quick Start

# 1. Clone and install
git clone https://kode.hannesahlin.se/n3wk/dickshow-index.git
cd dickshow-index
./install.sh

# 2. (Optional) Authenticate opencode for higher rate limits
opencode auth login

# 3. Run the Rekieta arc (28 episodes):
python pipeline_all.py --eps 84 87 89 93 96 121 124 152 166 176 198 209 213 275 285 299 321 344 352 368 376 412 416 424 425 435 439 455

# Or run the full backlog (520 episodes):
python pipeline_all.py

What You Get

After running, you'll have:

data/
├── episodes.jsonl          # Episode metadata (title, date, audio URL, duration)
├── index.db                # SQLite FTS5 search index
├── dickshow_analysis.json  # Master analysis JSON (for web apps)
└── analysis/
    ├── ep0084.json         # Per-episode analysis + fact-check
    └── ...

transcripts/
├── ep0084.json             # Raw transcript with timestamps
└── ...

Search Examples

# Simple search
python search.py "isom trademark"

# Filter by tag
python search.py maddox --tag lolsuit

# Phonetic search (Whisper mangles names)
python search.py "raketa OR riketa OR rakita" --tag nick_rekieta

# Date range + tag
python search.py maddox --from 2017-01-01 --to 2018-12-31 --tag lolsuit

# Context around matches
python search.py "pull up" --tag eric_july --context 2

Output includes timestamps for deep-linking:

[Ep 373 | Mon, 28 Aug 2023] 00:25:32-00:25:35 #nick_rekieta
    "I was on Nick's show, Busting Balls, Nick Riketa."
    -> https://dick.show/episode-373/
    mp3: https://traffic.libsyn.com/.../The_Dick_Show_373.mp3

Configuration

Set via environment variables or edit config.py:

Variable	Default	Description
`OPENCODE_ANALYZE_MODEL`	`opencode/nemotron-3-ultra-free`	LLM for analysis
`DS_ANALYZE_PAUSE`	`45`	Seconds between analysis calls
`DS_ANALYZE_TIMEOUT`	`600`	Analysis timeout (seconds)
`DS_MIN_FREE_RAM`	`1073741824` (1GB)	Pause if RAM below this
`DS_MODEL`	`small`	Whisper model: `tiny`, `base`, `small`, `medium`, `large-v3`
`DS_DEVICE`	`cpu`	`cpu` or `cuda`
`DS_COMPUTE`	`int8`	`int8`, `float16`, `float32`
`DS_KEEP_AUDIO`	`0`	Keep MP3s after transcription (`1` to keep)

The pipeline is fully resumable:

Re-run the same command → skips completed episodes
Only missing transcripts/analyses are processed
Share transcripts/, data/analysis/, and data/index.db to collaborate

Requirements

Python 3.10+
ffmpeg (for audio processing - usually auto-installed with faster-whisper)
opencode CLI (installed by ./install.sh)
~50 GB free disk for full backlog (MP3s auto-deleted after transcription by default)

Architecture

download.py     → fetches MP3s from Libsyn RSS (resumable, size-verified)
preprocess.py   → converts MP3 → clean 16kHz mono WAV (handles Libsyn quirks)
transcribe.py   → faster-whisper, chunked (10-min windows, no OOM)
analyze.py      → opencode subprocess with timeout + retries
verify.py       → second-pass fact-check against research dossier
index.py        → SQLite FTS5 index with hashtags
search.py       → CLI search with timestamps + context
pipeline_all.py → orchestrates everything with resume logic

For Contributors

The pipeline is designed for distributed workloads:

Each user runs a subset of episodes (--eps 84 87 89 ...)
Results are merged via data/analysis/ JSON files and index.db
Master JSON (data/dickshow_analysis.json) aggregates everything

License

Private research tool over public broadcasts. Content is adult/extreme — use responsibly.

Support

Issues: https://kode.hannesahlin.se/n3wk/dickshow-index/issues
Pipeline logs: rekieta_run.log, backlog_run.log
Analysis logs: analyze_run.log, analyze_err.log

4.6 KiB Raw Blame History