Ableton Meets AI: How I Turned Claude Into My Studio Partner

aiabletonmcpmusic-productionopen-source

I need to tell you about the most fun I've had coding in years.

Picture this: it's 3 AM, I'm in the studio, I've got a groove going in Ableton but I can't nail the chord progression. Normally I'd spend the next hour trying random voicings until something clicks. Instead, I opened Claude and typed "give me a jazzy chord progression in Eb minor with some tension." And I watched the MIDI notes appear in my session. In real time. Inside Ableton.

That moment — that's when I knew this project was going to eat my life for a while. And honestly? Totally worth it.

What Is AbletonMCP?

The original AbletonMCP was created by Siddharth Ahuja — a brilliant concept that connects Ableton Live to Claude AI through the Model Context Protocol (MCP). Claude can see your session, create tracks, write MIDI, load instruments, add effects, control playback. Through natural language. No mouse needed.

I saw it and my brain exploded. As someone who's been producing music for years and writing code even longer, this was the perfect collision of my two worlds. But when I actually tried to use it for real production work, I kept hitting walls. Only 15 basic tools. Blocking connections that froze everything. No real pattern generation. One massive file that was impossible to extend.

So I did what I always do. I forked it and went to town.

My fork: github.com/Jeff909Dev/ableton-mcp

How It Works

Three layers talking to each other. Simple concept, powerful result:

┌──────────────────────────────────────┐
│            CLAUDE AI                  │
│                                      │
│  "Create a synthwave track with      │
│   drums, bass, and a pad in Am"      │
└───────────────┬──────────────────────┘
                │  MCP Protocol
                ▼
┌──────────────────────────────────────┐
│          MCP SERVER (Python)         │
│                                      │
│  ┌────────────────────────────────┐  │
│  │  Async Connection Layer        │  │
│  │  Response Cache (TTL-based)    │  │
│  │  PatternEngine (1,105 MIDI)    │  │
│  └────────────────────────────────┘  │
│                                      │
│  8 Tool Modules:                     │
│  Session · Track · Clip · Arrange    │
│  Device · Transport · Browser · Prod │
└───────────────┬──────────────────────┘
                │  TCP JSON (port 9877)
                ▼
┌──────────────────────────────────────┐
│    ABLETON REMOTE SCRIPT             │
│    (Inside Ableton Live)             │
│                                      │
│  Socket Server → Command Router      │
│  ~60 handlers → Ableton Live API     │
└──────────────────────────────────────┘

When you ask Claude "add some reverb to the drums," it figures out which tools to call, sends JSON commands over TCP to a Remote Script living inside Ableton, and that script talks directly to Ableton's Python API. Results flow back the same way. The whole thing takes milliseconds.

What I Changed (And Why It Matters)

From 15 Tools to 39

The original could create a track and drop some notes. Cool for a demo. But try to actually produce a song and you'll need to delete tracks, mix levels, search your library, build drum racks, warp audio... I added everything a real session needs:

  • Production workflows (5 tools): one-call creation of beats, basslines, melodies, chord progressions, and pads — powered by 1,105 real MIDI patterns
  • Browser & library (7 tools): text search, folder browsing, user library navigation, load instruments/samples, build complete drum racks in a single call
  • Clip operations (8 tools): create MIDI and audio clips, add notes, duplicate, delete, warp modes, pitch shift, audio clip properties
  • Transport (6 tools): play, stop, set tempo, undo, redo, capture MIDI
  • Track management (4 tools): create, delete, rename, mix (volume, pan, mute, solo in one call)
  • Scene control (4 tools): create, duplicate, fire scenes, stop all clips
  • Device control (3 tools): find and load instruments by name, get parameters, tweak by name
  • Session (2 tools): full session snapshot, read clip notes

That's 2.5x more tools than the original. Every single one added because I personally needed it while producing.

Async Everything (No More Freezing)

The original used blocking TCP sockets with time.sleep() calls everywhere. Every. Single. Command. Froze the server while waiting for Ableton's response. If you asked Claude to create 5 tracks, it would go: create... wait... create... wait... create... wait...

I ripped all of that out and replaced it with asyncio. Now Claude fires commands without waiting. Auto-reconnect with up to 3 retry attempts, configurable timeouts, and health checks. The difference is ridiculous — what used to feel laggy now feels instant.

Smart Caching

Here's something I learned the hard way: Claude asks Ableton "what's your current state?" a LOT. Like, before almost every operation. And every time, that's a TCP round-trip.

So I built a TTL-based cache. Session state gets cached for 3 seconds (it changes when you're working). Browser searches get cached for 30 seconds, and the browser tree for 60 seconds (your sound library doesn't change mid-session). Device parameters and clip notes get 2 seconds. Write commands automatically invalidate the cache. This alone cut redundant network calls dramatically.

Batch Commands

Instead of one command per TCP call, I added batching. When Claude wants to create 5 tracks and set their volumes, that's 1 network call instead of 10. Simple optimization, massive impact.

Full Session Snapshot

Classic N+1 problem: the old code queried the session, then each track individually, then each clip on each track. I added get_session — one call, entire session with all tracks, clips, devices, scenes, return tracks, master, tempo, time signature, everything. Massive latency reduction on the most common operation.

The Pattern Engine

OK, this is the part where I got a little obsessed.

Instead of having Claude guess at notes or use simple hard-coded patterns, I built a PatternEngine that works with 1,105 real MIDI patterns extracted from actual music. This isn't random generation — it's real musical DNA.

The pattern library — organized by category:

  • Bass: 391 patterns
  • Synth: 250 patterns
  • Keys: 157 patterns
  • Chords: 96 patterns
  • Drums: 72 patterns
  • Pads: 65 patterns
  • Melody: 27 patterns

The pipeline — Search → Pick → Adapt → Vary:

When you say "create a bassline in Am," the PatternEngine searches the library for matching bass patterns, scores them by compatibility, picks the best match, transposes it to your target key, stretches it to your session's tempo, and optionally adds variations (humanize, simplify, ornament). The result is musically coherent because it started from real music.

Key detection — using a Krumhansl-Schmuckler variant algorithm that analyzes pitch class distributions to figure out what key a pattern is in. It also parses key info from filenames when available (like Bass_Am_125bpm.wav).

11 drum styles — house, techno, rock, hiphop, trap, dnb, reggaeton, bossa nova, jazz swing, funk, and basic. Each style has 3-4 template variations with humanized velocity (±20%) and timing (±50ms) to make beats feel human, not robotic. Uses GM drum mapping: C1 for kick, D1 for snare, F#1 for closed hat, and so on.

9 chord progression presets — pop (I-V-vi-IV), jazz (ii-V-I), blues, rock, sad, epic, house, reggaeton, andalusian. You can also pass custom Roman numeral progressions. The engine knows the intervals for major, minor, maj7, and other chord types, and places notes with proper voicings.

This is what happens when a DJ who studied computer engineering builds a pattern engine at 4 AM with a library of 1,105 real patterns. You get something unreasonably thorough.

Modular Architecture

The original was one giant server.py. Fine for a prototype. Impossible to maintain or extend.

I split everything into clean modules:

MCP_Server/
├── server.py             # FastMCP entry point
├── connection.py         # Async TCP, auto-reconnect, retry
├── cache.py              # TTL-based response cache
└── tools/
    ├── production_tools.py    # One-call workflows (beat, bass, melody, chords, pad)
    ├── browser_tools.py       # Library search, browse, load, drum rack building
    ├── session_tools.py       # Session state & clip notes
    ├── track_tools.py         # Track CRUD & mixing
    ├── clip_tools.py          # MIDI/audio clip operations
    ├── arrangement_tools.py   # Scene management
    ├── device_tools.py        # Instruments & parameters
    ├── transport_tools.py     # Playback & recording
    └── pattern_generator.py   # PatternEngine + drum templates

Want to add a new tool? Drop a function in the right module. It auto-registers. Zero merge conflicts, zero spaghetti code.

How To Set It Up

Want to try it yourself? Here's the quick version:

Prerequisites

Installation

1. Clone my fork:

git clone https://github.com/Jeff909Dev/ableton-mcp.git
cd ableton-mcp

2. Install the MCP server:

uv pip install -e .

3. Install the Remote Script in Ableton:

# Copy AbletonMCP_Remote_Script/ to your User Remote Scripts folder:
# macOS: ~/Library/Preferences/Ableton/Live <version>/User Remote Scripts/AbletonMCP/
# Windows: C:\Users\<user>\AppData\Roaming\Ableton\Live <version>\Preferences\User Remote Scripts\AbletonMCP\

4. Enable the Remote Script in Ableton:

  • Open Ableton Settings → Link, Tempo & MIDI
  • Under Control Surface, select "AbletonMCP"
  • You should see "Connected" in Ableton's status bar

5. Configure Claude Desktop:

Add this to your Claude Desktop MCP config (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):

{
  "mcpServers": {
    "AbletonMCP": {
      "command": "ableton-mcp"
    }
  }
}

Or if you prefer using uv directly:

{
  "mcpServers": {
    "ableton": {
      "command": "uv",
      "args": ["run", "ableton-mcp"],
      "cwd": "/path/to/ableton-mcp"
    }
  }
}

6. Restart Claude Desktop and start producing!

Open Ableton, open Claude, and try: "What's the current state of my session?"

If Claude responds with your session details, you're connected.

Real Examples (Things I Actually Do)

Here's how I use this in my production workflow:

One-call beat creation: "Create a tech house beat with 4 bars." Claude picks from the drum templates, loads a Drum Rack, and writes a full pattern with kick, snare, hats, and percussion — humanized timing and all. Done in seconds.

Browsing my sample packs: "List my user folders. Now browse the Sidney Charles pack and find a bass loop in Am." Claude navigates my library, finds the right sample, and loads it on a track. I can even say "build me a drum rack with a kick, snare, and hi-hat from that pack" and it does it in one call with build_drum_rack.

Chord progressions from presets: "Create a jazz chord progression in Bb minor." Claude picks a ii-V-I pattern, voices the chords with proper intervals, and places them. Or I can be specific: "Give me a I-vi-IV-V in C major" with custom Roman numerals.

Audio clip workflow: "Load this audio loop on track 3, set it to Complex Pro warp mode, and transpose it down 2 semitones." Claude handles the full audio pipeline — loading, warping, and tuning.

Quick fixes: "The snare on track 1 is too loud. Drop the volume and solo the drums so I can hear it." Takes 2 seconds with mix_track instead of hunting through mixer controls.

Learning: "Read the MIDI notes on track 4 and tell me what key they're in." Claude reads the clip, runs key detection through the PatternEngine, and teaches you music theory in context.

The Hard Parts

A few things that made this challenging:

Remote Script sandbox: The Remote Script runs inside Ableton's Python environment. No pip, no external packages. Everything is built with Python's standard library and Ableton's internal _Framework API. You appreciate requests a lot more when you have to write raw socket handling from scratch. Plus the 2,700-line __init__.py with ~60 command handlers — all in a single file because that's how Ableton Remote Scripts work.

Main thread constraint: Ableton's API is not thread-safe. Every state-modifying call must happen on the main thread. The Remote Script uses schedule_message() to dispatch safely while reads happen on the socket thread. Queue-based command processing with proper locking for socket I/O.

Browser API complexity: Navigating Ableton's browser programmatically is... an adventure. Direct path navigation is fast, but URI-based search requires recursive tree walking that can timeout on large libraries. I added multiple strategies (path-based, URI-based, filtered browsing) and aggressive caching to make it reliable.

Pattern adaptation: Transposing MIDI patterns while keeping them musical isn't just math. You need key detection, interval-aware transposition, and tempo stretching that respects note relationships. The Krumhansl-Schmuckler algorithm was key (pun intended) to making this work.

What's Next

I keep adding tools as I need them. Some things on my radar:

  • Automation lane control (draw volume curves, filter sweeps)
  • Export/bounce from Claude
  • A web UI to visualize what Claude is doing
  • More pattern categories and a larger MIDI library
  • Return track / send management

Links

Credits

Massive respect to Siddharth Ahuja for creating AbletonMCP and proving that this crazy idea actually works. The foundation was rock solid. I just... couldn't stop adding to it.

If you're a producer who uses Ableton and Claude, there's genuinely no reason not to try this. It's free, it's open source, and it will change how you think about your production workflow.

The future of music isn't AI replacing artists. It's artists with AI making things that neither could make alone. And honestly? We're just getting started.