How to Detect AI-Generated Music with the Modulate API

June 25, 2026

Parker Ennis

(HE/HIM/HIS)

AI-generated music is being uploaded to streaming platforms and other distribution mediums at a scale that no review team can handle, and most detection tools aren't built for the messy reality of real catalogs. They return a single score for a whole track and lack the ability to tell you where in a song any AI-generated content shows up. Worse, they routinely flag legitimate human recordings as synthetic, because autotune, heavy compression, and modern mastering look “artificial” to a model that was only taught to spot fully generated tracks.

Modulate's AI Music Detection API takes a different approach. It runs two independent models: one for vocals, one for instrumentals, and scores every four seconds of audio separately. This means you get a confident, evidence-backed verdict instead of a black-box probability. Below, we'll walk through how it works and how you can start detecting AI music in a few lines of code.

Overview: Modulate's API approach

One endpoint, structured output. POST an audio file (or open a WebSocket) and get back a primary_verdict of ai-vocal-music, ai-instrumental, or not-ai-music, plus four-second segment scores and confidence values.
Two independent detection paths. Vocals and instrumentals are scored separately, on every four-second window, so hybrid tracks (e.g. AI vocals over a human backing) don't miss it.
Per-window breakdown. You see where in a track AI content appears, not just a single score for the whole file.
Batch and streaming. Batch returns the full result in one response; the WebSocket endpoint returns vocal verdicts live and the instrumental verdict at end-of-stream.
Tunable thresholds. Adjust the precision/recall tradeoff per use case, no retraining.
Formats: .aac, .flac, .m4a, .mp3, .mp4, .ogg, .opus, .wav, up to 100 MB. For support for files larger than 100 MB, contact our sales team directly.

Developer note: The output is structured data you can branch on directly. A verdict enum, numeric percentages, and confidence values rather than a single opaque probability you have to second-guess. The per-window array means you can build segment-level logic (flag the chorus, ignore the verses) instead of treating every track as all-or-nothing. And because thresholds are a request-time parameter, you tune behavior in your own code without waiting on a model retrain or a vendor config change.

What the AI music detection API returns

The API answers this question: is this music AI-generated? If so, what type and where? It does that by classifying each four-second window of a clip and then aggregating those windows into a single clip-level result.

Each window is routed down one of two paths based on its content. Windows with enough vocal content are scored for AI-generated singing or rap, producing a vocal_ai_percentage and a vocal_ai_confidence. Windows without meaningful vocals are scored for AI-generated instrumental content, producing an instrumental_ai_percentage and instrumental_ai_confidence. Running the two paths separately is what lets the API handle hybrid tracks. For example, AI-generated vocals laid over an organic human backing.

Those window scores roll up into a clip-level primary_verdict, which is one of three values:

ai-vocal-music — AI-generated vocals were detected
ai-instrumental — AI-generated instrumental content was detected with no qualifying vocals
not-ai-music — no AI-generated music was found

Your first request

Before making your first request, remember to get an API key. You can get an API key here in just a couple of clicks.

Detection runs over a single endpoint. Send a complete audio file to the batch API and you get back the clip-level verdict plus the full per-window breakdown. Supported formats are .aac, .flac, .m4a, .mp3, .mp4, .ogg, .opus, and .wav, up to 100 MB.

Here's the simplest possible call with curl:‍

bash

curl -X POST https://platform.modulate.ai/api/velma-2-ai-music-detection-batch \
  -H "X-API-Key: $MODULATE_API_KEY" \
  -F "upload_file=@audio.mp3"

In Python it looks like this:‍

python

import os, requests

response = requests.post(
    "https://platform.modulate.ai/api/velma-2-ai-music-detection-batch",
    headers={"X-API-Key": os.environ["MODULATE_API_KEY"]},
    files={"upload_file": open("audio.mp3", "rb")},
)
response.raise_for_status()
result = response.json()

print(f"Verdict: {result['primary_verdict']}")
print(f"Vocal AI: {result['vocal_ai_percentage']}% ({result['vocal_ai_confidence']:.0%} conf)")
print(f"Instrumental AI: {result['instrumental_ai_percentage']}% ({result['instrumental_ai_confidence']:.0%} conf)")

A typical response:

json

{
  "filename": "my_audio.mp3",
  "duration_s": 89.28,
  "primary_verdict": "ai-vocal-music",
  "vocal_ai_percentage": 56.5,
  "vocal_ai_confidence": 0.96,
  "instrumental_ai_percentage": 10.5,
  "instrumental_ai_confidence": 0.95,
  "windows": [
    {
      "start_time_ms": 0,
      "end_time_ms": 4000,
      "vocal_ai_percentage": 100.0,
      "vocal_ai_confidence": 0.97,
      "instrumental_ai_percentage": 0.0
    }
  ]
}

‍‍

Reading the output

The fastest way to use the response is to read primary_verdict and act on it when using batch. The percentage and confidence fields give you the necessary supporting detail.

vocal_ai_percentage and instrumental_ai_percentage describe how much AI-generated content the model found, and the matching confidence values tell you how sure it is. The windows array is where the per-segment evidence lives. Each entry covers a four-second slice and shows exactly where in the track AI content turned up. That visibility is what makes hybrid tracks tractable: instead of a single number for an entire song, you can see that the chorus is AI-generated while the verses are not.

One important rule when you build on this is to rely on the clip-level verdict when you're judging a whole song or segment. Individual window scores can be noisier than the aggregated result, so the per-window data is best used as supporting evidence and for locating AI content, not as a standalone pass/fail signal.

Streaming for real-time use

If you're processing live audio or want results to arrive progressively rather than waiting for a full file, use the WebSocket endpoint instead of batch. Vocal AI verdicts are returned per window as the audio streams in; the instrumental verdict and the final clip-level summary arrive in the closing done message.

python

import os, asyncio, json, websockets

API_KEY = os.environ["MODULATE_API_KEY"]
AUDIO_FILE = "audio.mp3"
CHUNK_SIZE = 65536

async def stream():
    url = (
        f"wss://platform.modulate.ai/api/velma-2-ai-music-detection-streaming"
        f"?api_key={API_KEY}&audio_format=mp3"
    )
    async with websockets.connect(url) as ws:
        async def send():
            with open(AUDIO_FILE, "rb") as f:
                while chunk := f.read(CHUNK_SIZE):
                    await ws.send(chunk)
            await ws.send("")  # empty string signals end of stream

        async def receive():
            async for message in ws:
                msg = json.loads(message)
                if msg["type"] == "window":
                    w = msg["window"]
                    print(f"{w['start_time_ms']}ms vocal_ai={w['vocal_ai_percentage']}")
                elif msg["type"] == "done":
                    print(f"Verdict: {msg['primary_verdict']}")
                    break
                elif msg["type"] == "error":
                    raise RuntimeError(msg["error"])

        await asyncio.gather(send(), receive())

asyncio.run(stream())

‍Container formats such as wav, mp3, ogg, flac, webm, aac, and aiff only need the audio_format query parameter. Raw PCM additionally requires sample_rate and num_channels. Send an empty string to signal the end of the stream.

Tuning for your use case

Different workflows weigh precision and recall differently. A distributor screening millions of uploads may want to minimize false positives so it never wrongly flags a human artist, while a rights team doing due diligence on a handful of tracks may prefer to catch every possible AI element for manual review.

The API provides the values that you can use to create thresholds by applying your own logic, setting vocal and instrumental thresholds independently to shift that precision/recall tradeoff — and you can do it without retraining the model. That means you can adjust detection behavior to fit the cost of a mistake in your particular pipeline, rather than living with a fixed cutoff someone else chose. By default, the model detects if enough voice is detected to be considered vocals or not using a set value.

What to look out for

As we continue to improve the music detection API t's worth knowing where the current model has gaps so you can plan around them. The API reliably detects fully AI-generated songs, AI vocals over organic music, and AI-only instrumentals. The known current gaps are AI-generated choral or background vocals, which aren't always recognized as vocal content, and AI backing tracks sitting underneath a genuine live human performance. Heavily processed or high-production tracks are also occasionally mislabeled as AI-generated; this is a known area targeted by future model updates.

We're always looking for feedback, ways to improve, and what to build next. Contact us at support@modulate.ai with any requests or feedback.