# Uhm (`uhm-1`): Filler word detection and removal

Find every um, uh and false start. Get a cut list and a clean transcript.

Uhm finds the filler words in a recording: every um, uh and hmm, timed to 20 ms straight from the audio. It returns each filler with its start and end, a cut list your editor can apply as is, and how many fillers a minute the speaker uses.

Send `transcript: true` and it also writes the words, then finds discourse fillers ("like", "you know", "I mean"), repeated words ("the the", "three, uh, three") and false starts, and gives you the transcript with them taken out. Already have a timed transcript? Send it as `words` and only the audio check is billed.

In our tests on the AMI meeting test set (4.3 hours, 1,741 fillers) it scored an F1 of 0.68 at the default setting, and 0.85 precision with `bias: "precision"`.

- **Price:** $0.0003 per minute. Billed per second of audio, rounded up, at least one second. With `transcript: true` each second counts 4 units ($0.0012 per minute).
- **Languages:** English (um, uh and hmm in the audio; the transcript rules are English)
- **Input:** An audio or video file: an https URL, a data URL, or a file in a multipart upload. MP3, WAV, M4A, OGG, FLAC, WebM, MP4 and more, up to 25 MB and 10 minutes (5 minutes with `transcript: true`).
- **Batch:** no, one input per call
- **Status:** live

## Request

```bash
curl https://labs.greatwork.company/v1/run \
  -H "Authorization: Bearer $GWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"uhm-1","input":"https://labs.greatwork.company/examples/filler-voice-note.m4a"}'
```

Upload a local file with multipart: `curl https://labs.greatwork.company/v1/run -H "Authorization: Bearer $GWL_API_KEY" -F model=uhm-1 -F input=@file`

## Options

| Name | Type | Required | Default | Description |
|---|---|---|---|---|
| `input` | audio (URL, data URL or upload) | yes | | An audio or video file: an https URL, a data URL, or a file in a multipart upload. MP3, WAV, M4A, OGG, FLAC, WebM, MP4 and more, up to 25 MB and 10 minutes (5 minutes with `transcript: true`). |
| `bias` | string (precision \| balanced \| recall) | no | `"balanced"` | `precision` only marks fillers it's sure of (fewer wrong cuts), `recall` catches more of them. |
| `threshold` | number | no |  | Exact filler score cutoff, 0 to 1. Overrides `bias` (precision 0.9, balanced 0.45, recall 0.3). |
| `min_duration` | number | no | `0.12` | Shortest filler kept, in seconds. |
| `transcript` | boolean | no | `false` | Also transcribe the speech (English): adds `text`, `cleaned_text` and `words`, and finds "like", "you know", "I mean", repeats and false starts. Each second costs 4 units; audio up to 5 minutes. |
| `words` | object[] | no |  | Your own timed transcript instead of ours: `[{"text", "start", "end"}]` in seconds. Billed like audio only. |
| `remove` | string[] (filled \| discourse \| repeat \| false_start) | no |  | Which kinds go in the cut list and come out of `cleaned_text`. Default: all of them. |

## Output

| Field | Type | Description |
|---|---|---|
| `fillers` | object[] | In order: `kind` (`filled`, `discourse`, `repeat`, `false_start`), `text`, `start`, `end` (seconds), `confidence` (audio detections) and `source` (`audio`, `transcript` or `both`). |
| `cuts` | object[] | Spans to cut, `start` and `end` in seconds: padded a little, merged when close, and never cutting into a kept word. |
| `count` | integer | Number of fillers. |
| `counts` | object | Fillers by kind. |
| `per_minute` | number | Fillers per minute of audio. |
| `duration` | number | Audio length in seconds. |
| `removed_seconds` | number | Total length of the cuts. |
| `text` | string | The transcript (with `transcript: true` or `words`). |
| `cleaned_text` | string | The transcript with the removed kinds taken out and punctuation tidied. |
| `words` | object[] | `text`, `start`, `end`, plus `filler` (its kind) on words that are fillers. |

The response envelope is always `{"id", "object": "run", "model", "created", "output", "usage": {"units", "unit", "cost_usd", "cost_micros"}, "latency_ms"}`.

## Examples (real outputs from our servers)

### Voice note

```json
{
  "model": "uhm-1",
  "input": "https://labs.greatwork.company/examples/filler-voice-note.m4a"
}
```

```json
{
  "fillers": [
    {
      "kind": "filled",
      "text": "um",
      "start": 0.62,
      "end": 1.12,
      "confidence": 0.96,
      "source": "audio"
    },
    {
      "kind": "filled",
      "text": "uh",
      "start": 1.92,
      "end": 2.46,
      "confidence": 0.96,
      "source": "audio"
    },
    {
      "kind": "filled",
      "text": "uh",
      "start": 9.44,
      "end": 9.98,
      "confidence": 0.99,
      "source": "audio"
    }
  ],
  "cuts": [
    {
      "start": 0.59,
      "end": 1.15
    },
    {
      "start": 1.89,
      "end": 2.49
    },
    {
      "start": 9.41,
      "end": 10.01
    }
  ],
  "count": 3,
  "counts": {
    "filled": 3
  },
  "per_minute": 16.48,
  "duration": 10.92,
  "removed_seconds": 1.76
}
```

147 ms, $0.000055.

### With transcript

```json
{
  "model": "uhm-1",
  "input": "https://labs.greatwork.company/examples/filler-voice-note.m4a",
  "transcript": true
}
```

```json
{
  "fillers": [
    {
      "kind": "filled",
      "text": "um",
      "start": 0.62,
      "end": 1.12,
      "confidence": 0.96,
      "source": "both"
    },
    {
      "kind": "filled",
      "text": "uh",
      "start": 1.92,
      "end": 2.46,
      "confidence": 0.96,
      "source": "both"
    },
    {
      "kind": "discourse",
      "text": "like you know",
      "start": 4,
      "end": 5.02,
      "source": "transcript"
    },
    {
      "kind": "repeat",
      "text": "the",
      "start": 5.76,
      "end": 5.88,
      "source": "transcript"
    },
    {
      "kind": "filled",
      "text": "hmm",
      "start": 7.28,
      "end": 7.48,
      "source": "transcript"
    },
    {
      "kind": "discourse",
      "text": "i mean",
      "start": 7.68,
      "end": 8.26,
      "source": "transcript"
    },
    {
      "kind": "filled",
      "text": "uh",
      "start": 9.44,
      "end": 9.98,
      "confidence": 0.99,
      "source": "both"
    }
  ],
  "cuts": [
    {
      "start": 0.59,
      "end": 1.15
    },
    {
      "start": 1.92,
      "end": 2.49
    },
    {
      "start": 3.97,
      "end": 5.05
    },
    {
      "start": 5.76,
      "end": 5.88
    },
    {
      "start": 7.25,
      "end": 7.51
    },
    {
      "start": 7.65,
      "end": 8.29
    },
    {
      "start": 9.41,
      "end": 10.01
    }
  ],
  "count": 7,
  "counts": {
    "filled": 4,
    "discourse": 2,
    "repeat": 1
  },
  "per_minute": 38.45,
  "duration": 10.92,
  "removed_seconds": 3.83,
  "text": "So, um, I was, uh, thinking that we could, like, you know, maybe move the the meeting to Thursday, hmm, I mean, if that works for, uh, everyone.",
  "cleaned_text": "So, I was thinking that we could maybe move the meeting to Thursday if that works for everyone.",
  "words": [
    {
      "text": "So,",
      "start": 0.24,
      "end": 0.68
    },
    {
      "text": "um,",
      "start": 0.88,
      "end": 1.12,
      "filler": "filled"
    },
    {
      "text": "I",
      "start": 1.3,
      "end": 1.72
    },
    {
      "text": "was,",
      "start": 1.72,
      "end": 1.92
    },
    {
      "text": "uh,",
      "start": 2.14,
      "end": 2.44,
      "filler": "filled"
    },
    {
      "text": "thinking",
      "start": 2.76,
      "end": 3.02
    },
    {
      "text": "that",
      "start": 3.02,
      "end": 3.2
    },
    {
      "text": "we",
      "start": 3.2,
      "end": 3.46
    },
    {
      "text": "could,",
      "start": 3.46,
      "end": 3.72
    },
    {
      "text": "like,",
      "start": 4,
      "end": 4.3,
      "filler": "discourse"
    },
    {
      "text": "you",
      "start": 4.5,
      "end": 4.74,
      "filler": "discourse"
    },
    {
      "text": "know,",
      "start": 4.74,
      "end": 5.02,
      "filler": "discourse"
    },
    {
      "text": "maybe",
      "start": 5.32,
      "end": 5.6
    },
    {
      "text": "move",
      "start": 5.6,
      "end": 5.76
    },
    {
      "text": "the",
      "start": 5.76,
      "end": 5.88,
      "filler": "repeat"
    },
    {
      "text": "the",
      "start": 5.88,
      "end": 6.08
    },
    {
      "text": "meeting",
      "start": 6.08,
      "end": 6.32
    },
    {
      "text": "to",
      "start": 6.32,
      "end": 6.54
    },
    {
      "text": "Thursday,",
      "start": 6.54,
      "end": 7.14
    },
    {
      "text": "hmm,",
      "start": 7.28,
      "end": 7.48,
      "filler": "filled"
    },
    {
      "text": "I",
      "start": 7.68,
      "end": 7.92,
      "filler": "discourse"
    },
    {
      "text": "mean,",
      "start": 7.92,
      "end": 8.26,
      "filler": "discourse"
    },
    {
      "text": "if",
      "start": 8.42,
      "end": 8.58
    },
    {
      "text": "that",
      "start": 8.58,
      "end": 8.8
    },
    {
      "text": "works",
      "start": 8.8,
      "end": 9.16
    },
    {
      "text": "for,",
      "start": 9.16,
      "end": 9.52
    },
    {
      "text": "uh,",
      "start": 9.68,
      "end": 10,
      "filler": "filled"
    },
    {
      "text": "everyone.",
      "start": 10.36,
      "end": 11
    }
  ]
}
```

5574 ms, $0.00022.

### Noisy room

```json
{
  "model": "uhm-1",
  "input": "https://labs.greatwork.company/examples/filler-noisy.m4a"
}
```

```json
{
  "fillers": [
    {
      "kind": "filled",
      "text": "uh",
      "start": 1,
      "end": 1.38,
      "confidence": 0.98,
      "source": "audio"
    },
    {
      "kind": "filled",
      "text": "um",
      "start": 3.02,
      "end": 3.54,
      "confidence": 0.95,
      "source": "audio"
    },
    {
      "kind": "filled",
      "text": "um",
      "start": 14.28,
      "end": 14.86,
      "confidence": 0.98,
      "source": "audio"
    }
  ],
  "cuts": [
    {
      "start": 0.97,
      "end": 1.41
    },
    {
      "start": 2.99,
      "end": 3.57
    },
    {
      "start": 14.25,
      "end": 14.89
    }
  ],
  "count": 3,
  "counts": {
    "filled": 3
  },
  "per_minute": 11.43,
  "duration": 15.74,
  "removed_seconds": 1.66
}
```

196 ms, $0.00008.

### Fewer, surer cuts

```json
{
  "model": "uhm-1",
  "input": "https://labs.greatwork.company/examples/filler-update.m4a",
  "bias": "precision"
}
```

```json
{
  "fillers": [
    {
      "kind": "filled",
      "text": "um",
      "start": 0.62,
      "end": 1.1,
      "confidence": 0.99,
      "source": "audio"
    },
    {
      "kind": "filled",
      "text": "uh",
      "start": 1.92,
      "end": 2.42,
      "confidence": 0.98,
      "source": "audio"
    },
    {
      "kind": "filled",
      "text": "uh",
      "start": 9.46,
      "end": 9.96,
      "confidence": 0.99,
      "source": "audio"
    },
    {
      "kind": "filled",
      "text": "um",
      "start": 25.2,
      "end": 25.42,
      "confidence": 0.93,
      "source": "audio"
    },
    {
      "kind": "filled",
      "text": "um",
      "start": 27.24,
      "end": 27.58,
      "confidence": 0.98,
      "source": "audio"
    },
    {
      "kind": "filled",
      "text": "uh",
      "start": 28.54,
      "end": 29,
      "confidence": 0.97,
      "source": "audio"
    },
    {
      "kind": "filled",
      "text": "uh",
      "start": 36.08,
      "end": 36.56,
      "confidence": 0.99,
      "source": "audio"
    }
  ],
  "cuts": [
    {
      "start": 0.59,
      "end": 1.13
    },
    {
      "start": 1.89,
      "end": 2.45
    },
    {
      "start": 9.43,
      "end": 9.99
    },
    {
      "start": 25.17,
      "end": 25.45
    },
    {
      "start": 27.21,
      "end": 27.61
    },
    {
      "start": 28.51,
      "end": 29.03
    },
    {
      "start": 36.05,
      "end": 36.59
    }
  ],
  "count": 7,
  "counts": {
    "filled": 7
  },
  "per_minute": 11.2,
  "duration": 37.5,
  "removed_seconds": 3.4
}
```

326 ms, $0.00019.

## Good for

- Clean up podcast and video edits
- Tighter transcripts and captions
- Speaking coaches: fillers per minute
- Pre-cut interview recordings before editing

## Limits and caveats

- English first: the audio check is tuned on English um, uh and hmm, and the transcript rules are English.
- "like", "you know", repeats and false starts need `transcript: true` (or your own `words`).
- Over the phone or far from the mic it misses more fillers.
- The cut list is a starting point: listen to the edit, a few real words that sound like fillers can be marked.
