Great Work Labs
Models / Audio

Clips beta

Highlight clips from long recordings
clips-1

The best 15-60 second clips of a podcast, call or talk, ranked, with reasons.

Price$0.0001 per minute of transcript
Speed (our servers)10 to 51557 ms
LanguagesMeasured in English. Audio in about 100 languages; the cues it reads (questions, hooks, short replies) are English, so other languages get less from them.
InputAudio, up to 5 min, or a transcript up to 6 hours

Clips reads a long recording and returns the moments worth cutting into shorts: start and end times, a score, the words, and plain reasons (the strongest line, whether it opens at the start of a thought and ends on a full sentence). Clips never overlap and come back best first.

Send audio or video and we transcribe it with Scribe, word timings included. Or send your own transcript: Scribe JSON, SRT, WebVTT, [{"text", "start", "end"}], or plain text (times are then estimated from the word count). A transcript costs a twelfth of audio.

In our tests on the AMI meeting test set (20 meetings, scored against human summaries), on average 75% of the top clip was key content, and 62% of the top five, against 33% for the recording as a whole.

Use it

curl https://labs.greatwork.company/v1/run \
  -H "Authorization: Bearer $GWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "clips-1",
  "input": "https://labs.greatwork.company/examples/podcast.m4a",
  "top_k": 3
}'

Set GWL_API_KEY to a key from your dashboard. The bare name clips works too and follows new versions.

Options

input: An audio or video file (https URL, data URL or multipart upload; MP3, WAV, M4A, MP4, WebM and more, up to 25 MB and 5 minutes), or a transcript as a string: Scribe JSON, SRT, WebVTT, a JSON array of {text, start, end}, or plain text, covering up to 6 hours.

NameTypeDefaultWhat it does
top_kinteger10Most clips to return (short recordings return fewer).
min_secondsnumber15Shortest clip, in seconds.
max_secondsnumber60Longest clip, in seconds.
stylestring
informative | lively
"informative"informative favors the key points. lively favors quick back-and-forth, questions, exclamations and laughter.
languagestringSpoken language for audio input (en, es...). Detected when omitted.

Output

FieldTypeMeaning
clipsobject[]Best first: rank, start, end, duration (seconds), score (0 to 1, the expected share of the clip that is key content), percentile (0 to 1, against every possible clip in this recording), text, reasons (strings) and sentences ([first, last] index).
durationnumberRecording length in seconds.
sentencesintegerSentences the transcript was split into.
sourcestringaudio (we transcribed it) or transcript.
languagestringDetected or given language (audio input).
estimated_timesbooleanTrue when a plain-text transcript had no times and they were estimated at 150 words a minute.

Every response wraps this in {"id", "model", "output", "usage": {"units", "unit", "cost_usd"}, "latency_ms"}.

Examples

Podcast

{
  "model": "clips-1",
  "input": "/examples/podcast.m4a",
  "top_k": 3
}
{
  "clips": [
    {
      "rank": 1,
      "start": 21.54,
      "end": 47,
      "duration": 25.46,
      "score": 0.769,
      "percentile": 0.983,
      "text": "People kept asking if they could pay for it, and after a while I ran out of excuses to say no. And then you rented a kitchen? Yes, a shared kitchen across town, we paid by the hour, so every loaf had a clock running on it. Okay, let's get to the hard part, what happened that week? Our oven broke on a Tuesday, the repair quote was $11,000 and we had about $4,000 in the bank.",
      "reasons": [
        "Strongest line: \"Yes, a shared kitchen across town, we paid by the hour, so every loaf had a clock running on it.\"",
        "Dense: 5 of 5 sentences carry key points",
        "Stays on one topic",
        "Has concrete numbers"
      ],
      "sentences": [
        4,
        8
      ]
    },
    {
      "rank": 2,
      "start": 0.28,
      "end": 21.26,
      "duration": 20.98,
      "score": 0.767,
      "percentile": 0.977,
      "text": "Welcome back to small shop notes, today we are talking about the week we almost shut down the bakery. Thanks for having me, it still feels strange to say that out loud. So let's start at the beginning, how did you get into baking at all? Honestly, by accident, I was working in an office and I started bringing bread in on Fridays.",
      "reasons": [
        "Strongest line: \"Honestly, by accident, I was working in an office and I started bringing bread in on Fridays.\"",
        "Dense: 4 of 4 sentences carry key points",
        "Stays on one topic",
        "Starts at the beginning of a thought"
      ],
      "sentences": [
        0,
        3
      ]
    },
    {
      "rank": 3,
      "start": 97.68,
      "end": 126.88,
      "duration": 29.2,
      "score": 0.685,
      "percentile": 0.717,
      "text": "Mostly, I come in at 5, I check the proofing, and I drink too much coffee. Any plans for the next year? We are looking at a second location, but only if the numbers work, I don't trust my gut on money anymore. That seems like a good place to stop, where can people find you? We're on the corner of Elman Fifth, and we open at 7, come early, the bread goes fast. Thanks so much for coming on, this was great. Thank you, it was fun.",
      "reasons": [
        "Strongest line: \"Mostly, I come in at 5, I check the proofing, and I drink too much coffee.\"",
        "Dense: 7 of 7 sentences carry key points",
        "Starts at the beginning of a thought",
        "Ends on a complete sentence",
        "Has concrete numbers"
      ],
      "sentences": [
        19,
        25
      ]
    }
  ],
  "duration": 127.55,
  "sentences": 26,
  "source": "audio",
  "language": "en"
}

Captured on our production servers: 48216 ms, 36 minutes, $0.0036.

Meeting transcript (SRT)

{
  "model": "clips-1",
  "input": "1\n00:00:00,500 --> 00:00:05,900\nOkay, let's get started. This is the weekly review for the mobile app.\n\n2\n00:00:06,250 --> 00:00:10,112\nFirst item is the onboarding numbers from last week.\n\n3\n00:00:10,462 --> 00:00:13,554\nWe had about twelve thousand new installs.\n\n4\n00:00:13,904 --> 00:00:17,381\nOnly thirty one percent of them finished onboarding.\n\n5\n00:00:17,731 --> 00:00:21,208\nThat's down from forty percent the week before.\n\n6\n00:00:22,108 --> 00:00:24,815\nDo we know why it dropped?\n\n7\n00:00:25,715 --> 00:00:30,346\nI think so. We added the account step before the tour.\n\n8\n00:00:30,696 --> 00:00:35,712\nPeople hit a sign-up form before they've seen anything the app does.\n\n9\n00:00:36,062 --> 00:00:40,308\nSo the fix is to move sign-up after the tour?\n\n10\n00:00:41,208 --> 00:00:46,223\nYes, show the three tour screens first, then ask for the account.\n\n11\n00:00:47,123 --> 00:00:52,523\nWe tested that order in March and completion went up by nine points.\n\n12\n00:00:52,873 --> 00:00:55,581\nOkay, let's do that this sprint.\n\n13\n00:00:55,931 --> 00:00:57,869\nNext item, crash reports.\n\n14\n00:00:58,219 --> 00:01:04,004\nThe crash rate on Android went from point two percent to point eight percent.\n\n15\n00:01:04,354 --> 00:01:10,138\nAlmost all of it is one bug in the photo picker on older phones.\n\n16\n00:01:11,038 --> 00:01:13,746\nHow many users does that hit?\n\n17\n00:01:14,646 --> 00:01:18,508\nAbout four thousand a day, mostly on Android eleven.\n\n18\n00:01:18,858 --> 00:01:23,104\nThat's the top priority then, ahead of the onboarding change.\n\n19\n00:01:23,454 --> 00:01:27,700\nAgreed. I can have a fix in review by Wednesday.\n\n20\n00:01:28,050 --> 00:01:30,758\nGreat. Last thing, the pricing test.\n\n21\n00:01:31,658 --> 00:01:36,673\nWe showed half the new users a yearly plan at forty dollars.\n\n22\n00:01:37,023 --> 00:01:40,500\nYearly sign-ups tripled, and monthly sign-ups barely moved.\n\n23\n00:01:40,850 --> 00:01:45,481\nSo total revenue per new user went up about eighteen percent.\n\n24\n00:01:45,831 --> 00:01:49,308\nThat's the best result we've had all year.\n\n25\n00:01:49,658 --> 00:01:52,750\nShould we roll it out to everyone?\n\n26\n00:01:53,650 --> 00:01:59,435\nLet's give it one more week to be sure it holds, then ship it.\n\n27\n00:01:59,785 --> 00:02:05,954\nOkay. So the plan is the crash fix first, then onboarding, and pricing next week.\n\n28\n00:02:06,304 --> 00:02:08,627\nAnything else? No? Thanks, everyone.",
  "top_k": 3
}
{
  "clips": [
    {
      "rank": 1,
      "start": 97.02,
      "end": 128.63,
      "duration": 31.6,
      "score": 0.769,
      "percentile": 1,
      "text": "Yearly sign-ups tripled, and monthly sign-ups barely moved. So total revenue per new user went up about eighteen percent. That's the best result we've had all year. Should we roll it out to everyone? Let's give it one more week to be sure it holds, then ship it. Okay. So the plan is the crash fix first, then onboarding, and pricing next week. Anything else? No? Thanks, everyone.",
      "reasons": [
        "Strongest line: \"Okay. So the plan is the crash fix first, then onboarding, and pricing next week.\"",
        "Dense: 7 of 7 sentences carry key points",
        "Ends on a complete sentence"
      ],
      "sentences": [
        21,
        27
      ]
    },
    {
      "rank": 2,
      "start": 13.9,
      "end": 35.71,
      "duration": 21.81,
      "score": 0.739,
      "percentile": 0.96,
      "text": "Only thirty one percent of them finished onboarding. That's down from forty percent the week before. Do we know why it dropped? I think so. We added the account step before the tour. People hit a sign-up form before they've seen anything the app does.",
      "reasons": [
        "Strongest line: \"People hit a sign-up form before they've seen anything the app does.\"",
        "Dense: 5 of 5 sentences carry key points",
        "Stays on one topic"
      ],
      "sentences": [
        3,
        7
      ]
    },
    {
      "rank": 3,
      "start": 41.21,
      "end": 64,
      "duration": 22.8,
      "score": 0.738,
      "percentile": 0.955,
      "text": "Yes, show the three tour screens first, then ask for the account. We tested that order in March and completion went up by nine points. Okay, let's do that this sprint. Next item, crash reports. The crash rate on Android went from point two percent to point eight percent.",
      "reasons": [
        "Strongest line: \"The crash rate on Android went from point two percent to point eight percent.\"",
        "Dense: 5 of 5 sentences carry key points",
        "Stays on one topic",
        "Starts at the beginning of a thought"
      ],
      "sentences": [
        9,
        13
      ]
    }
  ],
  "duration": 128.63,
  "sentences": 28,
  "source": "transcript"
}

Captured on our production servers: 10 ms, 3 minutes, $0.0003.

Lively style

{
  "model": "clips-1",
  "input": "/examples/podcast.m4a",
  "top_k": 3,
  "style": "lively"
}
{
  "clips": [
    {
      "rank": 1,
      "start": 83.86,
      "end": 104.92,
      "duration": 21.06,
      "score": 1,
      "percentile": 1,
      "text": "That is a huge swing, what would you tell someone starting out today? Know your cost per item before you know your logo, most people do it the other way around. Let switch gears for a second, do you still bake every morning? Mostly, I come in at 5, I check the proofing, and I drink too much coffee. Any plans for the next year?",
      "reasons": [
        "Strongest line: \"Mostly, I come in at 5, I check the proofing, and I drink too much coffee.\"",
        "Dense: 5 of 5 sentences carry key points",
        "Opens mid-thought: consider starting a sentence earlier",
        "Opens with a question",
        "Has concrete numbers"
      ],
      "sentences": [
        16,
        20
      ]
    },
    {
      "rank": 2,
      "start": 21.54,
      "end": 39.68,
      "duration": 18.14,
      "score": 0.978,
      "percentile": 0.983,
      "text": "People kept asking if they could pay for it, and after a while I ran out of excuses to say no. And then you rented a kitchen? Yes, a shared kitchen across town, we paid by the hour, so every loaf had a clock running on it. Okay, let's get to the hard part, what happened that week?",
      "reasons": [
        "Strongest line: \"Yes, a shared kitchen across town, we paid by the hour, so every loaf had a clock running on it.\"",
        "Dense: 4 of 4 sentences carry key points",
        "Stays on one topic"
      ],
      "sentences": [
        4,
        7
      ]
    },
    {
      "rank": 3,
      "start": 52.84,
      "end": 69.88,
      "duration": 17.04,
      "score": 0.892,
      "percentile": 0.936,
      "text": "When I finally did the math, the proessongs we were famous for were losing 40 cents each. Wait, your most popular item was losing money? Every single one, we were working harder to lose more, that's why the busy weeks always felt worse. So what did you change?",
      "reasons": [
        "Strongest line: \"When I finally did the math, the proessongs we were famous for were losing 40 cents each.\"",
        "Dense: 3 of 4 sentences carry key points",
        "Stays on one topic",
        "Has concrete numbers"
      ],
      "sentences": [
        10,
        13
      ]
    }
  ],
  "duration": 127.55,
  "sentences": 26,
  "source": "audio",
  "language": "en"
}

Captured on our production servers: 51557 ms, 36 minutes, $0.0036.

Good for

  • Cut a podcast episode into shorts
  • Highlight reels from webinars and talks
  • Quotable moments in an interview
  • Skim the important parts of a long meeting

Limits and caveats

  • The default style finds informative moments, not jokes, laughter or heated exchanges: style: "lively" leans toward those, but it's only a little better than chance at them in our tests.
  • The cues it reads (questions, hook phrases, short replies) are English.
  • Clip edges follow sentence ends; a clip can still open on a word like "it" that points back, and reasons says so.
  • Audio up to 5 minutes per call. For longer recordings, transcribe in parts with Scribe and send the joined transcript (up to 6 hours).
  • Talk only: music, sports and on-screen action aren't scored.

Pricing

$0.0001 per minute of transcript. Per started minute of the recording, at least one: a transcript you send is 1 unit ($0.0001) a minute; audio or video is 12 units ($0.0012) a minute, transcription included. Failed calls aren't charged, and a call never costs more than its estimate (POST /v1/estimate tells you before you run it).