Shapes turns a hand-drawn sketch into a clean shape. Send the strokes from a pen, finger or mouse and get back what was drawn (line, curve, arrow, rectangle, square, ellipse, circle, triangle, diamond, pentagon, hexagon, octagon, star, heart, cloud or scribble) with tidy geometry: corners, center, size, rotation, and an outline you can draw as is.
When a drawing isn't a shape, shape is null, so you keep the ink as drawn. Coordinates come back in your own canvas space. It scored 94.6% on Quick, Draw! single-stroke shapes in our tests, at about 2 ms a drawing. You can also send a picture of a sketch instead of strokes.
Use it
curl https://labs.greatwork.company/v1/run \
-H "Authorization: Bearer $GWL_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "shapes-1",
"input": [[[1242,409],[1197,436],[1152,464],[1108,492],[1065,522],[1022,551],[975,574],[932,548],[902,505],[873,462],[843,418],[816,374],[827,325],[868,293],[911,263],[954,232],[997,202],[1040,172],[1082,142],[1126,112],[1169,82],[1212,52],[1256,24],[1306,21],[1343,57],[1374,100],[1405,142],[1436,184],[1460,230],[1439,275],[1397,306],[1354,336],[1311,366],[1267,395],[1224,424],[1179,452]]]
}'const res = await fetch("https://labs.greatwork.company/v1/run", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.GWL_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
"model": "shapes-1",
"input": [[[1242,409],[1197,436],[1152,464],[1108,492],[1065,522],[1022,551],[975,574],[932,548],[902,505],[873,462],[843,418],[816,374],[827,325],[868,293],[911,263],[954,232],[997,202],[1040,172],[1082,142],[1126,112],[1169,82],[1212,52],[1256,24],[1306,21],[1343,57],[1374,100],[1405,142],[1436,184],[1460,230],[1439,275],[1397,306],[1354,336],[1311,366],[1267,395],[1224,424],[1179,452]]]
}),
});
const { output, usage } = await res.json();import os, requests
res = requests.post(
"https://labs.greatwork.company/v1/run",
headers={"Authorization": f"Bearer {os.environ['GWL_API_KEY']}"},
json={
"model": "shapes-1",
"input": [[[1242,409],[1197,436],[1152,464],[1108,492],[1065,522],[1022,551],[975,574],[932,548],[902,505],[873,462],[843,418],[816,374],[827,325],[868,293],[911,263],[954,232],[997,202],[1040,172],[1082,142],[1126,112],[1169,82],[1212,52],[1256,24],[1306,21],[1343,57],[1374,100],[1405,142],[1436,184],[1460,230],[1439,275],[1397,306],[1354,336],[1311,366],[1267,395],[1224,424],[1179,452]]]
},
)
output = res.json()["output"]Set GWL_API_KEY to a key from your dashboard. The bare name shapes works too and follows new versions.
Options
input: One drawing: an array of strokes, each an array of [x, y] points (or a single stroke, or {"strokes": [...]}), as JSON or a JSON string. For a batch, an array of up to 64 such JSON strings. Or an image of a sketch: an https URL, a data URL or a file in a multipart upload. Up to 200 strokes and 20,000 points per drawing.
| Name | Type | Default | What it does |
|---|---|---|---|
rotation | string snapped | upright | free | "snapped" | snapped rounds angles to 15° steps and straightens shapes within 5° of level, upright keeps shapes level, free keeps the angle as drawn (and never turns a near-square into a square). |
kinds | string[] line | curve | arrow | rectangle | square | ellipse | circle | triangle | diamond | pentagon | hexagon | octagon | star | heart | cloud | scribble | Only recognize these shapes (square means rectangle, circle means ellipse). | |
min_confidence | number | Keep the ink (shape null) below this confidence. By default each shape has its own tuned cutoff. |
Output
| Field | Type | Meaning |
|---|---|---|
shape | string | What was drawn: line, curve, arrow, rectangle, square, ellipse, circle, triangle, diamond, pentagon, hexagon, octagon, star, heart, cloud or scribble. Null when it isn't a shape: keep the ink. |
kind | string | The shape family (rectangle for squares, ellipse for circles). |
confidence | number | 0 to 1. |
candidates | object[] | Top 3 guesses with shape and score. |
closed | boolean | True for closed outlines. |
outline | number[][] | The clean shape as [x, y] points, ready to draw. |
center | number[] | Rectangles, ellipses, stars and polygons. |
width | number | Rectangles (with height, rotation in degrees and corners). |
rx | number | Ellipses (with ry, rotation, and radius for circles). |
corners | number[][] | Rectangles, triangles, diamonds and polygons. |
tips | number[][] | Stars (with points, outer_radius, inner_radius, rotation). |
from | number[] | Lines, curves and arrows, with to. Lines and arrows add length and angle; curves add controls. |
arrowheads | object[] | Arrows: each head, and heads (end or both). |
image | object | Image input only: its width and height; coordinates are in its pixels. |
Every response wraps this in {"id", "model", "output", "usage": {"units", "unit", "cost_usd"}, "latency_ms"}. Send an array as input and output is an array in the same order.
Examples
Rectangle
{
"model": "shapes-1",
"input": [
[
[
1242,
409
],
[
1197,
436
],
[
1152,
464
],
[
1108,
492
],
[
1065,
522
],
[
1022,
551
],
[
975,
574
],
[
932,
548
],
[
902,
505
],
[
873,
462
],
[
843,
418
],
[
816,
374
],
[
827,
325
],
[
868,
293
],
[
911,
263
],
[
954,
232
],
[
997,
202
],
[
1040,
172
],
[
1082,
142
],
[
1126,
112
],
[
1169,
82
],
[
1212,
52
],
[
1256,
24
],
[
1306,
21
],
[
1343,
57
],
[
1374,
100
],
[
1405,
142
],
[
1436,
184
],
[
1460,
230
],
[
1439,
275
],
[
1397,
306
],
[
1354,
336
],
[
1311,
366
],
[
1267,
395
],
[
1224,
424
],
[
1179,
452
]
]
]
}{
"shape": "rectangle",
"kind": "rectangle",
"confidence": 0.942,
"center": [
1136.2,
293.55
],
"width": 605.79,
"height": 304.83,
"rotation": -30,
"corners": [
[
797.68,
313
],
[
1322.31,
10.1
],
[
1474.73,
274.09
],
[
950.1,
576.99
]
],
"outline": [
[
797.68,
313
],
[
1322.31,
10.1
],
[
1474.73,
274.09
],
[
950.1,
576.99
]
],
"closed": true,
"candidates": [
{
"shape": "rectangle",
"score": 0.942
},
{
"shape": "ellipse",
"score": 0.02
},
{
"shape": "hexagon",
"score": 0.016
}
]
}Captured on our production servers: 16 ms, 1 drawing, $0.000005.
Circle
{
"model": "shapes-1",
"input": [
[
[
228,
544
],
[
237,
565
],
[
250,
584
],
[
266,
602
],
[
286,
616
],
[
309,
626
],
[
333,
633
],
[
356,
637
],
[
381,
637
],
[
405,
632
],
[
428,
623
],
[
449,
612
],
[
469,
597
],
[
484,
578
],
[
497,
557
],
[
506,
534
],
[
511,
510
],
[
510,
485
],
[
507,
461
],
[
500,
438
],
[
489,
416
],
[
475,
396
],
[
457,
379
],
[
437,
366
],
[
416,
357
],
[
392,
351
],
[
368,
350
],
[
344,
354
],
[
319,
360
],
[
298,
371
],
[
277,
385
],
[
259,
400
],
[
245,
417
],
[
232,
439
],
[
223,
463
],
[
212,
463
]
]
]
}{
"shape": "circle",
"kind": "ellipse",
"confidence": 0.93,
"center": [
366.34,
494.37
],
"rx": 144.3,
"ry": 144.3,
"rotation": 0,
"radius": 144.3,
"outline": [
[
510.63,
494.37
],
[
509.94,
508.51
],
[
507.86,
522.52
],
[
504.42,
536.25
],
[
499.65,
549.59
],
[
493.6,
562.39
],
[
486.32,
574.53
],
[
477.88,
585.91
],
[
468.37,
596.4
],
[
457.88,
605.91
],
[
446.5,
614.34
],
[
434.36,
621.62
],
[
421.56,
627.68
],
[
408.23,
632.45
],
[
394.49,
635.89
],
[
380.48,
637.97
],
[
366.34,
638.66
],
[
352.19,
637.97
],
[
338.19,
635.89
],
[
324.45,
632.45
],
[
311.12,
627.68
],
[
298.32,
621.62
],
[
286.17,
614.34
],
[
274.8,
605.91
],
[
264.3,
596.4
],
[
254.8,
585.91
],
[
246.36,
574.53
],
[
239.08,
562.39
],
[
233.03,
549.59
],
[
228.25,
536.25
],
[
224.81,
522.52
],
[
222.74,
508.51
],
[
222.04,
494.37
],
[
222.74,
480.22
],
[
224.81,
466.21
],
[
228.25,
452.48
],
[
233.03,
439.15
],
[
239.08,
426.34
],
[
246.36,
414.2
],
[
254.8,
402.82
],
[
264.3,
392.33
],
[
274.8,
382.82
],
[
286.17,
374.39
],
[
298.32,
367.11
],
[
311.12,
361.05
],
[
324.45,
356.28
],
[
338.19,
352.84
],
[
352.19,
350.76
],
[
366.34,
350.07
],
[
380.48,
350.76
],
[
394.49,
352.84
],
[
408.23,
356.28
],
[
421.56,
361.05
],
[
434.36,
367.11
],
[
446.5,
374.39
],
[
457.88,
382.82
],
[
468.37,
392.33
],
[
477.88,
402.82
],
[
486.32,
414.2
],
[
493.6,
426.34
],
[
499.65,
439.15
],
[
504.42,
452.48
],
[
507.86,
466.21
],
[
509.94,
480.22
]
],
"closed": true,
"candidates": [
{
"shape": "ellipse",
"score": 0.93
},
{
"shape": "curve",
"score": 0.01
},
{
"shape": null,
"score": 0.009
}
]
}Captured on our production servers: 2 ms, 1 drawing, $0.000005.
Arrow
{
"model": "shapes-1",
"input": [
[
[
114,
90
],
[
138,
100
],
[
161,
109
],
[
185,
119
],
[
209,
129
],
[
233,
138
],
[
256,
149
],
[
280,
158
],
[
304,
169
],
[
328,
178
],
[
351,
189
],
[
375,
198
],
[
399,
208
],
[
423,
218
],
[
447,
227
],
[
471,
237
],
[
495,
246
],
[
519,
256
],
[
542,
266
],
[
566,
276
],
[
590,
285
],
[
614,
296
],
[
638,
306
],
[
661,
317
],
[
685,
327
],
[
709,
336
]
],
[
[
612,
383
],
[
622,
379
],
[
632,
375
],
[
643,
371
],
[
653,
366
],
[
663,
361
],
[
674,
357
],
[
684,
352
],
[
694,
347
],
[
704,
342
],
[
714,
338
],
[
709,
328
],
[
702,
319
],
[
696,
310
],
[
689,
300
],
[
684,
291
],
[
678,
281
],
[
673,
271
],
[
667,
261
],
[
661,
252
],
[
654,
243
],
[
649,
233
]
]
]
}{
"shape": "arrow",
"kind": "arrow",
"confidence": 0.971,
"from": [
114.08,
89.81
],
"to": [
713.58,
337.46
],
"length": 648.64,
"angle": 22.44,
"heads": "end",
"arrowheads": [
{
"tip": [
713.58,
337.46
],
"left": [
642.23,
244.65
],
"right": [
597.53,
352.85
],
"length": 117.07
}
],
"outline": [
[
114.08,
89.81
],
[
713.58,
337.46
]
],
"closed": false,
"candidates": [
{
"shape": "arrow",
"score": 0.971
},
{
"shape": "line",
"score": 0.004
},
{
"shape": "octagon",
"score": 0.004
}
]
}Captured on our production servers: 4 ms, 1 drawing, $0.000005.
Triangle, free angle
{
"model": "shapes-1",
"input": [
[
[
463,
25
],
[
466,
26
],
[
471,
30
],
[
475,
34
],
[
481,
36
],
[
487,
40
],
[
492,
41
],
[
494,
38
],
[
493,
32
],
[
494,
27
],
[
494,
22
],
[
494,
16
],
[
493,
9
],
[
492,
5
],
[
490,
-1
],
[
488,
-4
],
[
485,
-4
],
[
480,
-4
],
[
475,
-3
],
[
468,
-1
],
[
464,
0
],
[
457,
3
],
[
452,
3
],
[
446,
5
],
[
442,
6
],
[
437,
7
],
[
434,
7
],
[
435,
9
],
[
441,
12
],
[
445,
14
],
[
450,
17
],
[
453,
19
],
[
457,
23
],
[
461,
26
],
[
466,
31
],
[
469,
31
]
]
],
"rotation": "free"
}{
"shape": "triangle",
"kind": "triangle",
"confidence": 0.756,
"center": [
473.24,
15.08
],
"corners": [
[
434.23,
7.9
],
[
491.99,
-6.83
],
[
493.52,
44.18
]
],
"rotation": 40.54,
"outline": [
[
434.23,
7.9
],
[
491.99,
-6.83
],
[
493.52,
44.18
]
],
"closed": true,
"candidates": [
{
"shape": "triangle",
"score": 0.756
},
{
"shape": null,
"score": 0.056
},
{
"shape": "rectangle",
"score": 0.044
}
]
}Captured on our production servers: 95 ms, 1 drawing, $0.000005.
Photo of a sketch
{
"model": "shapes-1",
"input": "/examples/sketch-star.png"
}{
"shape": "star",
"kind": "star",
"confidence": 0.845,
"center": [
189.38,
190.03
],
"points": 6,
"outer_radius": 179.97,
"inner_radius": 120.75,
"rotation": 0,
"tips": [
[
189.38,
10.06
],
[
345.24,
100.04
],
[
345.24,
280.01
],
[
189.38,
370
],
[
33.53,
280.01
],
[
33.53,
100.04
]
],
"outline": [
[
189.38,
10.06
],
[
249.76,
85.45
],
[
345.24,
100.04
],
[
310.14,
190.03
],
[
345.24,
280.01
],
[
249.76,
294.6
],
[
189.38,
370
],
[
129.01,
294.6
],
[
33.53,
280.01
],
[
68.63,
190.03
],
[
33.53,
100.04
],
[
129.01,
85.45
]
],
"closed": true,
"candidates": [
{
"shape": "star",
"score": 0.845
},
{
"shape": "octagon",
"score": 0.041
},
{
"shape": "hexagon",
"score": 0.037
}
],
"image": {
"width": 384,
"height": 384
}
}Captured on our production servers: 370 ms, 10 drawings, $0.00005.
Good for
- Snap whiteboard sketches to clean shapes
- Draw diagrams and flowcharts with a pen or finger
- Shape recognition in note-taking and drawing apps
- Turn photos of sketches into vector outlines
Limits and caveats
- About 1 in 10 doodles that aren't shapes still get read as one: raise
min_confidenceor limitkindswhen that matters. - Octagons are the weakest shape (80% in our tests).
- One shape per drawing: send each shape on a page as its own drawing.
- Images are traced into strokes first, so dark lines on a plain background work best.
Pricing
$0.005 per 1,000 drawings. One unit per drawing sent as strokes; an image of a sketch is 10 units. Failed calls aren't charged, and a call never costs more than its estimate (POST /v1/estimate tells you before you run it).
