Skip to content

Commit 2396680

Browse files
committed
Add fragment on SQLite bulk inserts
1 parent 89b4e98 commit 2396680

2 files changed

Lines changed: 51 additions & 0 deletions

File tree

‎content/atoms/_meta.toml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,12 @@
1010
#
1111
################################################################################
1212

13+
[[atoms]]
14+
published_at = 2026-06-06T13:15:27+02:00
15+
description = """\
16+
Published fragment [SQLite bulk insert with sqlc](/fragments/sqlc-sqlite-bulk-insert), on using `json_each` and `json_extract` to achieve this end.
17+
"""
18+
1319
[[atoms]]
1420
published_at = 2026-06-05T20:18:34+02:00
1521
description = """\
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
+++
2+
hook = "Using `json_each` and `json_extract` to get bulk inserts in SQLite with sqlc."
3+
# image = ""
4+
published_at = 2026-06-06T13:15:27+02:00
5+
title = "SQLite bulk insert with sqlc"
6+
+++
7+
8+
One of [sqlc](/sqlc)'s long-standing quirks is that it doesn't support `INSERT INTO table ( ... ) VALUES ( ... )` with multiple tuples. This forces you to find a workaround where you can inject multiple values as a single value, like using an array in Postgres and `unnest`ing it in SQL.
9+
10+
Back when I first added SQLite support to River, I had a *really* hard time finding a way to do this. Not so much because SQLite didn't support it (though it doesn't have arrays), but rather due to a number of longstanding bugs in sqlc that prevented the use of [table-valued functions](https://sqlite.org/vtab.html#tabfunc2) like `json_each`. I ended up giving up on bulk insert in SQLite and looping over single insert operations instead.
11+
12+
But that was over a year ago. Happily, I took a look at this again today and found the sqlc bugs fixed.
13+
14+
To insert a series of single values, inject a JSON array of scalars as a binary blob, then iterate over it with `json_each` and select `value`:
15+
16+
``` sql
17+
-- name: RiverMigrationInsertMany :many
18+
INSERT INTO /* TEMPLATE: schema */river_migration (
19+
line,
20+
version
21+
)
22+
SELECT
23+
@line,
24+
value
25+
FROM json_each(cast(@versions AS blob))
26+
RETURNING *;
27+
```
28+
29+
To insert a series of multiple values, inject a JSON array of objects as a binary blob, then iterate over it with `json_each` and extract each value with `json_extract`:
30+
31+
``` sql
32+
-- name: NotificationInsertMany :exec
33+
INSERT INTO /* TEMPLATE: schema */river_notification (
34+
payload,
35+
topic
36+
)
37+
SELECT
38+
json_extract(value, '$.payload'),
39+
json_extract(value, '$.topic')
40+
FROM json_each(cast(@notifications AS blob));
41+
```
42+
43+
If using `jsonb` instead of `json`, you'll use `jsonb_each` and `jsonb_extract` instead.
44+
45+
I'm not sure about the use of sqlc anymore these days -- it seems to have largely fallen out of maintenance, and isn't quite as necessary in the LLM age. In River, we haven't had a strong reason to migrate off it yet, so for now we're holding the line.

0 commit comments

Comments
 (0)