A SQLite extension for making HTTP requests purely in SQL.
- Create GET, POST, and other HTTP requests, like
curl,wget, andfetch - Download response bodies, header, status codes, timing info
- Set rate limits, timeouts
.load ./http0 select http_get_body('https://text.npr.org/'); /* <!DOCTYPE html> <html lang="en"> <head> <title>NPR : National Public Radio</title> .... */Query for all custom headers in an endpoint.
select name, value from http_headers_each( http_get_headers('https://api.github.com/') ) where name like 'X-%'; /* ┌────────────────────────┬────────────────────────────────────┐ │ name │ value │ ├────────────────────────┼────────────────────────────────────┤ │ X-Ratelimit-Limit │ 60 │ │ X-Ratelimit-Used │ 8 │ │ X-Content-Type-Options │ nosniff │ │ X-Github-Media-Type │ github.v3; format=json │ │ X-Github-Request-Id │ CCCA:5FDF:1014BC2:10965F9:62F3DE4E │ │ X-Ratelimit-Remaining │ 52 │ │ X-Ratelimit-Resource │ core │ │ X-Frame-Options │ deny │ │ X-Ratelimit-Reset │ 1660152798 │ │ X-Xss-Protection │ 0 │ └────────────────────────┴────────────────────────────────────┘ */Scrape data from a JSON endpoint.
select http_get_body('https://api.github.com/repos/sqlite/sqlite') ->> '$.description' as description; /* ┌───────────────────────────────────────────────┐ │ description │ ├───────────────────────────────────────────────┤ │ Official Git mirror of the SQLite source tree │ └───────────────────────────────────────────────┘ */Pass in specific headers in a request.
select value from json_each( http_get_body( 'https://api.github.com/issues', http_headers( 'Authorization', 'token ghp_16C7e42F292c6912E7710c8' ) ) ); See docs.md for a full API reference.
| Language | Install | |
|---|---|---|
| Python | pip install sqlite-http |  |
| Datasette | datasette install datasette-sqlite-http |  |
| Node.js | npm install sqlite-http |  |
| Deno | deno.land/x/sqlite_http |  |
| Ruby | gem install sqlite-http |  |
| Github Release |  |
The Releases page contains pre-built binaries for Linux amd64, MacOS amd64 (no arm), and Windows.
If you want to use sqlite-http as a Runtime-loadable extension, Download the http0.dylib (for MacOS), http0.so (Linux), or http0.dll (Windows) file from a release and load it into your SQLite environment.
Note: The
0in the filename (http0.dylib/http0.so/http0.dll) denotes the major version ofsqlite-http. Currentlysqlite-httpis pre v1, so expect breaking changes in future versions.
For example, if you are using the SQLite CLI, you can load the library like so:
.load ./http0 select http_version(); -- v0.0.1Or in Python, using the builtin sqlite3 module:
import sqlite3 con = sqlite3.connect(":memory:") con.enable_load_extension(True) con.load_extension("./http0") print(con.execute("select http_version()").fetchone()) # ('v0.0.1',)Or in Node.js using better-sqlite3:
const Database = require("better-sqlite3"); const db = new Database(":memory:"); db.loadExtension("./http0"); console.log(db.prepare("select http_version()").get()); // { 'http_version()': 'v0.0.1' }Or with Datasette, with the "no network" option to limit DDoS attacks:
datasette data.db --load-extension ./http0-no-net - sqlite-html, for parsing and querying HTML using CSS selectors in SQLite (pairs great with this tool)
- pgsql-http, a similar yet very different HTTP library for POstgreSQL (didn't know about this before I started this, but interestingly enough came up with a very similar API)
- riyaz-ali/sqlite, the brilliant Go library that this library depends on
- nalgeon/sqlean, several pre-compiled handy SQLite functions, in C