API & MCP

Everything a person can do on Structured Page Extractor, a program or AI agent can do with the same inputs and get the identical result.

HTTP API

Structured Page Extractor

Reads one public HTML page (or HTML you paste) with a small recipe of CSS selectors, text/attribute/regex rules and an optional same-site next-page link, and returns a table as JSON and CSV. Compares with an earlier run to show new, removed and changed rows, and suggests replacement selectors when a site changed. robots.txt is checked first; no JavaScript, cookies or logins.

Inputs of Structured Page Extractor
FieldTypeDefaultDescription
urltext—Public page address. http(s) only. robots.txt is checked first; pages behind logins, paywalls or bot checks are not read.
htmltextarea—Or paste the page HTML. Used instead of the address; nothing is fetched.
baseUrltext—Address of the pasted page (for relative links)
recipetextarea"{\n \"mode\": \"list\",\n \"item\": \"li.notice\",\n \"fields\": [\n {\n Recipe (JSON). JSON: {"mode":"list","item":"li.notice","fields":[{"name":"title","selector":"a"},{"name":"link","selector":"a::attr(href)","type":"url"},{"name":"price","selector":".price","type":"number"}],"next":"a[rel=next]","maxPages":2,"key":"link"}. Field options: take text|own-text|attr (+ attr), regex (first group kept), type text|number|url, all (join every match).
previoustextarea—Earlier run to compare (JSON export or CSV, optional)
returnHtmlbooleanfalseInclude the first page’s HTML in the result (up to 1 MB, for a live preview)

Example request

Notices list from pasted HTML

curl -X POST https://obscuriqpageextractor.usefulweb.app/api/v1/calculate -H 'content-type: application/json' -d '{"html":"<!doctype html><html lang=\"en\"><head><title>Northwind Notices — public register</title></head><body>\n<main><h1>Public notices</h1>\n<ul class=\"notices\">\n<li class=\"notice\"><a class=\"title\" href=\"/notices/101\">Road works on Harbour Street</a><span class=\"date\">2026-11-02</span><span class=\"fee\">€ 1.250,00</span></li>\n<li class=\"notice\"><a class=\"title\" href=\"/notices/102\">New opening hours: library</a><span class=\"date\">2026-11-05</span><span class=\"fee\">free</span></li>\n<li class=\"notice featured\"><a class=\"title\" href=\"/notices/103\">Tender: school roof repair</a><span class=\"date\">2026-11-09</span><span class=\"fee\">€ 84.000</span></li>\n</ul>\n<nav class=\"pager\"><a rel=\"next\" href=\"/notices?page=2\">Next</a></nav></main></body></html>","baseUrl":"https://notices.example/notices","recipe":"{\"mode\":\"list\",\"item\":\"li.notice\",\"fields\":[{\"name\":\"title\",\"selector\":\"a.title\",\"take\":\"text\",\"type\":\"text\"},{\"name\":\"link\",\"selector\":\"a.title\",\"take\":\"attr\",\"attr\":\"href\",\"type\":\"url\"},{\"name\":\"date\",\"selector\":\".date\",\"take\":\"text\",\"type\":\"text\",\"regex\":\"(\\\\d{4}-\\\\d{2}-\\\\d{2})\"},{\"name\":\"fee (EUR)\",\"selector\":\".fee\",\"take\":\"text\",\"type\":\"number\"}],\"next\":\"a[rel=next]\",\"maxPages\":1,\"key\":\"link\"}"}'

Every result contains

inputs (normalised, defaults applied), summary, result (validated against the published schema), units, assumptions, sources, warnings, computedAt and links.

MCP (Model Context Protocol)

{
  "mcpServers": {
    "obscuriq-page-extractor": { "type": "http", "url": "https://obscuriqpageextractor.usefulweb.app/mcp" }
  }
}