
documents-to-data
io.github.owlabee/documents-to-data
Turn PDFs, documents, podcasts and drone photos into Markdown, structured JSON, summaries, reports.
Documentation
owlabee
owlabee publishes 20 pay-per-result Apify Actors that turn public data sources (permit portals, government and procurement APIs, court feeds, license registries, DNS, sitemaps, storefronts) into normalized JSON.
This repo has runnable examples for each Actor: curl, Python, Node.js and an MCP config for AI agents. Each Actor is billed per result to your own Apify account; see its Store page for current pricing.
Guides to the data sources: owlabee.com
Made by Jacob Hite.
Actors
| Actor | What it returns | Examples |
|---|---|---|
| Building Permits Scraper — Roofing & Construction Leads | Building permits from 120 open-data datasets in 97 US and Canadian cities, counties and states: type, status, dates, cost, address, contractor | examples/building-permits-scraper |
| Accela Citizen Access & EnerGov Permit Scraper (Any City) | Permits from Accela Citizen Access and Tyler EnerGov portals in 31 cities and counties, in the same schema as the open-data scraper | examples/accela-energov-permit-scraper |
| Grants.gov API & SAM.gov Scraper — Federal Grants Feed | Grants.gov grant opportunities and SAM.gov contract notices by keyword, plus optional USAspending awards and six state and local bid datasets | examples/grants-gov-api |
| Grant Matching & RFP Bid/No-Bid Scorer (AI) — Grants.gov | Grants.gov, SAM.gov and USAspending opportunities ranked by fit to your organization, with reasons and disqualifiers | examples/grant-matching |
| Tenders Scraper: EU TED, UK Find a Tender & AusTender | Public tenders and contract award notices from EU TED, UK Find a Tender, UK Contracts Finder and AusTender in one record shape | examples/tenders-scraper |
| PACER RSS Docket Monitor — New Federal Court Filings | New docket entries from the public PACER RSS feeds of federal district, bankruptcy, appellate and national courts, filtered by party, case or entry type | examples/pacer-docket-monitor |
| OSHA Inspections by Company + EPA Enforcement Report | OSHA inspections, violations and penalties and EPA ECHO enforcement cases by company, facility, state or ZIP, plus opt-in MSHA, Wage and Hour, FMCSA and OSHA accident records | examples/osha-epa-inspections |
| Product Recall Feed — FDA, NHTSA & CPSC Recalls API | FDA drug, device and food recalls, NHTSA vehicle recalls and CPSC consumer product recalls in one normalized record, plus opt-in Canada, UK and EU recall feeds | examples/product-recall-feed |
| License Verification API — 7 States + NPI Registry | Professional license records by number, or by name within one board, from 10 boards in 7 states (TX, FL, CO, WA, CT, NJ, MI) and the NPI Registry | examples/license-verification-api |
| SPF, DMARC & DKIM Checker — Bulk Domain WHOIS, DNS & SSL | One record per domain: SPF, DMARC, DKIM, MTA-STS and BIMI with a 0-100 email security score, RDAP WHOIS, DNS, TLS and HTTP headers | examples/spf-dmarc-dkim-checker |
| Website Change Monitor — Sitemap Monitor & Page Diff | Pages discovered through a site's sitemaps, and new, removed, updated and content-changed pages between runs with diff previews | examples/website-change-monitor |
| Shopify Catalog Monitor & Price Tracker — Products Scraper | Every product or variant of a public Shopify store from products.json, or price, stock and new or removed product changes | examples/shopify-catalog-monitor |
| E-commerce Scraper & Price Monitor — WooCommerce, Magento | Products and variants of WooCommerce, Magento, Shopify and other stores (schema.org Product JSON-LD), or price, stock and new or removed product changes | examples/ecommerce-scraper |
| Bluesky Monitor & Scraper — Posts, Keywords, Followers | Bluesky posts by keyword or hashtag, profiles, author feeds, follower lists, post threads and quotes, custom feeds and lists from the public AT Protocol API | examples/bluesky-monitor |
| Hiring Signals & ATS Jobs Scraper — Greenhouse, Lever, Ashby | Open roles from Greenhouse, Lever, Ashby, SmartRecruiters and Workable job boards, or new, removed and changed roles per company | examples/job-postings-api |
| Podcast RSS Scraper — Episodes, Transcripts & Apple Podcasts | Show and episode records from podcast RSS feeds, Apple Podcasts links, Apple charts, directory searches or Podcast Index GUIDs, with Podcasting 2.0 transcripts and chapters | examples/podcast-rss-scraper |
| Podcast Episode Summarizer (AI) — Transcript to Summary | Per-episode abstract, key topics, chapters, quotes, guests and action items from Podcasting 2.0 transcripts or show notes | examples/podcast-summarizer |
| PDF to Markdown for RAG — DOCX, OCR & Chunks | Clean Markdown from PDF, Office, OpenDocument, EPUB, HTML, text, CSV and image URLs, with optional RAG chunks, OCR in 21 languages, table records and images | examples/pdf-to-markdown |
| PDF & Document to JSON Extractor (AI) — Invoices, Forms | One JSON object per PDF, Office, EPUB, HTML or scanned document that matches the JSON Schema you supply, with per-field confidence | examples/pdf-to-json |
| AI Drone Roof Inspection Report — Findings & Repair Scope | Per-photo roof defect findings, per-building condition grade and repair scope, a Markdown report and a contact sheet | examples/drone-roof-inspection-report |
Quick start
-
Create an Apify account and copy your API token from Console → Settings → API & Integrations.
-
Put it in your environment. Never commit it.
export APIFY_TOKEN=<your Apify API token> -
Run any example. Every folder has an
input.jsontaken from that Actor's own quick-start input:cd examples/product-recall-feed ./curl.shOr call the API directly.
run-sync-get-dataset-itemsstarts a run, waits for it (up to 300 seconds) and returns the dataset items:curl -sS -X POST \ "https://api.apify.com/v2/acts/owlabee~product-recall-feed/run-sync-get-dataset-items" \ -H "Authorization: Bearer ${APIFY_TOKEN}" \ -H "Content-Type: application/json" \ -d '{"agencies": ["fda-drug", "nhtsa", "cpsc"], "daysBack": 30, "maxResultsPerSource": 5}'
Python (pip install apify-client) and Node.js (npm install apify-client) versions are in each folder as python.py and node.mjs. To cap spend on a run, set its maximum total charge in the Console or pass maxTotalChargeUsd as a run option.
Use from AI agents (MCP)
Apify hosts an MCP server at mcp.apify.com. Add ?tools= to expose only the Actors you want. This config gives an agent the recall feed and the license lookup:
{
"mcpServers": {
"owlabee": {
"url": "https://mcp.apify.com/?tools=owlabee/product-recall-feed,owlabee/license-verification-api",
"headers": { "Authorization": "Bearer ${APIFY_TOKEN}" }
}
}
}
Claude Code expands ${APIFY_TOKEN} from your environment; Cursor uses ${env:APIFY_TOKEN}. Each examples/<actor>/mcp.json is the same config for one Actor, and its README shows the mcp-remote variant for Claude Desktop.
Use without code (Zapier, Make, n8n)
Apify has official apps on Zapier, Make and n8n, so any owlabee Actor can run from them with your own Apify account. Step-by-step recipes with ready inputs:
- Get new PACER docket entries in Slack with Zapier
- New Grants.gov opportunities to Google Sheets with Zapier
- Building permits to Google Sheets with Make
- Competitor Shopify price changes to Slack with Make
- FDA, NHTSA and CPSC product recalls to email with n8n
- Website change alerts in Slack with n8n
Repository layout
| Path | What it is |
|---|---|
examples/<actor>/ | input.json, curl.sh, python.py, node.mjs, mcp.json and a README per Actor |
docs/ | The static guide site, served by GitHub Pages (generated, plain HTML and one CSS file) |
tools/ | Zero-dependency Node scripts that generate examples/ and docs/, refresh the free data snapshots (tools/data-pages/refresh.mjs) and check links |
To change the site's base URL, edit tools/site-config.mjs, then run node tools/build-examples.mjs && node tools/build-site.mjs and commit the output. Check the result with node tools/check-links.mjs (internal links, anchors, sitemap.xml and llms.txt; add --external to request every outside link once).
Free data and comparison pages
Besides the guides, the site has:
- Free data (owlabee.com/data/): weekly snapshots of product recalls, grant and bid opportunities, and building permits, each made by one or two small runs of our own Actors and published as a page plus
docs/data/<id>.json. Records are cut to an allowlist of fields: no person names or contact details, permits at street level (no house number), and every licensed source's attribution line on its page. Rendered bytools/data-pages.mjs. - Comparisons (owlabee.com/compare/): dated, sourced comparisons of the ways to get permit, PACER alert, recall and grant data, ours included, each saying where the alternatives are better. Content and sources live in
tools/compare-pages.mjs; re-check the numbers when you edit it and update itsCHECKEDdate. llms.txtat the site root: a plain-text map of the Actors and pages for AI assistants, generated with the site.
tools/discovery-pages.mjs registers all of these with tools/build-site.mjs (nav, home-page section, sitemap). The sitemap lists them, so the IndexNow workflow submits them on the next push to main that changes docs/.
Weekly refresh. The mmm coordinator re-runs the snapshots once a week (its daily run does it on Mondays) and lands the result through a PR:
NODE_USE_ENV_PROXY=1 APIFY_TOKEN=<token from the environment> node tools/data-pages/refresh.mjs
node tools/check-links.mjs
The script starts eight small runs on the owlabee Apify account, each capped with maxTotalChargeUsd 0.10, rewrites docs/data/*.json and rebuilds the site; --build-only re-renders from the saved snapshots, and recalls, grants or permits refreshes one. It reads the token from the environment and never prints or stores it. There is deliberately no GitHub Actions workflow for it, since that would need the token as a repository secret.
Notes
- Data comes from public sources as each source publishes it. Coverage, freshness and field completeness vary by source; check the linked source record before relying on a value.
- owlabee is an independent developer on Apify Store. It is not affiliated with or endorsed by Apify, or by any agency, court, registry or company named in this repo.
- Example code is MIT licensed (see LICENSE).
No installation packages available.