Repository avatar
Monitoring
v1.0.0
active

nyc-school-ratings

io.github.biserd/nyc-school-ratings

Search, compare, and read NYC school and early-childhood program profiles from public data.

Documentation

NYC School Ratings

A parent-friendly web dashboard for browsing, comparing, and researching NYC's public, charter, and private schools — backed by real data from the NYC DOE, NYC Open Data, NYSED, and the NCES.

Live at nycschoolsratings.com.

What's inside

  • 1,500+ public elementary schools with academic, climate, progress, and survey scores. Overall score is a weighted blend: Test Proficiency × 40% + Climate × 30% + Progress × 30%.
  • 600+ private schools (NCES PSS data) with filtering by borough, religious affiliation, and coed status.
  • All NYC public high schools with graduation rates, Regents results, multi-year trends, and subgroup analysis (NYC DOE InfoHub).
  • HS admissions & programs — Fall 2025 directory data for 452 high schools and 943 programs (admission methods, demand, eligibility, offer rates, SHSAT stats).
  • NYCEEC Early Childhood Centers for 3-K and Pre-K.
  • K / 3-K / Pre-K admissions & demand metrics from NYC DOE LL72 reports.
  • Attendance & chronic absenteeism (2018-19 → 2024-25) with subgroup breakdowns.
  • Discipline & suspensions (2018-19 → 2024-25, LL93) by type, race, gender, SWD, ELL, and STH.
  • Neighborhood Safety Index — a 0-100 score on every public, private, and NYCEEC school detail page, computed from NYPD complaint data within a configurable radius (0.25 / 0.5 / 1 / 5 miles), severity-weighted, and ranked as a citywide percentile. Free tier shows the score; Premium unlocks radius selector, top categories, trend, and percentile.
  • Interactive map with school zone detection (turf.js + NYC DOE boundary polygons).
  • Side-by-side comparison with shareable, SEO-friendly URLs (e.g. /compare/PS006-M-vs-PS290-M).
  • District comparison dashboard.
  • AI chat assistant powered by Cloudflare Workers AI with GLM-4.7-Flash (Premium).
  • Smart Recommendations ("Find My Match") — a short questionnaire that produces personalized school suggestions.
  • Application Tracker to manage applications across schools (Premium).
  • 3-K / Pre-K Lottery Simulator.
  • Special Education (IEP) support information.
  • Commute Time Calculator (auth-gated; uses Google Distance Matrix).
  • Parent reviews & ratings on every school.
  • Data-driven blog with analytical articles and Recharts visualizations.
  • Favorites that sync across devices for logged-in users.
  • Developer API (Premium) — public REST API at /api/v1 (see below).

Stack

  • Frontend: React 18 + TypeScript, Vite, Tailwind CSS, Shadcn UI, Wouter, TanStack Query, Recharts, Leaflet, turf.js
  • Backend: Cloudflare Workers + Express + TypeScript, Drizzle ORM, PostgreSQL through Hyperdrive, in-memory caching layer with webhook-based invalidation
  • Auth: Email + password with secure session cookies; password reset via signed email tokens; magic-link sign-in
  • Payments: Stripe (Checkout + Customer Portal + webhooks) for the Freemium / Premium subscription model
  • AI: Cloudflare Workers AI with @cf/zai-org/glm-4.7-flash for chat, recommendations, and early-childhood insights
  • Email: Cloudflare Email Service (welcome emails, password reset, magic links, alerts, and drip campaign)
  • Maps & geo: Google Maps Geocoding API, Google Distance Matrix API, Leaflet for rendering, NYC Open Data point locations and zone polygons
  • Data: NYC DOE InfoHub, NYC Open Data, NYSED State Report Card, NCES Private School Universe Survey, NYC School Survey CSVs

Running locally

npm install
cp .dev.vars.example .dev.vars
npm run dev          # Cloudflare Worker + Vite, served on a single origin
npm run db:push      # Push Drizzle schema changes to the configured Postgres

Replace the example values in .dev.vars before starting. Both the API and the SPA are served from the same origin; local email delivery remains disabled until EMAIL_DELIVERY_ENABLED is explicitly changed.

Environment variables

The app reads the following from Worker secrets and variables. The ones marked required are needed for the app to boot; the rest enable specific features.

VariablePurpose
DATABASE_URL (required)PostgreSQL connection string
SESSION_SECRET (required)Express session signing secret
STRIPE_LIVE_SECRET_KEY / STRIPE_TEST_SECRET_KEYStripe server keys (Premium subscriptions)
STRIPE_LIVE_PUBLISHABLE_KEY / STRIPE_TEST_PUBLISHABLE_KEYStripe publishable keys (frontend Checkout)
STRIPE_WEBHOOK_SECRETStripe webhook signature verification
EMAIL_FROM / EMAIL_FROM_NAMECloudflare Email Service sender identity
GOOGLE_MAPS_API_KEYGeocoding + Distance Matrix (commute calc)
SOCRATA_APP_TOKENNYC Open Data / NYPD complaint dataset access
CRON_SECRETProtects POST /api/cron/* endpoints

Project layout

client/src/
├── components/        # Shared UI (AppHeader, FilterBar, ApiAccessCard, …)
│   └── ui/            # Shadcn primitives
├── pages/             # One file per route — registered in App.tsx
├── hooks/             # useAuth, useCheckout, useToast, …
└── lib/               # queryClient, helpers

server/
├── index-worker.ts    # Cloudflare Worker entrypoint + scheduled handlers
├── index-dev.ts       # Dev entrypoint (Vite middleware)
├── index-prod.ts      # Production entrypoint (static SPA)
├── routes.ts          # Main API routes + auth, Stripe webhooks, etc.
├── routesV1.ts        # Public Developer API (/api/v1)
├── apiKeyAuth.ts      # API key generation, hashing, Bearer middleware,
│                      # rate limiter, Premium recheck
├── auth.ts            # Sessions, password hashing, magic links
├── storage.ts         # Drizzle queries (the only place SQL lives)
├── cache.ts           # In-memory cache with TTLs + invalidation
├── services/          # safetyIndex, geocoding, schoolZones, …
└── scripts/           # One-off data import / sync scripts

shared/
└── schema.ts          # Drizzle tables, Zod insert schemas, shared types

scripts/               # CLI data importers (NYSED, HS data, NYCEEC, …)
public/                # Static assets, sitemap.xml, robots.txt

Key conventions:

  • Schema first. All data models live in shared/schema.ts (Drizzle + drizzle-zod insert schemas). Frontend and backend both import from there for type safety.
  • Reviewed D1 migrations. Schema changes live in migrations-d1/. Generate with npm run db:generate:d1, inspect the SQL, then test on staging before applying to production. db:push is intentionally blocked.
  • No SQL outside server/storage.ts. Routes are thin and call into the storage layer.
  • Caching by default. Hot endpoints go through server/cache.ts (centralized TTLs, mutation protection via structuredClone, webhook-based invalidation).

Subscription tiers

The app uses a Freemium model powered by Stripe.

  • Free: browse all schools, see overall scores, save favorites, use the smart recommendations questionnaire, see the Neighborhood Safety score at the default radius, view one premium school detail page in full ("One Free School View").
  • Enrollment Season Pass ($29 one-time / 6 months): AI chat assistant, side-by-side & district comparison, application tracker, full safety drill-down (radius selector, trend, percentile, top categories), commute time calculator, and the Developer API (see below).

Stripe state is kept in sync with the local DB via webhooks, so there is no "stale subscription" drift between Stripe and the app.

SEO

Every page sets a unique title, meta description, canonical URL, Open Graph + Twitter Card tags, and JSON-LD Structured Data where applicable. A static sitemap.xml plus pre-generated comparison and school pages keep the site fully indexable. Performance is optimized with code-splitting, lazy loading for heavy pages, an eager-imported home page so the LCP text is in the initial chunk, deferred analytics, a trimmed Inter webfont, and aggressive static asset caching.

Developer API

A public REST API is available to Premium subscribers at /api/v1. Full documentation and live curl examples live at /developers/docs.

Authentication

All requests require a Bearer token issued from your account:

  1. Purchase the Enrollment Season Pass ($29 for six months; no automatic renewal).
  2. Go to Settings → API Access (/settings#api-access).
  3. Click Generate API Key. The plaintext key is shown exactly once — copy it immediately. Only a SHA-256 hash is stored server-side.
  4. Pass the key in the Authorization header on every request:
curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://nycschoolsratings.com/api/v1/schools?limit=5"

You can have up to 5 active keys at a time, and any key can be revoked instantly from the same settings page.

Endpoints

MethodPathDescription
GET/api/v1/schoolsList NYC schools with filters + pagination (district, grade_band, has_3k, has_prek, has_gifted, limit, offset)
GET/api/v1/schools/:dbnGet a single school by DBN (e.g. 02M545) — flat object
GET/api/v1/districtsAggregate stats for all 32 NYC school districts
GET/api/v1/early-childhoodNYC Early Education Centers (3-K / Pre-K) with borough, program_type, center_type filters
GET/api/v1/trends/:dbnMulti-year ELA / Math / Science score history for a school

All field names use snake_case (e.g. overall_score, ela_proficiency, grade_band). List endpoints return { data: [...], pagination: {...} }; detail endpoints (/schools/:dbn, /trends/:dbn) return a flat object.

Rate limits

Each API key is rate-limited globally (the counters are persisted in Postgres so they survive deploys and span multiple server instances):

  • 60 requests / minute
  • 10,000 requests / day

Every authenticated response includes the standard headers:

X-RateLimit-Limit: 60
X-RateLimit-Remaining: 45
X-RateLimit-Reset: 1699574400

Exceeding either bucket returns 429 Too Many Requests with a Retry-After header.

Unauthenticated requests (missing or invalid Bearer token) are also throttled per IP at 30 / minute to discourage key-guessing.

Abuse monitoring

Every Developer API request is logged for security review. We email the account owner and revoke the key when we detect signals like a sudden spike in 429s or use from many distinct IPs at once (which usually means the key has been pasted into client-side code and leaked). If you think your key was leaked, revoke it from Settings → API Access and issue a new one — there's no penalty.

Error format

All errors use a stable JSON envelope:

{
  "error": {
    "code": "INVALID_PARAMETER",
    "message": "district must be between 1 and 32.",
    "details": { "parameter": "district", "provided": "99" }
  }
}

Common status codes: 400 (validation), 401 (missing/invalid key), 403 (Premium subscription required or revoked), 404 (not found), 429 (rate limited), 500 (internal).

Data sources & methodology

  • Academic / climate / progress scores: NYC DOE School Quality Reports
  • Test proficiency (ELA / Math / Science): NYSED State Report Card
  • 3-K / Pre-K / K admissions & demand: NYC DOE InfoHub LL72 reports
  • Private schools: NCES Private School Universe Survey (PSS), 2023-24
  • High school graduation & Regents: NYC DOE InfoHub
  • Attendance & absenteeism: NYC DOE InfoHub (2018-19 → 2024-25)
  • Discipline & suspensions: NYC DOE InfoHub LL93 reports
  • HS admissions & programs: NYC DOE Fall 2025 HS Directory
  • NYCEEC early childhood centers: NYC Open Data
  • School zones & locations: NYC Open Data + turf.js point-in-polygon
  • Neighborhood Safety Index: NYPD complaint datasets 5uac-w243 (YTD) and qgea-i56i (historic) on Socrata, severity-weighted over a rolling 12-month window. Methodology is published at /safety-methodology.

Deployment

Production uses Cloudflare Workers and D1. Vite builds the frontend; Wrangler bundles the Express Worker and deploys its static assets. Email Service handles transactional messages. Cron Triggers run observability and drip email only; safety data refresh is manual-only, using the authenticated administrative flow and production safety queue. Neon is retained for migration recovery, not used by the production runtime.

npm run check
npm run cf-dry-run
npm run deploy

The Worker is deployed at https://nyc-schools-ratings.biser-d.workers.dev. Canonical production config is wrangler.jsonc; staging uses wrangler.d1-staging.jsonc and a separate database. wrangler.d1-production.jsonc is the explicit maintenance-on configuration, not the normal deployment command. See docs/d1-production-cutover.md for verification, deferred checks, and recovery constraints. Do not roll back to Neon after D1 accepts writes without reconciling those new writes first.

License

All rights reserved. Data is sourced from the publicly available datasets linked above.