Repository avatar
AI Tools
v0.2.0
active

cachepin

io.github.pgg898200-ship-it/cachepin

Zero-dep Rust+SIMD LLM prompt-cache compiler, exact mixed-TTL DP solver, auditor & Shadow-KV router

Documentation

A pin holds the shared prefix of three prompt sheets.

CachePin

Find where two LLM requests stop sharing a prompt prefix. Two agents can use the same runbook and tools but list the tools in different orders. CachePin shows the first differing byte, identifies the changed block, and estimates how much of the next request could be reused if both clients serialized it the same way.

Compare two prompts in your browser · Download the CLI · Connect an MCP client

The bundled example goes from zero to 1,099 estimated matched tokens after sorting its tools.

Try a pair of requests

The two example requests share a runbook and two tools. The tool order and the final user question differ. From a checkout with the release binary on your PATH, run:

cachepin diff examples/prefix-diff/a.json examples/prefix-diff/b.json --mode json --provider anthropic --json

The v0.2.0 report includes these fields:

{
  "reason": "tool_order_drift",
  "divergence_path": "tools[0]",
  "raw_lcp_bytes": 68,
  "raw_matched_tokens": 0,
  "opt_matched_tokens": 1099,
  "total_tokens": 1113
}

The files share 68 literal bytes before the first tool name changes. In CachePin's replay model, sorting the tools and JSON keys raises the matching prefix from 0 to 1,099 estimated tokens for the second request. The different user questions still reach the model. You can run the comparison locally without an API key, model, or GPU. For billing or latency decisions, compare provider-reported usage and timing on your own workload.

Where it fits

Your taskPut CachePin hereWhat comes out
Diagnose two requestsAfter your client serializes request JSON, before it calls a modelcachepin diff a.json b.json --json reports the first differing byte, likely cause, and modeled reuse.
Audit a workloadOffline, alongside a JSONL trace or local Claude Code / Codex logscachepin audit trace.jsonl --json summarizes observed records and modeled cache behavior.
Standardize requestsBetween your prompt builder and provider SDKcachepin optimize --mode json --no-breakpoints reads JSON from stdin and writes the transformed request to stdout.
Route live trafficBetween the agent and a model API or a pool of vLLM / SGLang backendsThe optional reverse proxy forwards requests, can coalesce same-prefix bursts, and exposes /metrics.
Invoke from an agentAs a local stdio MCP serverLaunch cachepin mcp, discover its five tools with tools/list, and call the one your client needs.

Start with --mode json for key and tool ordering. The hoist and swarm modes can move volatile values and reorder shared context blocks. Inspect the resulting request and evaluate response quality on representative tasks before using those modes in live traffic.

Install

Download a Linux or macOS binary from the release assets, or build the Rust CLI locally:

git clone https://github.com/pgg898200-ship-it/cachepin.git
cd cachepin
cargo build --release --locked
./target/release/cachepin diff examples/prefix-diff/a.json examples/prefix-diff/b.json --mode json --provider anthropic --json

The Rust crate has zero external crate dependencies. The installer, Homebrew formula, Nix flake, container, and Python binding provide other entry points.

Measurements

WorkloadRecorded result
Compile an 18,535-byte request on an AVX2 CPU84.23 µs median, 138.44 µs p99
Proxy a request to a loopback upstream219.43 µs median added overhead
Send 32 concurrent requests to the local test upstream32 cold writes in the baseline, 1 with coalescing
Replay a synthetic 32-agent coding workload2.24% modeled cache hits before normalization and 98.44% after

The first three results are local measurements. The last uses a simulated access pattern and pricing profile. Recorded results · Reproduce the benchmark · Method and implementation

Integrations and related tools

Browse other PGG tools by task.

See the license file and crate metadata for licensing details.

MCPB
https://github.com/pgg898200-ship-it/cachepin/releases/download/v0.2.0/cachepin-mcp.mcpb
Install Command
# mcpb: https://github.com/pgg898200-ship-it/cachepin/releases/download/v0.2.0/cachepin-mcp.mcpb