DocDrifter is a GitHub Action that watches pull requests which change code but leave documentation untouched. An LLM reads the diff against your docs tree and answers one question: is anything written here now wrong? If yes, it comments once, on the PR, with the files to check. If no — internal refactor, test-only change, dependency bump — it says nothing at all.
GitHub Action runs on pull_request no API key to configure free for public repos
The action splits the diff by path into source and documentation. If the PR touches any file under your docs path, it exits immediately — no model call, no comment. That's a coarse check: it assumes touching docs at all means the right docs got touched, so a one-line docs edit next to a bigger undocumented change will still go quiet. Point docs-path narrowly, or use skip-labels to opt specific PRs back in.
The changed hunks and the PR title go to an LLM together. It judges user-visible surface: public APIs, config keys, defaults, behavior. Internal helpers and test-only changes are excluded by design.
If docs drifted, DocDrifter leaves a single comment explaining why. It never fails your build, never blocks the merge, and updates the same comment instead of piling up new ones.
DocDrifter: possible docs drift detected
This PR changes source files under src/ but doesn't update anything under README.md. The PR adds a new user-facing input option (min_disclosure_lag_days) that external users can configure, which should be documented in the README.
If this doesn't need a docs update, feel free to ignore this comment.
Silence is the feature. A bot that comments on every diff gets muted in a week — the whole point is separating the internal refactor from the change your users will actually notice. That comment above is real, not a mock — click through and see it on the actual PR.
68–85% precision, 92–100% recall, across two repos. Not one repo above the other — apiflask lands lower than pygeoapi even after correction.
We pulled 92 merged PRs from two open-source projects (apiflask, pygeoapi), read each one, and recorded whether documentation genuinely needed an update — then ran DocDrifter against the same set. This was not a blind eval: the prompt went through a few iterations against this same labeled set before we settled on the version shipping today. Along the way we also found two labeling misses of our own on apiflask — PRs we'd marked "no drift" that a manual re-read confirmed were real drift, mislabeled before the model ever saw them. We're showing both the raw numbers (original labels) and the corrected numbers (after fixing those two), so you can judge either one. Nothing is held back — the labels, the prompt history, and every run are in the repository.
| Repo | Precision | Recall |
|---|---|---|
| apiflask (raw labels) | 68.2% | 100% |
| apiflask (2 labels corrected) | 77.3% | 100% |
| pygeoapi (raw labels) | 84.6% | 91.7% |
apiflask: 17 of 44 PRs were real drift, corrected precision 77.3% is below pygeoapi's — we're not claiming a uniform 85%+ across both repos, apiflask is the weaker of the two. pygeoapi's ground truth was never revised — only apiflask had labels we caught and fixed.
The baseline is what most teams write first: flag any PR touching src/ without touching docs/. It fires on almost every refactor, which is why nobody keeps it switched on.
no card required
one seat per private repo, start with just one
Each private repo you activate takes one $9/month seat. Having 30 private repos doesn't mean paying for 30 — activate one, or ten, and add more when you want them. Seats aren't tied to a specific repo: free one up and point it somewhere else whenever you like. Public repos never need a seat.
Fair use: 100 PR checks/month per private repo, 50 for public — plenty for almost everyone, and there if you ever need more. Details in the terms.
Install DocDrifter Dashboard on your account or org to activate private repos, see a repo list, and manage billing in one place. Then commit the workflow below, open your next PR, and see whether it stays quiet. repo-description is optional but not just decoration — a real one-liner about what your project does noticeably sharpens the model's judgment calls.
Public repo, don't need the dashboard? View the Action on GitHub and add just the workflow below.
# .github/workflows/docdrifter.yml name: DocDrifter on: pull_request: types: [opened, synchronize, reopened, labeled, unlabeled] permissions: pull-requests: write contents: read id-token: write jobs: check-docs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: seralifatih/docdrifter@v1 with: src-path: "src/" docs-path: "docs/" repo-description: "describe your project here"
Leave your email and we'll let you know when there's something worth coming back for — new results, a pricing change, nothing spammy.