Design a GitHub code review automation workflow that reacts to the right events, handles updates safely, posts readable feedback, and stays out of the merge path until it earns trust.
GitHub code review automation is often described as a webhook followed by a model call. The production workflow is broader. Pull requests change while a review is running, commits are force-pushed, patches omit unchanged context, API permissions vary by repository, and GitHub only accepts inline comments at valid diff positions. A reliable design treats those constraints as the product, not as plumbing around the interesting AI part.
Start by writing the event contract. Decide whether a review runs when a pull request opens, becomes ready for review, receives a new commit, or is explicitly requested. Draft pull requests usually benefit from a different policy than review-ready work. Synchronize events can arrive quickly during rebases. Manual triggers can be useful for expensive repositories. There is no universal answer, but an undocumented mix of triggers guarantees duplicate runs and confused maintainers.
Every run should be tied to a repository, pull request number, head commit SHA, configuration version, selected reviewer set, and model identity. Fetching the current diff at several points in a long-running job can mix revisions. Capture or address the intended revision once, then mark the output stale if the head changes before posting. This makes the run explainable later and prevents feedback for an old commit from masquerading as a review of the new one.
A model may refer to any line it has seen, but GitHub inline comments belong to lines represented in the pull request diff. Parse unified-diff hunks and map eligible new or context lines to the API representation you use. Findings that do not map should not disappear; they can be included in a summary or stored in run history. The important behavior is deterministic degradation: one unmappable comment must not cause the entire review to fail.
Aggregate before posting. Normalize severity, remove duplicates, filter below the confidence floor, and apply a repository-level inline cap. A single review containing a compact summary and a few ordered findings is easier to scan than separate comments from several agents. If more findings exist, link to run history rather than filling the conversation. GitHub attention is a constrained interface, and automation should spend it deliberately.
When a new commit arrives, compare against the previously reviewed head rather than treating the pull request as entirely new. Track stable finding identities using file, line context, rule or title, and the reviewed commit. Do not repost an unchanged observation merely because line numbers moved. Do re-evaluate findings whose surrounding code changed. Incremental behavior is one of the clearest differences between a helpful reviewer and a bot that restates itself after every push.
Begin with COMMENT reviews and observable run history. Monitor failures, latency, accepted findings, dismissals, and repeated comments. Security or policy checks may eventually justify enforcement, but encode those as narrow deterministic rules where possible. A broad generative review should remain advisory until a team has measured its behavior on its code. Automation succeeds when maintainers know when it runs, what it reviewed, how to disagree, and why the output deserves attention.
Before calling the workflow complete, rehearse its failure modes. Revoke the GitHub installation, expire a provider key, exceed a rate limit, send an empty patch, and push a new commit during a run. Each case should leave an understandable status and a safe retry path. Operational clarity is part of review quality: maintainers should never mistake a missing or partial automated run for evidence that a change is clean.
PR Quorum turns specialist reviewer output into one clean GitHub review, with noise controls and predictable usage caps.