Designing repository YAML policy for AI code review

Repository configuration turns AI code review from a generic service into a review policy maintainers can inspect. A YAML file works well because it can be versioned with the code, reviewed through the same pull-request process, and resolved at a specific commit. The danger is creating a bag of model knobs whose interactions nobody understands. Good configuration expresses product decisions in repository language.

Begin with a small stable surface: enabled reviewer roles, minimum confidence, maximum inline comments, path includes and excludes, model selection, and concise repository instructions. Every field needs a documented default and validation rule. A missing configuration file should produce a safe predictable review, while an invalid file should produce a visible configuration error rather than silently falling back and surprising the team.

Separate policy from prompt prose

Policy says what matters: authorization must be tenant-scoped, generated clients are not reviewed, database migrations need rollback consideration, or architecture comments are limited to new boundaries. Prompts say how a reviewer should reason and format output. Expose repository instructions and role focus without requiring teams to replace the entire system prompt. Full prompt overrides are powerful but make safety, schema behavior, and future upgrades difficult to support.

version: 1
review:
  min_confidence: 0.80
  max_inline_comments: 8
  reviewers: [correctness, security, architecture]
paths:
  exclude:
    - "**/*.generated.ts"
    - "vendor/**"
models:
  default: "team-approved-model"
policy:
  - "Authorization checks must include tenant scope."
  - "Do not comment on formatting or naming."

Make path behavior explicit

Define glob syntax, case sensitivity, precedence, and whether exclusions affect model context, posted findings, or both. Generated output may be excluded as a finding target but still needed to understand an interface; the configuration should distinguish those cases if the product supports them. In a monorepo, define how root and nested files merge. Avoid hidden administrator overrides that cannot be seen from the repository.

• Validate unknown keys and type errors; typos should not become ignored policy.
• Resolve configuration from the pull request head so the reviewed policy matches the reviewed code.
• Require extra authorization or a protected path for changes to provider routing and sensitive policy.
• Show the effective configuration and its source commit in run history.
• Provide a dry-run or config-check command so teams can test changes before consuming a review run.

Keep limits understandable

A confidence floor controls which generated findings survive; an inline cap controls how many surviving findings enter the GitHub conversation. They solve different problems and should be named separately. Compute or usage limits are different again. Document whether the cap applies per reviewer or after aggregation—after aggregation is usually fairer because it allows severity and confidence to compete across roles.

Version for evolution, not ceremony

Include a schema version and reject versions the service cannot interpret. Add fields backward-compatibly when possible and provide migration guidance when semantics change. Record the effective config with each review so old findings remain explainable after the file evolves. The best YAML policy is short enough that a maintainer can read it during review and specific enough that two runs against the same commit behave consistently.

Treat configuration changes as product changes. A lower threshold or broader path scope can increase comments, provider usage, and the code sent for processing. Show that impact in the pull request and require appropriate owners for sensitive fields. After merging, compare run volume and resolution reasons with the previous configuration version. Version control is most valuable when it connects policy intent to observable review behavior.