GitHub, Copilot coding agent and CI review

Run a Quorum panel on every pull request from GitHub Actions, and give the Copilot coding agent the same tools.

There are two ways to use Quorum from GitHub. A workflow calls the REST API on a pull request and posts the result as a comment. The Copilot coding agent calls the MCP server while it works on an issue. Both use an API key.

Create the key in your organization and store it as a secret. The key bills your organization's API account.

Pull request review with GitHub Actions

Create the secret first: repository Settings, Secrets and variables, Actions, new secret named QUORUM_API_KEY.

Save this as .github/workflows/quorum-review.yml.

name: Quorum review
on:
  pull_request:
    types: [opened, synchronize]

permissions:
  contents: read
  pull-requests: write

jobs:
  review:
    if: github.event.pull_request.head.repo.full_name == github.repository
    runs-on: ubuntu-latest
    timeout-minutes: 10
    env:
      QUORUM_API_KEY: ${{ secrets.QUORUM_API_KEY }}
      BASE: https://www.quorum.dog/v1
      MODEL: QRUM:STAN
      GH_TOKEN: ${{ github.token }}
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Build the request from the diff
        run: |
          git diff "origin/${{ github.base_ref }}...HEAD" | head -c 24000 > diff.txt
          jq -n --arg model "$MODEL" --rawfile diff diff.txt '{
            model: $model,
            messages: [
              {role: "system", content: "You review pull requests. List correctness, security and data-loss risks in this diff. Cite file and line. Say if the change is safe to merge."},
              {role: "user", content: $diff}
            ]
          }' > request.json

      - name: Estimate
        run: |
          curl -sS "$BASE/estimate" \
            -H "Authorization: Bearer $QUORUM_API_KEY" \
            -H "Content-Type: application/json" \
            --max-time 30 -d @request.json | tee estimate.json
          if [ "$(jq -r '.deliberation_value.verdict // ""' estimate.json)" = "likely_hurts" ]; then
            echo "skip=true" >> "$GITHUB_ENV"
          fi

      - name: Deliberate
        if: env.skip != 'true'
        run: |
          curl -sS "$BASE/chat/completions" \
            -H "Authorization: Bearer $QUORUM_API_KEY" \
            -H "Content-Type: application/json" \
            -H "Idempotency-Key: pr-${{ github.event.pull_request.number }}-${{ github.event.pull_request.head.sha }}" \
            --max-time 300 -d @request.json > response.json
          jq -e '.choices[0].message.content' response.json > /dev/null

      - name: Post the result
        if: env.skip != 'true'
        run: |
          RID=$(jq -r '.quorum.request_id' response.json)
          REPLAYED=$(jq -r '.quorum.replayed // false' response.json)
          if [ "$REPLAYED" = "true" ]; then
            SPLIT=unknown
          else
            SPLIT=$(jq -r '(.quorum.converged == false) or (.quorum.remaining_friction != null)' response.json)
          fi
          echo "split=$SPLIT" >> "$GITHUB_ENV"
          {
            if [ "$SPLIT" = "true" ]; then
              echo "**The panel split on this change. A person should read it before merge.**"
              echo
              jq -r '.quorum.contested_passage | if . then "Contested (\(.seat // "unknown seat")): \"\(.quote)\"\n\(.why // "")" else "Contested: see the answer below" end' response.json
            elif [ "$SPLIT" = "unknown" ]; then
              echo "**Replayed answer from an earlier run of this commit. The panel's agreement was not stored, so a person should read it.**"
            else
              echo "**The panel agreed.**"
            fi
            echo
            jq -r '.choices[0].message.content' response.json
            echo
            if [ "$REPLAYED" = "true" ]; then
              echo "<sub>Receipt: \`$RID\` · replayed, no new charge</sub>"
            else
              echo "<sub>Receipt: \`$RID\` · billed \$$(jq -r '.quorum.billed_usd // empty' response.json)</sub>"
            fi
          } > comment.md
          gh pr comment "${{ github.event.pull_request.number }}" --body-file comment.md

What each step does:

The diff is cut at 24,000 characters. The API counts about four characters to a token and accepts 8000 tokens per request by default, so the cut keeps the diff and the instructions under the limit. Input over the limit returns 413 context_length_exceeded. For larger changes, send one call per directory.
The estimate is free. The job skips the deliberation when deliberation_value.verdict is likely_hurts, the verdict for answers that are a passage to reproduce.
The Idempotency-Key is the pull request number and head commit. A re-run of the same commit replays the original answer without a second charge when the first call succeeded. A call that ended capped is not replayed, so a re-run runs again and bills again. A new push gets a new key.
--max-time is 300 seconds, because a deliberation runs several models.
The comment opens with the split state. converged is false, or remaining_friction is set, when the seats disagreed. That is the case to send to a reviewer. contested_passage is an object with seat, quote and why, or null.
A replay carries quorum.replayed: true and none of converged, remaining_friction, contested_passage or billed_usd. The job checks replayed first, says the agreement was not stored, and shows no cost.

To make a split block the merge, add a final step with if: env.split == 'true' that runs exit 1, and mark the job as a required check in branch protection.

Pull requests from forks run without repository secrets. The if on the job line limits the job to branches in the same repository.

Copilot coding agent

The Copilot coding agent reads MCP servers from the repository's settings. Open Settings, Copilot, Coding agent, MCP configuration, and add:

{
  "mcpServers": {
    "quorum": {
      "type": "http",
      "url": "https://www.quorum.dog/mcp",
      "headers": { "Authorization": "$COPILOT_MCP_QUORUM_AUTH" },
      "tools": ["estimate", "deliberate", "get_receipt", "list_modes"]
    }
  }
}

Then add a secret named COPILOT_MCP_QUORUM_AUTH in the repository's copilot environment. Its value is the whole header value: Bearer followed by your Quorum API key. The server URL is https://www.quorum.dog/mcp.

The tools list sets what the agent can call. deliberate costs money, so estimate and get_receipt sit beside it: the agent estimates first and reads the receipt after. The loop is estimate, deliberate, get_receipt, act. You can add a line to the repository's agent instructions:

Before you open or update a pull request that touches migrations, auth or billing, call
estimate, then deliberate on the diff. If the seats split, stop and ask
for a human review.

The paper

This guide is also a paper: GitHub, Copilot coding agent and CI review (PDF).

More papers are on the whitepaper shelf.

GitHub, Copilot coding agent and CI review

Pull request review with GitHub Actions

Copilot coding agent

Next

The paper