Structure Your AI Code Reviews: A JSON-Powered CI/CD Pipeline

AI code reviewers are only as good as their output. If your review bot returns a wall of text, you can't act on it programmatically. By forcing the AI to return structured JSON, you can capture review comments, assign severity levels, and auto-post them to your PR. Here's how to build a practical, two-part pipeline with GitHub Actions.

Step 1: Define the response schema

We'll use OpenAI's function calling to guarantee a JSON array of review comments. Each object has a file, line, severity, and suggestion. Here's the schema we'll include in our prompt:

review_schema = {
  "type": "object",
  "properties": {
    "comments": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "file": {"type": "string"},
          "line": {"type": "integer"},
          "severity": {"type": "string", "enum": ["info", "warning", "critical"]},
          "suggestion": {"type": "string"}
        },
        "required": ["file", "line", "severity", "suggestion"]
      }
    }
  }
}

This schema is strict: the model must output exactly these fields. No extra prose.

Step 2: Write the review script

Create a Python file build_ai_review.py in your repo. It reads the diff, sends it to OpenAI, parses the JSON, and writes a cleaned file to disk for later steps.

import json, os, openai
from openai import OpenAI

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

diff = os.getenv("PR_DIFF")  # Provided by the workflow

response = client.chat.completions.create(
    model="gpt-4-turbo",
    messages=[
        {"role": "system", "content": "You are a senior code reviewer. Analyze the diff and return structured comments."},
        {"role": "user", "content": f"Here is the diff:\n{diff}"}
    ],
    functions=[{
        "name": "review_code",
        "parameters": review_schema
    }],
    function_call={"name": "review_code"}
)

args = json.loads(response.choices[0].message.function_call.arguments)
with open("review_results.json", "w") as f:
    json.dump(args["comments"], f)
print(f"Found {len(args['comments'])} comments")

Step 3: Add the GitHub Action workflow

Now create .github/workflows/ai-review.yml. This workflow triggers on pull requests, runs the script, and then uses the GitHub API to post inline review comments.

name: AI Review
on: pull_request
jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Get diff
        id: diff
        run: |
          echo "PR_DIFF=$(git diff origin/${{ github.event.pull_request.base.ref }}...HEAD | tr '\n' ' ')" >> $GITHUB_ENV

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install deps
        run: pip install openai

      - name: Generate AI review
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: python build_ai_review.py

      - name: Post comments
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          jq -c '.[]' review_results.json | while read comment; do
            file=$(echo $comment | jq -r '.file')
            line=$(echo $comment | jq -r '.line')
            body=$(echo $comment | jq -r '.suggestion')
            curl -s -X POST \
              -H "Authorization: token $GH_TOKEN" \
              -H "Accept: application/vnd.github+json" \
              https://api.github.com/repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/comments \
              -d "{\"body\": \"$body\", \"commit_id\": \"${{ github.event.pull_request.head.sha }}\", \"path\": \"$file\", \"line\": $line}"
          done

This pipeline uses jq to iterate each comment and posts it to the PR via the REST API. The GITHUB_TOKEN is automatically available, but note it can only leave comments, not review summaries.

Step 4: Test and iterate

Open a test PR and watch the action run. Check the pull request comments tab—you should see structured feedback. You can now filter by severity, ignore info messages, or block merge if any critical comments exist.