Automate Code Review with Structured AI Responses in CI/CD

AI-powered code reviews are great, but parsing natural language responses can be messy. With OpenAI's structured outputs (JSON mode or function calling), you can force the model to return a predictable format. This tutorial shows you how to integrate that into a GitHub Actions pipeline to automatically post review comments.

Step 1: Define Your Structured Output Schema

First, decide what fields you need. For a code review, we want: line number, severity, category, and suggestion. Use Pydantic for validation.

from pydantic import BaseModel
from typing import Literal

class ReviewComment(BaseModel):
    line: int
    severity: Literal["critical", "warning", "info"]
    category: str
    suggestion: str

Step 2: Build the AI Review Function

Create a Python script that sends a diff to OpenAI and parses the structured response.

import openai
from pydantic import BaseModel

client = openai.OpenAI()

def get_review_comments(diff_text: str) -> list[ReviewComment]:
    completion = client.beta.chat.completions.parse(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a code reviewer. Provide structured feedback."},
            {"role": "user", "content": f"Review this diff:\n{diff_text}"}
        ],
        response_format=ReviewComment
    )
    return completion.choices[0].message.parsed

Step 3: Format Comments for GitHub

Convert the parsed objects into GitHub PR review comments format (JSON array of objects with path, line, body).

import json

def format_comments(comments: list[ReviewComment], file_path: str) -> str:
    github_comments = []
    for c in comments:
        github_comments.append({
            "path": file_path,
            "line": c.line,
            "body": f"**{c.severity.upper()}** - {c.category}\n\n{c.suggestion}"
        })
    return json.dumps(github_comments)

Step 4: Create a GitHub Actions Workflow

Now wire it all together. Create .github/workflows/ai-review.yml.

name: AI Code Review
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      - name: Install dependencies
        run: pip install openai pydantic
      - name: Get diff
        id: diff
        run: |
          git fetch origin ${{ github.event.pull_request.base.ref }}
          git diff origin/${{ github.event.pull_request.base.ref }}...HEAD > diff.txt
      - name: Run AI review
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          python review.py > comments.json
        # review.py reads diff.txt and outputs GitHub-formatted JSON
      - name: Post comments
        uses: actions/github-script@v6
        with:
          script: |
            const fs = require('fs');
            const comments = JSON.parse(fs.readFileSync('comments.json','utf8'));
            for (const comment of comments) {
              await github.rest.pulls.createReviewComment({
                ...context.repo,
                pull_number: context.issue.number,
                ...comment
              });
            }

import os
# (imports from steps 1-3)

diff = open("diff.txt").read()
comments = get_review_comments(diff)
print(format_comments(comments, "path/to/changed/file.py"))

Step 5: Test and Iterate

Open a pull request and see the AI comments appear. Adjust the system prompt or schema as needed. Pro tip: Use structured outputs to enforce consistent severity levels, making it easy to filter critical issues from warnings.