Parsing AI Responses in CI/CD with Structured Outputs

AI models can produce powerful analysis, but their unstructured text responses are tricky to parse programmatically. Structured outputs—like OpenAI's JSON mode or function calling—let you enforce a schema so your CI/CD pipeline can reliably extract and act on AI-generated content.

This tutorial shows you how to create a Python script that uses OpenAI's structured outputs to review code, then integrate that script into a GitHub Actions workflow to post comments on pull requests automatically.

Step 1: Define your output schema with Pydantic

First, define the exact structure you want the AI to return. We'll use Pydantic to model a code review result.

from pydantic import BaseModel
from typing import List
from enum import Enum

class Severity(str, Enum):
    critical = "critical"
    warning = "warning"
    info = "info"

class Issue(BaseModel):
    line: int
    message: str
    severity: Severity

class CodeReviewResult(BaseModel):
    summary: str
    issues: List[Issue]

This schema ensures the AI returns a clear, parseable JSON structure.

Step 2: Create a script that calls OpenAI with structured output

Now write a Python script that reads a code file, asks the AI for a review, and enforces the schema using the response_format parameter.

import os
import sys
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

def review_code(file_path: str) -> CodeReviewResult:
    with open(file_path, "r") as f:
        code = f.read()

    response = client.beta.chat.completions.parse(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a senior code reviewer. Analyze for bugs, security issues, and style problems."},
            {"role": "user", "content": f"Review this code:\n```\n{code}\n```"}
        ],
        response_format=CodeReviewResult
    )
    return response.choices[0].message.parsed

if __name__ == "__main__":
    result = review_code(sys.argv[1])
    print(result.model_dump_json(indent=2))

Step 3: Integrate into GitHub Actions

Create a workflow file that runs on pull requests, executes the review script, and posts a comment with the results using the GitHub API.

name: AI Code Review
on:
  pull_request:
    paths:
      - '**.py'

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - name: Install dependencies
        run: pip install openai pydantic
      - name: Run AI review
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          python review.py ${{ github.event.pull_request.head.sha }}.diff 2> review.json
      - name: Post comment
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const review = JSON.parse(fs.readFileSync('review.json', 'utf8'));
            const body = `## AI Code Review\n**Summary:** ${review.summary}\n**Issues:**\n${review.issues.map(i => `- 🔴 ${i.severity}: ${i.message} (line ${i.line})`).join('\n')}`;
            github.rest.issues.createComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
              body: body
            });

Step 4: Handle errors and edge cases

Add error handling for API failures, invalid schemas, or large files. Validate the parsed result before posting.

You can now extend this pattern to generate changelogs, unit tests, or PR summaries—all with reliable, parseable output. The structured output approach makes your AI integrations robust and production-ready.