Building a Multi-Agent Code Review Pipeline with GitHub Actions

Manual code reviews are time-consuming and inconsistent. By using a multi-agent AI pipeline, you can automate initial checks and let reviewers focus on logic. This tutorial shows how to build such a pipeline with GitHub Actions and Python, using three specialized agents: a Linter, a Test Analyzer, and a Security Scanner.

Step 1: Define the Agent Roles

Each agent is a simple Python script that receives a diff/changed file and returns a structured JSON report. Create three files:

# agents/linter_agent.py
import sys
import ast

def check_syntax(filepath):
    with open(filepath) as f:
        content = f.read()
    try:
        ast.parse(content)
        return {"passed": True, "issues": []}
    except SyntaxError as e:
        return {"passed": False, "issues": [{"line": e.lineno, "message": str(e)}]}

if __name__ == "__main__":
    print(json.dumps(check_syntax(sys.argv[1])))

# agents/test_agent.py
import subprocess

def run_tests(test_path):
    result = subprocess.run(["pytest", test_path, "--json-report"], capture_output=True, text=True)
    return result.stdout

if __name__ == "__main__":
    print(run_tests("tests/"))

# agents/security_agent.py
import requests

def scan_dependencies():
    # Simulate scanning requirements.txt
    with open("requirements.txt") as f:
        deps = f.read()
    # Simple check for known vulnerable packages
    vulnerabilities = []
    if "requests==2.25.0" in deps:
        vulnerabilities.append({"package": "requests", "version": "2.25.0", "issue": "CVE-2023-1234"})
    return {"vulnerabilities": vulnerabilities}

if __name__ == "__main__":
    print(scan_dependencies())

Step 2: Orchestrate with an AI Coordinator

The coordinator collects agent outputs and uses an LLM to generate a summary. Create coordinator.py:

import json
import subprocess
import openai

def run_agents(pr_files):
    results = {}
    for file in pr_files:
        if file.endswith(".py"):
            results[f"linter_{file}"] = subprocess.check_output(["python", "agents/linter_agent.py", file]).decode()
            results[f"security_{file}"] = subprocess.check_output(["python", "agents/security_agent.py"]).decode()
    results["tests"] = subprocess.check_output(["python", "agents/test_agent.py"]).decode()
    return results

def format_prompt(results):
    prompt = "You are a code review assistant. Summarize the following agent reports and list any critical issues:"
    for name, report in results.items():
        prompt += f"\n{name}: {report}"
    prompt += "\nProvide a concise review in bullet points."
    return prompt

def generate_review(results):
    prompt = format_prompt(results)
    response = openai.Completion.create(
        engine="gpt-4",
        prompt=prompt,
        max_tokens=500,
        temperature=0.3
    )
    return response.choices[0].text

if __name__ == "__main__":
    # Simulate PR files
    changed_files = ["app.py", "utils.py"]
    agent_results = run_agents(changed_files)
    review = generate_review(agent_results)
    print("=== AI Review ===")
    print(review)

Step 3: Wire It Into GitHub Actions

Create .github/workflows/code-review.yml:

name: Multi-Agent Code Review
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'
      - name: Install dependencies
        run: pip install openai requests pytest
      - name: Run agents
        run: python coordinator.py
      - name: Post review comment
        uses: actions/github-script@v6
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          script: |
            const fs = require('fs');
            const review = fs.readFileSync('review_output.txt', 'utf8');
            github.rest.issues.createComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
              body: review
            });

Going Further

• Add more agents: code style, documentation coverage, etc.
• Use openai.StructuredOutput to enforce JSON format for agent reports.
• Implement caching to avoid rerunning agents on unchanged files.

This pipeline scales review capacity, catches surface-level issues instantly, and lets human reviewers focus on architecture and design. Start with the skeleton above and customize agent logic for your stack.