Writing detailed pull request descriptions is a chore that often gets skipped, leading to confusing code reviews. In this tutorial, you'll build an AI agent that automatically generates PR descriptions from the diff, including a summary, key changes, testing notes, and potential impacts. By the end, you'll have a working GitHub Action that posts descriptions as PR comments.

Prerequisites: A GitHub repository with at least one PR, a GitHub account, and an OpenAI API key (or any LLM API key). Basic familiarity with YAML and GitHub Actions is helpful but not required.

Step 1: Create the AI Agent Script

First, we'll write a Python script that takes a git diff, calls an LLM, and outputs a structured JSON description. Create a file named generate_pr_desc.py in your repo's .github/scripts folder.

import os, json, subprocess
from openai import OpenAI

def get_diff(base, head):
    result = subprocess.run(
        ["git", "diff", base, head, "--", ":!*.lock"],
        capture_output=True, text=True
    )
    return result.stdout

def generate_description(diff):
    client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
    prompt = f"""You are a senior engineer reviewing a PR. Based on the diff below, output a JSON object with keys:
- summary (one-line)
- changes (list of changed files with brief description)
- testing_notes (how to test)
- impact (any breaking changes or performance concerns)

Diff:
{diff[:8000]}
"""
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        response_format={"type": "json_object"}
    )
    return json.loads(response.choices[0].message.content)

if __name__ == "__main__":
    base = os.environ["BASE_SHA"]
    head = os.environ["HEAD_SHA"]
    diff = get_diff(base, head)
    desc = generate_description(diff)
    with open("pr_description.json", "w") as f:
        json.dump(desc, f)
    print("Generated pr_description.json")
The script limits diff to 8000 characters to avoid token overflow. For large diffs, consider chunking or summarizing individual files.

Step 2: Create the GitHub Action Workflow

Next, create a workflow file at .github/workflows/pr-description.yml. This action triggers on pull request events, runs the script, and posts the description as a comment.

name: Generate PR Description
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  generate:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
      checks: write
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install openai
      - name: Run PR description generator
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          BASE_SHA: ${{ github.event.pull_request.base.sha }}
          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
        run: python .github/scripts/generate_pr_desc.py
      - name: Post comment
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const desc = JSON.parse(fs.readFileSync('pr_description.json', 'utf8'));
            const body = `## 🤖 AI-Generated PR Description
**Summary:** ${desc.summary}
### Changes
${desc.changes.map(c => `- **${c.file}**: ${c.description}`).join('\n')}
### Testing
${desc.testing_notes}
### Impact
${desc.impact}`;
            await github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: body
            });
Make sure to add your OPENAI_API_KEY as a repository secret in GitHub Settings → Secrets and variables → Actions.

Step 3: Test the Workflow

Create a new branch, make some changes, and open a pull request. Within a minute, the action will run and post a comment like this:

Successful Output Example:
## 🤖 AI-Generated PR Description
**Summary:** Refactor user authentication to use JWT tokens instead of sessions.
### Changes
- **auth.py**: Rewrote login/logout endpoints
- **models.py**: Added JWTToken model
### Testing
Test by logging in with existing credentials, verify token is returned.
### Impact
Breaking: Session-based middleware must be removed.

Customization Tips

  • Change model: Use gpt-4o for better accuracy, or gpt-3.5-turbo for lower cost.
  • Add structure: Modify the prompt to include risk assessment or checklist items.
  • Update PR body: Instead of a comment, you can use the GitHub API to edit the PR body directly.

That's it! You now have an automated PR description agent that saves reviewers time and ensures every PR has context. Experiment with different prompts and models to fit your team's style.