Automating PR Descriptions with AI Agents: A Step-by-Step Workflow

Pull request descriptions are often overlooked, but they are critical for code reviews and collaboration. Automating them with an AI agent saves time and ensures consistency. In this tutorial, you'll set up a GitHub Actions workflow that triggers on new PRs, fetches the diff, and calls an AI model to generate a summary.

Step 1: Create a GitHub Actions Workflow

Create a file .github/workflows/pr-description.yml in your repository:

name: Auto PR Description
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  generate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: Get diff
        id: diff
        run: |
          git fetch origin ${{ github.event.pull_request.base.ref }}
          git diff origin/${{ github.event.pull_request.base.ref }}...HEAD > changes.diff
      - name: Run AI agent
        run: |
          python scripts/generate_pr_description.py
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          PR_NUMBER: ${{ github.event.number }}
          REPO: ${{ github.repository }}

Step 2: Write the AI Agent Script

Create scripts/generate_pr_description.py:

import os
import requests

# Read the diff file
with open("changes.diff", "r") as f:
    diff = f.read()

# Truncate if too long (AI models have token limits)
if len(diff) > 10000:
    diff = diff[:10000] + "\n... (truncated)"

# Prompt the AI
prompt = f"""You are a senior developer writing a pull request description.
Given the following code diff, generate a concise description in this format:
- Summary: one paragraph
- Changes: bullet points of key modifications
- Related issues: if any are mentioned in commit messages

Diff:
{diff}
"""

response = requests.post(
    "https://api.openai.com/v1/chat/completions",
    headers={
        "Authorization": f"Bearer {os.environ['OPENAI_API_KEY']}",
        "Content-Type": "application/json"
    },
    json={
        "model": "gpt-4",
        "messages": [{"role": "user", "content": prompt}],
        "max_tokens": 500
    }
)

description = response.json()["choices"][0]["message"]["content"]

# Update PR description via GitHub API
headers = {
    "Authorization": f"Bearer {os.environ['GITHUB_TOKEN']}",
    "Accept": "application/vnd.github+json"
}
url = f"https://api.github.com/repos/{os.environ['REPO']}/pulls/{os.environ['PR_NUMBER']}"
data = {"body": description}
requests.patch(url, json=data, headers=headers)
print("PR description updated.")

Step 3: Configure Permissions

Add this to the top of your workflow file (before jobs) to allow the token to update PRs:

permissions:
  pull-requests: write

Step 4: Test and Refine

You can extend this by using different models (like Claude-3), adding a fallback, or including commit messages in the prompt. Experiment with the prompt to match your team's style.