Automate PR Descriptions with AI Agents: A Step-by-Step Workflow

Writing pull request descriptions is tedious, but it’s the first thing reviewers see. Instead of doing it manually, let an AI agent draft it for you. In this tutorial, you’ll create a GitHub Action that runs on every new PR, extracts the diff and commit messages, and uses a structured LLM call to generate and post a detailed PR description automatically.

Prerequisites

• GitHub repository (the tutorial works with any language)
• A GitHub Actions runner with access to secrets
• An OpenAI API key (or any LLM API)
• Basic familiarity with YAML and Python

Step 1: Add a workflow file

Create .github/workflows/pr-description.yml in your repo:

name: Generate PR Description

on:
  pull_request:
    types: [opened]

jobs:
  generate:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - name: Run AI PR Description Generator
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
          REPO: ${{ github.repository }}
        run: python .github/scripts/generate_pr_description.py

We set fetch-depth: 0 to get the full diff between the base and head branches.

Step 2: Write the AI agent script

Create .github/scripts/generate_pr_description.py. This script fetches the diff, builds a prompt, calls the LLM with structured JSON output, and updates the PR body via the GitHub API.

import json, os, subprocess
import urllib.request

# 1. Get the diff
base_branch = os.environ['GITHUB_BASE_REF'] or 'main'
ose.system('git fetch origin %s' % base_branch)
diff = subprocess.check_output(['git', 'diff', 'origin/' + base_branch + '...HEAD', '--', '.', ':(exclude).lock']).decode()

# 2. Get commit messages
commits = subprocess.check_output(['git', 'log', 'origin/' + base_branch + '..HEAD', '--pretty=%s']).decode()

# 3. Build the prompt
prompt = f"""You are a senior developer writing a PR description.
Write a clear and structured description for the changes below.
Include: Summary, Key Changes, Testing Done, and Notes.

Commit messages:
{commits}

Diff (truncated to 5000 chars):
{diff[:5000]}
"""

# 4. Call OpenAI with structured output
request_body = json.dumps({
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": prompt}],
    "response_format": { "type": "json_object" }
}).encode()

req = urllib.request.Request(
    "https://api.openai.com/v1/chat/completions",
    data=request_body,
    headers={
        "Authorization": "Bearer " + os.environ['OPENAI_API_KEY'],
        "Content-Type": "application/json"
    }
)
try:
    with urllib.request.urlopen(req) as resp:
        data = json.loads(resp.read().decode())
        description = json.loads(data['choices'][0]['message']['content'])
except Exception as e:
    print("OpenAI call failed:", e)
    exit(1)

# 5. Format and post to the PR
body = f"## Summary\n{description['summary']}\n\n## Key Changes\n{description['key_changes']}\n\n## Testing Done\n{description['testing']}\n\n## Notes\n{description['notes']}"

pr_number = os.environ['PR_NUMBER']
repo = os.environ['REPO']
url = f"https://api.github.com/repos/{repo}/issues/{pr_number}"
data = json.dumps({"body": body}).encode()
req = urllib.request.Request(url, data=data, headers={
    "Authorization": "Bearer " + os.environ['GITHUB_TOKEN'],
    "Accept": "application/vnd.github+json",
    "Content-Type": "application/json"
}, method="PATCH")
urllib.request.urlopen(req)

This script uses only Python’s standard library, so no dependencies. The response_format parameter forces the LLM to return valid JSON, making parsing safe and predictable.

Step 3: Add the OpenAI API key as a secret

Go to your GitHub repo → Settings → Secrets and variables → Actions → New repository secret. Add OPENAI_API_KEY with your key.

Step 4: Test it

Create a new branch, make a change, and open a PR. Within seconds, the Action runs and updates the PR description. You’ll see the AI-generated summary, key changes, testing notes, and follow-up comments.