Automating Pull Request Descriptions with OpenAI and Vercel

25th February 2025

In this tutorial, we'll build a PR description generator using OpenAI's GPT-4 and deploy it on Vercel. This tool will automatically summarize code changes when a PR is opened or marked as ready for review.

What This Bot Does?

Our bot will:

Listen for pull request events on GitHub via webhooks.
Fetch the PR’s changed files.
Generate a concise PR description using OpenAI.
Automatically update the PR description.

This is a standalone feature and does not require the previous PR reviewer bot, but if you're interested in AI-assisted PR reviews, check out this tutorial. 🚀

Prerequisites

Before starting, ensure you have:

A GitHub account
A Vercel account (for easy deployment)
An OpenAI API key
Node.js installed on your machine

Writing a good Prompt

To get high-quality PR descriptions from OpenAI, it's crucial to craft a well-structured prompt. A good prompt should:

Clearly define the task (e.g., "Summarize the following code changes into a concise PR description.")
Include context about the code changes (e.g., file diffs, modified lines)
Specify the desired format (e.g., bullet points, a paragraph, or specific sections)

This ensures that OpenAI understands the intent and generates a useful summary.

1Summarize the following code changes into a concise PR description.Include:
21. A high - level overview of what was modified.
32. The key improvements or fixes.
43. Any breaking changes or important considerations.
5
6  Changes:
7[file - diffs]

Understanding OpenAI API Parameters

When making API requests to OpenAI's https://api.openai.com/v1/chat/completions endpoint, the following parameters are important in our case:

model: Specifies the model to use (e.g., gpt-4).
messages: The conversation history; typically structured as an array of objects with role (developer, user, assistant) and content (the actual text input).
temperature: Controls randomness (lower values make responses more deterministic, higher values make them more creative).
max_tokens: Sets a limit on the number of tokens in the response to control response length. This can be use to control the costs of the completions.

As per OpenAI docs:

In the chat completions API, you create prompts by providing an array of messages that contain instructions for the model. Each message can have a different role, which influences how the model might interpret the input.

—

Example API call:

1const aiResponse = await openai.chat.completions.create({
2  model: "gpt-4",
3  messages: [
4    { role: "developer", content: "Summarize the following code changes into a concise PR description." },
5    { role: "user", content: fileDiffs }
6  ],
7  temperature: 0.5,
8  max_tokens: 200,
9});

Step 1: Setting Up the Project

Create a new directory and initialize the project:

1mkdir github - pr - describer
2cd github - pr - describer
3npm init - y
4npm install @octokit/rest openai dotenv express

Step 2: Handling Webhooks

Create an api directory for our webhook handler:

1mkdir api

Then, create api/webhook.js and add the following code:

1const express = require('express');
2const { Octokit } = require('@octokit/rest');
3const OpenAI = require('openai');
4const { verifyWebhookSignature } = require('../utils/security');
5
6const router = express.Router();
7const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN });
8const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
9
10const prompt = `
11Summarize the following code changes into a concise PR description. Include:
121. A high - level overview of what was modified.
132. The key improvements or fixes.
143. Any breaking changes or important considerations.
15
16  Changes:
17[file - diffs]`;
18
19router.post('/webhook', async (req, res) => {
20  try {
21    verifyWebhookSignature(req);
22    const { pull_request, repository, action } = req.body;
23
24    if (!pull_request || !['opened', 'ready_for_review'].includes(action)) {
25      return res.status(400).send('Not a relevant PR event');
26    }
27
28    const owner = repository.owner.login;
29    const repo = repository.name;
30    const prNumber = pull_request.number;
31
32    // Fetch PR files
33    const { data: files } = await octokit.pulls.listFiles({ owner, repo, pull_number: prNumber });
34    const fileDiffs = files.map(f => `${f.filename}:\n${f.patch}`).join('\n');
35
36    // Generate PR description using OpenAI
37    const aiResponse = await openai.chat.completions.create({
38      model: "gpt-4o-mini",
39      messages: [
40        { role: "developer", content: prompt },
41        { role: "user", content: fileDiffs }
42      ],
43      temperature: 0.5,
44      max_tokens: 200,
45    });
46
47    const newDescription = aiResponse.choices[0].message.content;
48
49    // Update PR description
50    await octokit.pulls.update({ owner, repo, pull_number: prNumber, body: newDescription });
51
52    res.status(200).send('PR description updated');
53  } catch (error) {
54    console.error(error);
55    res.status(500).send('Something went wrong');
56  }
57});
58
59module.exports = router;

Step 3: Securing Webhooks

Create a utils/security.js file to verify GitHub’s webhook signature:

1const crypto = require('crypto');
2
3const verifyWebhookSignature = (req) => {
4  const signature = req.headers['x-hub-signature-256'];
5  if (!signature) throw new Error('No signature found');
6
7  const hmac = crypto.createHmac('sha256', process.env.WEBHOOK_SECRET);
8  const digest = 'sha256=' + hmac.update(JSON.stringify(req.body)).digest('hex');
9
10  if (signature !== digest) throw new Error('Invalid signature');
11};
12
13module.exports = { verifyWebhookSignature };

Step 4: Setting Up Secrets

Create a .env file in the project root:

1GITHUB_TOKEN = your_github_token
2OPENAI_API_KEY = your_openai_api_key
3WEBHOOK_SECRET = your_random_secret

Generate a random secret using:

1openssl rand -hex 20

Step 5: Deploying to Vercel

Create a vercel.json file:

1{
2  "version": 2,
3    "functions": {
4    "api/*.js": {
5      "maxDuration": 60
6    }
7  },
8  "routes": [
9    {
10      "src": "/webhook",
11      "dest": "/api/webhook.js"
12    }
13  ]
14}

Deploy using:

1vercel login
2vercel

Add your environment variables in Vercel’s Project Settings → Environment Variables.

Step 6: Configuring the GitHub Webhook

Go to your repo → Settings → Webhooks
Click Add webhook
Set the Payload URL to https://your-vercel-url/webhook
Choose application/json as the content type
Enter your WEBHOOK_SECRET
Select Pull request events
Save it 💾

Testing It Out

Open a new pull request.
The bot will generate and update the PR description!

You can check an example PR with AI description here.

What’s Next?

This is just the beginning! You could:

Add customizable summary styles
Support different AI models
Improve description formatting

Want to add AI-powered PR comments too? Check out this tutorial. 🎯

Michał Winiarski

Fullstack Software Developer