Building a GitHub PR Reviewer Bot with OpenAI and Vercel

Code reviews are one of those things that we all know are important, but let’s be honest - they can be tedious. Wouldn’t it be nice to have a bot that does an initial review for you? 😉

In this tutorial, I’ll show you how to build a simple PR reviewer using OpenAI’s GPT-4 and deploy it to Vercel. It won’t replace human reviewers (yet), but it can save time by catching obvious mistakes early.

What This Bot Does?

Our bot will:

• Listen for new pull requests on GitHub via webhooks
• Fetch the PR’s diff
• Ask OpenAI for an analysis of the changes
• Post review comments automatically

Let’s get started! 🚀

Before we dive in, make sure you have:

• A GitHub account (obviously)
• A Vercel account (for easy deployment)
• An OpenAI API key (to do the actual review work)
• Node.js installed on your machine

If you don’t have these set up yet, take a moment to do so. I'll wait ⏰

Step 1: Setting Up the Project

First, create a new directory for our project and initialize it:

1$ mkdir github - pr - reviewer
2$ cd github - pr - reviewer
3$ npm init - y
4$ npm install @octokit/rest openai dotenv express

We’re installing:

• @octokit/rest to talk to GitHub
• openai to send requests to GPT-4
• dotenv to manage secrets
• express to handle incoming webhook requests

Step 2: Handling Webhooks

GitHub needs somewhere to send PR events, so we’ll create a simple webhook handler. Inside the project folder, create an `api` directory. This will contain our API function that will be later deployed to Vercel.

1$ mkdir api

Then, create api/webhook.js and add this code:

1const express = require('express');
2const { Octokit } = require('@octokit/rest');
3const OpenAI = require('openai');
4const { verifyWebhookSignature } = require('../utils/security'); // This will be created in the next step
5
6const router = express.Router();
7
8const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN });
9const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
10
11router.post('/webhook', async (req, res) => {
12  try {
13    verifyWebhookSignature(req);
14    const { pull_request, repository } = req.body;
15
16    if (!pull_request) return res.status(400).send('Not a PR event');
17
18    const owner = repository.owner.login;
19    const repo = repository.name;
20    const prNumber = pull_request.number;
21
22    //F Fetch all of the PR files
23    const { data: files } = await octokit.pulls.listFiles({ owner, repo, pull_number: prNumber });
24    // Concatenate  the diffs
25    const fileDiffs = files.map(f => `${f.filename}:\n${f.patch}`).join('\n');
26
27    // Send files to OpenAI for review
28    const aiResponse = await openai.chat.completions.create({
29      model: "gpt-4",
30      messages: [{ role: "system", content: "Analyze the following code changes and provide review comments." },
31      { role: "user", content: fileDiffs }],
32    });
33
34    // Get extract review from response
35    const reviewComment = aiResponse.choices[0].message.content;
36
37    // Create a comment in Github
38    await octokit.issues.createComment({ owner, repo, issue_number: prNumber, body: reviewComment });
39
40    res.status(200).send('Review added');
41  } catch (error) {
42    console.error(error);
43    res.status(500).send('Something went wrong');
44  }
45});
46
47module.exports = router;

This script:

1. Verifies the webhook request.
2. Fetches the changed files in the PR.
3. Sends the diff to OpenAI for analysis.
4. Posts a comment with AI-generated feedback.

We’ll also need a simple helper to verify GitHub’s webhook signature. Create utils/security.js:

1const crypto = require('crypto');
2
3const verifyWebhookSignature = (req) => {
4  const signature = req.headers['x-hub-signature-256'];
5
6  if (!signature) throw new Error('No signature found');
7
8  const hmac = crypto.createHmac('sha256', process.env.WEBHOOK_SECRET);
9  const digest = 'sha256=' + hmac.update(JSON.stringify(req.body)).digest('hex');
10
11  if (signature !== digest) throw new Error('Invalid signature');
12};
13
14module.exports = { verifyWebhookSignature };

Step 3: Setting Up Secrets

Create a .env file in the project root:

1GITHUB_TOKEN = your_github_token
2OPENAI_API_KEY = your_openai_api_key
3WEBHOOK_SECRET = your_random_secret
4

Generate a random secret and paste it into the .env with:

1openssl rand -hex 20

This ensures that only valid GitHub webhooks are accepted.

Step 4: Deploying to Vercel

Now let’s deploy this thing! First, create a `vercel.json` file:

1{
2  "version": 2,
3    "functions": {
4    "api/*.js": {
5      "maxDuration": 60
6    }
7  },
8  "routes": [
9    {
10      "src": "/webhook",
11      "dest": "/api/webhook.js"
12    }
13  ]
14}

Then run:

1$ vercel login
2$ vercel

Add the environment variables on Vercel’s dashboard under Project Settings → Environment Variables.

Step 5: Configuring the GitHub Webhook

1. Go to your repo → Settings → Webhooks
2. Click Add webhook
3. Set the Payload URL to https://your-vercel-url/webhook (you will find it in the terminal after running vervel command)
4. Choose application/json as the content type
5. Enter your WEBHOOK_SECRET
6. Select **Pull request **events
7. Save it 💾

Testing It Out

1. Create a new branch
2. Make some changes
3. Open a pull request
4. Watch the bot post a comment!

What’s Next?

This is just a starting point. You could:

• Add inline comments for specific lines
• Implement configurable review rules
• Support different AI models

You can check my code here. It does more than in this tutorial including:

• posting inline comments instead of a single one per PR
• optimized OpenAI prompt that uses less tokens

Enjoy your new AI-powered code reviewer! 🤖