About automatically redelivering failed deliveries
This article describes how to write a script to find and redeliver failed deliveries for a repository webhook. For more information about failed deliveries, see Handling failed webhook deliveries.
This example shows you:
- A script that will find and redeliver failed deliveries for a repository webhook
- What credentials your script will need, and how to store the credentials securely as GitHub Actions secrets
- A GitHub Actions workflow that can securely access your credentials and run the script periodically
This example uses GitHub Actions, but you can also run this script on your server that handles webhook deliveries. For more information, see Alternative methods.
Storing credentials for the script
The built in GITHUB_TOKEN
does not have sufficient permissions to redeliver webhooks. Instead of using GITHUB_TOKEN
, this example uses a personal access token. Alternatively, instead of creating a personal access token, you can create a GitHub App and use the app's credentials to create an installation access token during the GitHub Actions workflow. For more information, see Making authenticated API requests with a GitHub App in a GitHub Actions workflow.
- Create a personal access token with the following access. For more information, see Managing your personal access tokens.
- For a fine-grained personal access token, grant the token:
- Access to the repository where your webhook was created
- Access to the repository where this workflow will run
- Write access to the repository webhooks permission
- Write access to the repository variables permission
- For a personal access token (classic), grant the token the
repo
scope.
- For a fine-grained personal access token, grant the token:
- Store your personal access token as a GitHub Actions secret in the repository where you want the workflow to run. For more information, see Using secrets in GitHub Actions.
Adding a workflow that will run the script
This section demonstrates how you can use a GitHub Actions workflow to securely access the credentials that you stored in the previous section, set environment variables, and periodically run a script to find and redeliver failed deliveries.
Copy this GitHub Actions workflow into a YAML file in the .github/workflows
directory in the repository where you want the workflow to run. Replace the placeholders in the Run script
step as described below.
# name: Redeliver failed webhook deliveries # This workflow runs every 6 hours or when manually triggered. on: schedule: - cron: '20 */6 * * *' workflow_dispatch: # This workflow will use the built in `GITHUB_TOKEN` to check out the repository contents. This grants `GITHUB_TOKEN` permission to do that. permissions: contents: read # jobs: redeliver-failed-deliveries: name: Redeliver failed deliveries runs-on: ubuntu-latest steps: # This workflow will run a script that is stored in the repository. This step checks out the repository contents so that the workflow can access the script. - name: Check out repo content uses: actions/checkout@v4 # This step sets up Node.js. The script that this workflow will run uses Node.js. - name: Setup Node.js uses: actions/setup-node@v4 with: node-version: '18.x' # This step installs the octokit library. The script that this workflow will run uses the octokit library. - name: Install dependencies run: npm install octokit # This step sets some environment variables, then runs a script to find and redeliver failed webhook deliveries. # - Replace `YOUR_SECRET_NAME` with the name of the secret where you stored your personal access token. # - Replace `YOUR_REPO_OWNER` with the owner of the repository where the webhook was created. # - Replace `YOUR_REPO_NAME` with the name of the repository where the webhook was created. # - Replace `YOUR_HOOK_ID` with the ID of the webhook. # - Replace `YOUR_LAST_REDELIVERY_VARIABLE_NAME` with the name that you want to use for a configuration variable that will be stored in the repository where this workflow is stored. The name can be any string that contains only alphanumeric characters and `_`, and does not start with `GITHUB_` or a number. For more information, see "[AUTOTITLE](/actions/learn-github-actions/variables#defining-configuration-variables-for-multiple-workflows)." - name: Run script env: TOKEN: ${{ secrets.YOUR_SECRET_NAME }} REPO_OWNER: 'YOUR_REPO_OWNER' REPO_NAME: 'YOUR_REPO_NAME' HOOK_ID: 'YOUR_HOOK_ID' LAST_REDELIVERY_VARIABLE_NAME: 'YOUR_LAST_REDELIVERY_VARIABLE_NAME' WORKFLOW_REPO_NAME: ${{ github.event.repository.name }} WORKFLOW_REPO_OWNER: ${{ github.repository_owner }} run: | node .github/workflows/scripts/redeliver-failed-deliveries.js
name: Redeliver failed webhook deliveries
on:
schedule:
- cron: '20 */6 * * *'
workflow_dispatch:
This workflow runs every 6 hours or when manually triggered.
permissions:
contents: read
This workflow will use the built in GITHUB_TOKEN
to check out the repository contents. This grants GITHUB_TOKEN
permission to do that.
jobs:
redeliver-failed-deliveries:
name: Redeliver failed deliveries
runs-on: ubuntu-latest
steps:
- name: Check out repo content
uses: actions/checkout@v4
This workflow will run a script that is stored in the repository. This step checks out the repository contents so that the workflow can access the script.
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '18.x'
This step sets up Node.js. The script that this workflow will run uses Node.js.
- name: Install dependencies
run: npm install octokit
This step installs the octokit library. The script that this workflow will run uses the octokit library.
- name: Run script
env:
TOKEN: ${{ secrets.YOUR_SECRET_NAME }}
REPO_OWNER: 'YOUR_REPO_OWNER'
REPO_NAME: 'YOUR_REPO_NAME'
HOOK_ID: 'YOUR_HOOK_ID'
LAST_REDELIVERY_VARIABLE_NAME: 'YOUR_LAST_REDELIVERY_VARIABLE_NAME'
WORKFLOW_REPO_NAME: ${{ github.event.repository.name }}
WORKFLOW_REPO_OWNER: ${{ github.repository_owner }}
run: |
node .github/workflows/scripts/redeliver-failed-deliveries.js
This step sets some environment variables, then runs a script to find and redeliver failed webhook deliveries.
- Replace
YOUR_SECRET_NAME
with the name of the secret where you stored your personal access token. - Replace
YOUR_REPO_OWNER
with the owner of the repository where the webhook was created. - Replace
YOUR_REPO_NAME
with the name of the repository where the webhook was created. - Replace
YOUR_HOOK_ID
with the ID of the webhook. - Replace
YOUR_LAST_REDELIVERY_VARIABLE_NAME
with the name that you want to use for a configuration variable that will be stored in the repository where this workflow is stored. The name can be any string that contains only alphanumeric characters and_
, and does not start withGITHUB_
or a number. For more information, see "Store information in variables."
#
name: Redeliver failed webhook deliveries
# This workflow runs every 6 hours or when manually triggered.
on:
schedule:
- cron: '20 */6 * * *'
workflow_dispatch:
# This workflow will use the built in `GITHUB_TOKEN` to check out the repository contents. This grants `GITHUB_TOKEN` permission to do that.
permissions:
contents: read
#
jobs:
redeliver-failed-deliveries:
name: Redeliver failed deliveries
runs-on: ubuntu-latest
steps:
# This workflow will run a script that is stored in the repository. This step checks out the repository contents so that the workflow can access the script.
- name: Check out repo content
uses: actions/checkout@v4
# This step sets up Node.js. The script that this workflow will run uses Node.js.
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '18.x'
# This step installs the octokit library. The script that this workflow will run uses the octokit library.
- name: Install dependencies
run: npm install octokit
# This step sets some environment variables, then runs a script to find and redeliver failed webhook deliveries.
# - Replace `YOUR_SECRET_NAME` with the name of the secret where you stored your personal access token.
# - Replace `YOUR_REPO_OWNER` with the owner of the repository where the webhook was created.
# - Replace `YOUR_REPO_NAME` with the name of the repository where the webhook was created.
# - Replace `YOUR_HOOK_ID` with the ID of the webhook.
# - Replace `YOUR_LAST_REDELIVERY_VARIABLE_NAME` with the name that you want to use for a configuration variable that will be stored in the repository where this workflow is stored. The name can be any string that contains only alphanumeric characters and `_`, and does not start with `GITHUB_` or a number. For more information, see "[AUTOTITLE](/actions/learn-github-actions/variables#defining-configuration-variables-for-multiple-workflows)."
- name: Run script
env:
TOKEN: ${{ secrets.YOUR_SECRET_NAME }}
REPO_OWNER: 'YOUR_REPO_OWNER'
REPO_NAME: 'YOUR_REPO_NAME'
HOOK_ID: 'YOUR_HOOK_ID'
LAST_REDELIVERY_VARIABLE_NAME: 'YOUR_LAST_REDELIVERY_VARIABLE_NAME'
WORKFLOW_REPO_NAME: ${{ github.event.repository.name }}
WORKFLOW_REPO_OWNER: ${{ github.repository_owner }}
run: |
node .github/workflows/scripts/redeliver-failed-deliveries.js
Adding the script
This section demonstrates how you can write a script to find and redeliver failed deliveries.
Copy this script into a file called .github/workflows/scripts/redeliver-failed-deliveries.js
in the same repository where you saved the GitHub Actions workflow file above.
// This script uses GitHub's Octokit SDK to make API requests. For more information, see "[AUTOTITLE](/rest/guides/scripting-with-the-rest-api-and-javascript)." const { Octokit } = require("octokit"); // async function checkAndRedeliverWebhooks() { // Get the values of environment variables that were set by the GitHub Actions workflow. const TOKEN = process.env.TOKEN; const REPO_OWNER = process.env.REPO_OWNER; const REPO_NAME = process.env.REPO_NAME; const HOOK_ID = process.env.HOOK_ID; const LAST_REDELIVERY_VARIABLE_NAME = process.env.LAST_REDELIVERY_VARIABLE_NAME; const WORKFLOW_REPO_NAME = process.env.WORKFLOW_REPO_NAME; const WORKFLOW_REPO_OWNER = process.env.WORKFLOW_REPO_OWNER; // Create an instance of `Octokit` using the token values that were set in the GitHub Actions workflow. const octokit = new Octokit({ auth: TOKEN, }); try { // Get the last time that this script ran from the configuration variable. If the variable is not defined, use the current time minus 24 hours. const lastStoredRedeliveryTime = await getVariable({ variableName: LAST_REDELIVERY_VARIABLE_NAME, repoOwner: WORKFLOW_REPO_OWNER, repoName: WORKFLOW_REPO_NAME, octokit, }); const lastWebhookRedeliveryTime = lastStoredRedeliveryTime || (Date.now() - (24 * 60 * 60 * 1000)).toString(); // Record the time that this script started redelivering webhooks. const newWebhookRedeliveryTime = Date.now().toString(); // Get the webhook deliveries that were delivered after `lastWebhookRedeliveryTime`. const deliveries = await fetchWebhookDeliveriesSince({ lastWebhookRedeliveryTime, repoOwner: REPO_OWNER, repoName: REPO_NAME, hookId: HOOK_ID, octokit, }); // Consolidate deliveries that have the same globally unique identifier (GUID). The GUID is constant across redeliveries of the same delivery. let deliveriesByGuid = {}; for (const delivery of deliveries) { deliveriesByGuid[delivery.guid] ? deliveriesByGuid[delivery.guid].push(delivery) : (deliveriesByGuid[delivery.guid] = [delivery]); } // For each GUID value, if no deliveries for that GUID have been successfully delivered within the time frame, get the delivery ID of one of the deliveries with that GUID. // // This will prevent duplicate redeliveries if a delivery has failed multiple times. // This will also prevent redelivery of failed deliveries that have already been successfully redelivered. let failedDeliveryIDs = []; for (const guid in deliveriesByGuid) { const deliveries = deliveriesByGuid[guid]; const anySucceeded = deliveries.some( (delivery) => delivery.status === "OK" ); if (!anySucceeded) { failedDeliveryIDs.push(deliveries[0].id); } } // Redeliver any failed deliveries. for (const deliveryId of failedDeliveryIDs) { await redeliverWebhook({ deliveryId, repoOwner: REPO_OWNER, repoName: REPO_NAME, hookId: HOOK_ID, octokit, }); } // Update the configuration variable (or create the variable if it doesn't already exist) to store the time that this script started. // This value will be used next time this script runs. await updateVariable({ variableName: LAST_REDELIVERY_VARIABLE_NAME, value: newWebhookRedeliveryTime, variableExists: Boolean(lastStoredRedeliveryTime), repoOwner: WORKFLOW_REPO_OWNER, repoName: WORKFLOW_REPO_NAME, octokit, }); // Log the number of redeliveries. console.log( `Redelivered ${ failedDeliveryIDs.length } failed webhook deliveries out of ${ deliveries.length } total deliveries since ${Date(lastWebhookRedeliveryTime)}.` ); } catch (error) { // If there was an error, log the error so that it appears in the workflow run log, then throw the error so that the workflow run registers as a failure. if (error.response) { console.error( `Failed to check and redeliver webhooks: ${error.response.data.message}` ); } console.error(error); throw(error); } } // This function will fetch all of the webhook deliveries that were delivered since `lastWebhookRedeliveryTime`. // It uses the `octokit.paginate.iterator()` method to iterate through paginated results. For more information, see "[AUTOTITLE](/rest/guides/scripting-with-the-rest-api-and-javascript#making-paginated-requests)." // // If a page of results includes deliveries that occurred before `lastWebhookRedeliveryTime`, // it will store only the deliveries that occurred after `lastWebhookRedeliveryTime` and then stop. // Otherwise, it will store all of the deliveries from the page and request the next page. async function fetchWebhookDeliveriesSince({ lastWebhookRedeliveryTime, repoOwner, repoName, hookId, octokit, }) { const iterator = octokit.paginate.iterator( "GET /repos/{owner}/{repo}/hooks/{hook_id}/deliveries", { owner: repoOwner, repo: repoName, hook_id: hookId, per_page: 100, headers: { "x-github-api-version": "2022-11-28", }, } ); const deliveries = []; for await (const { data } of iterator) { const oldestDeliveryTimestamp = new Date( data[data.length - 1].delivered_at ).getTime(); if (oldestDeliveryTimestamp < lastWebhookRedeliveryTime) { for (const delivery of data) { if ( new Date(delivery.delivered_at).getTime() > lastWebhookRedeliveryTime ) { deliveries.push(delivery); } else { break; } } break; } else { deliveries.push(...data); } } return deliveries; } // This function will redeliver a failed webhook delivery. async function redeliverWebhook({ deliveryId, repoOwner, repoName, hookId, octokit, }) { await octokit.request( "POST /repos/{owner}/{repo}/hooks/{hook_id}/deliveries/{delivery_id}/attempts", { owner: repoOwner, repo: repoName, hook_id: hookId, delivery_id: deliveryId, } ); } // This function gets the value of a configuration variable. // If the variable does not exist, the endpoint returns a 404 response and this function returns `undefined`. async function getVariable({ variableName, repoOwner, repoName, octokit }) { try { const { data: { value }, } = await octokit.request( "GET /repos/{owner}/{repo}/actions/variables/{name}", { owner: repoOwner, repo: repoName, name: variableName, } ); return value; } catch (error) { if (error.status === 404) { return undefined; } else { throw error; } } } // This function will update a configuration variable (or create the variable if it doesn't already exist). For more information, see "[AUTOTITLE](/actions/learn-github-actions/variables#defining-configuration-variables-for-multiple-workflows)." async function updateVariable({ variableName, value, variableExists, repoOwner, repoName, octokit, }) { if (variableExists) { await octokit.request( "PATCH /repos/{owner}/{repo}/actions/variables/{name}", { owner: repoOwner, repo: repoName, name: variableName, value: value, } ); } else { await octokit.request("POST /repos/{owner}/{repo}/actions/variables", { owner: repoOwner, repo: repoName, name: variableName, value: value, }); } } // This will execute the `checkAndRedeliverWebhooks` function. (async () => { await checkAndRedeliverWebhooks(); })();
const { Octokit } = require("octokit");
This script uses GitHub's Octokit SDK to make API requests. For more information, see "Scripting with the REST API and JavaScript."
async function checkAndRedeliverWebhooks() {
const TOKEN = process.env.TOKEN;
const REPO_OWNER = process.env.REPO_OWNER;
const REPO_NAME = process.env.REPO_NAME;
const HOOK_ID = process.env.HOOK_ID;
const LAST_REDELIVERY_VARIABLE_NAME = process.env.LAST_REDELIVERY_VARIABLE_NAME;
const WORKFLOW_REPO_NAME = process.env.WORKFLOW_REPO_NAME;
const WORKFLOW_REPO_OWNER = process.env.WORKFLOW_REPO_OWNER;
Get the values of environment variables that were set by the GitHub Actions workflow.
const octokit = new Octokit({
auth: TOKEN,
});
try {
Create an instance of Octokit
using the token values that were set in the GitHub Actions workflow.
const lastStoredRedeliveryTime = await getVariable({
variableName: LAST_REDELIVERY_VARIABLE_NAME,
repoOwner: WORKFLOW_REPO_OWNER,
repoName: WORKFLOW_REPO_NAME,
octokit,
});
const lastWebhookRedeliveryTime = lastStoredRedeliveryTime || (Date.now() - (24 * 60 * 60 * 1000)).toString();
Get the last time that this script ran from the configuration variable. If the variable is not defined, use the current time minus 24 hours.
const newWebhookRedeliveryTime = Date.now().toString();
Record the time that this script started redelivering webhooks.
const deliveries = await fetchWebhookDeliveriesSince({
lastWebhookRedeliveryTime,
repoOwner: REPO_OWNER,
repoName: REPO_NAME,
hookId: HOOK_ID,
octokit,
});
Get the webhook deliveries that were delivered after lastWebhookRedeliveryTime
.
let deliveriesByGuid = {};
for (const delivery of deliveries) {
deliveriesByGuid[delivery.guid]
? deliveriesByGuid[delivery.guid].push(delivery)
: (deliveriesByGuid[delivery.guid] = [delivery]);
}
Consolidate deliveries that have the same globally unique identifier (GUID). The GUID is constant across redeliveries of the same delivery.
let failedDeliveryIDs = [];
for (const guid in deliveriesByGuid) {
const deliveries = deliveriesByGuid[guid];
const anySucceeded = deliveries.some(
(delivery) => delivery.status === "OK"
);
if (!anySucceeded) {
failedDeliveryIDs.push(deliveries[0].id);
}
}
For each GUID value, if no deliveries for that GUID have been successfully delivered within the time frame, get the delivery ID of one of the deliveries with that GUID.
This will prevent duplicate redeliveries if a delivery has failed multiple times. This will also prevent redelivery of failed deliveries that have already been successfully redelivered.
for (const deliveryId of failedDeliveryIDs) {
await redeliverWebhook({
deliveryId,
repoOwner: REPO_OWNER,
repoName: REPO_NAME,
hookId: HOOK_ID,
octokit,
});
}
Redeliver any failed deliveries.
await updateVariable({
variableName: LAST_REDELIVERY_VARIABLE_NAME,
value: newWebhookRedeliveryTime,
variableExists: Boolean(lastStoredRedeliveryTime),
repoOwner: WORKFLOW_REPO_OWNER,
repoName: WORKFLOW_REPO_NAME,
octokit,
});
Update the configuration variable (or create the variable if it doesn't already exist) to store the time that this script started. This value will be used next time this script runs.
console.log(
`Redelivered ${
failedDeliveryIDs.length
} failed webhook deliveries out of ${
deliveries.length
} total deliveries since ${Date(lastWebhookRedeliveryTime)}.`
);
} catch (error) {
Log the number of redeliveries.
if (error.response) {
console.error(
`Failed to check and redeliver webhooks: ${error.response.data.message}`
);
}
console.error(error);
throw(error);
}
}
If there was an error, log the error so that it appears in the workflow run log, then throw the error so that the workflow run registers as a failure.
async function fetchWebhookDeliveriesSince({
lastWebhookRedeliveryTime,
repoOwner,
repoName,
hookId,
octokit,
}) {
const iterator = octokit.paginate.iterator(
"GET /repos/{owner}/{repo}/hooks/{hook_id}/deliveries",
{
owner: repoOwner,
repo: repoName,
hook_id: hookId,
per_page: 100,
headers: {
"x-github-api-version": "2022-11-28",
},
}
);
const deliveries = [];
for await (const { data } of iterator) {
const oldestDeliveryTimestamp = new Date(
data[data.length - 1].delivered_at
).getTime();
if (oldestDeliveryTimestamp < lastWebhookRedeliveryTime) {
for (const delivery of data) {
if (
new Date(delivery.delivered_at).getTime() > lastWebhookRedeliveryTime
) {
deliveries.push(delivery);
} else {
break;
}
}
break;
} else {
deliveries.push(...data);
}
}
return deliveries;
}
This function will fetch all of the webhook deliveries that were delivered since lastWebhookRedeliveryTime
.
It uses the octokit.paginate.iterator()
method to iterate through paginated results. For more information, see "Scripting with the REST API and JavaScript."
If a page of results includes deliveries that occurred before lastWebhookRedeliveryTime
,
it will store only the deliveries that occurred after lastWebhookRedeliveryTime
and then stop.
Otherwise, it will store all of the deliveries from the page and request the next page.
async function redeliverWebhook({
deliveryId,
repoOwner,
repoName,
hookId,
octokit,
}) {
await octokit.request(
"POST /repos/{owner}/{repo}/hooks/{hook_id}/deliveries/{delivery_id}/attempts",
{
owner: repoOwner,
repo: repoName,
hook_id: hookId,
delivery_id: deliveryId,
}
);
}
This function will redeliver a failed webhook delivery.
async function getVariable({ variableName, repoOwner, repoName, octokit }) {
try {
const {
data: { value },
} = await octokit.request(
"GET /repos/{owner}/{repo}/actions/variables/{name}",
{
owner: repoOwner,
repo: repoName,
name: variableName,
}
);
return value;
} catch (error) {
if (error.status === 404) {
return undefined;
} else {
throw error;
}
}
}
This function gets the value of a configuration variable.
If the variable does not exist, the endpoint returns a 404 response and this function returns undefined
.
async function updateVariable({
variableName,
value,
variableExists,
repoOwner,
repoName,
octokit,
}) {
if (variableExists) {
await octokit.request(
"PATCH /repos/{owner}/{repo}/actions/variables/{name}",
{
owner: repoOwner,
repo: repoName,
name: variableName,
value: value,
}
);
} else {
await octokit.request("POST /repos/{owner}/{repo}/actions/variables", {
owner: repoOwner,
repo: repoName,
name: variableName,
value: value,
});
}
}
This function will update a configuration variable (or create the variable if it doesn't already exist). For more information, see "Store information in variables."
(async () => {
await checkAndRedeliverWebhooks();
})();
This will execute the checkAndRedeliverWebhooks
function.
// This script uses GitHub's Octokit SDK to make API requests. For more information, see "[AUTOTITLE](/rest/guides/scripting-with-the-rest-api-and-javascript)."
const { Octokit } = require("octokit");
//
async function checkAndRedeliverWebhooks() {
// Get the values of environment variables that were set by the GitHub Actions workflow.
const TOKEN = process.env.TOKEN;
const REPO_OWNER = process.env.REPO_OWNER;
const REPO_NAME = process.env.REPO_NAME;
const HOOK_ID = process.env.HOOK_ID;
const LAST_REDELIVERY_VARIABLE_NAME = process.env.LAST_REDELIVERY_VARIABLE_NAME;
const WORKFLOW_REPO_NAME = process.env.WORKFLOW_REPO_NAME;
const WORKFLOW_REPO_OWNER = process.env.WORKFLOW_REPO_OWNER;
// Create an instance of `Octokit` using the token values that were set in the GitHub Actions workflow.
const octokit = new Octokit({
auth: TOKEN,
});
try {
// Get the last time that this script ran from the configuration variable. If the variable is not defined, use the current time minus 24 hours.
const lastStoredRedeliveryTime = await getVariable({
variableName: LAST_REDELIVERY_VARIABLE_NAME,
repoOwner: WORKFLOW_REPO_OWNER,
repoName: WORKFLOW_REPO_NAME,
octokit,
});
const lastWebhookRedeliveryTime = lastStoredRedeliveryTime || (Date.now() - (24 * 60 * 60 * 1000)).toString();
// Record the time that this script started redelivering webhooks.
const newWebhookRedeliveryTime = Date.now().toString();
// Get the webhook deliveries that were delivered after `lastWebhookRedeliveryTime`.
const deliveries = await fetchWebhookDeliveriesSince({
lastWebhookRedeliveryTime,
repoOwner: REPO_OWNER,
repoName: REPO_NAME,
hookId: HOOK_ID,
octokit,
});
// Consolidate deliveries that have the same globally unique identifier (GUID). The GUID is constant across redeliveries of the same delivery.
let deliveriesByGuid = {};
for (const delivery of deliveries) {
deliveriesByGuid[delivery.guid]
? deliveriesByGuid[delivery.guid].push(delivery)
: (deliveriesByGuid[delivery.guid] = [delivery]);
}
// For each GUID value, if no deliveries for that GUID have been successfully delivered within the time frame, get the delivery ID of one of the deliveries with that GUID.
//
// This will prevent duplicate redeliveries if a delivery has failed multiple times.
// This will also prevent redelivery of failed deliveries that have already been successfully redelivered.
let failedDeliveryIDs = [];
for (const guid in deliveriesByGuid) {
const deliveries = deliveriesByGuid[guid];
const anySucceeded = deliveries.some(
(delivery) => delivery.status === "OK"
);
if (!anySucceeded) {
failedDeliveryIDs.push(deliveries[0].id);
}
}
// Redeliver any failed deliveries.
for (const deliveryId of failedDeliveryIDs) {
await redeliverWebhook({
deliveryId,
repoOwner: REPO_OWNER,
repoName: REPO_NAME,
hookId: HOOK_ID,
octokit,
});
}
// Update the configuration variable (or create the variable if it doesn't already exist) to store the time that this script started.
// This value will be used next time this script runs.
await updateVariable({
variableName: LAST_REDELIVERY_VARIABLE_NAME,
value: newWebhookRedeliveryTime,
variableExists: Boolean(lastStoredRedeliveryTime),
repoOwner: WORKFLOW_REPO_OWNER,
repoName: WORKFLOW_REPO_NAME,
octokit,
});
// Log the number of redeliveries.
console.log(
`Redelivered ${
failedDeliveryIDs.length
} failed webhook deliveries out of ${
deliveries.length
} total deliveries since ${Date(lastWebhookRedeliveryTime)}.`
);
} catch (error) {
// If there was an error, log the error so that it appears in the workflow run log, then throw the error so that the workflow run registers as a failure.
if (error.response) {
console.error(
`Failed to check and redeliver webhooks: ${error.response.data.message}`
);
}
console.error(error);
throw(error);
}
}
// This function will fetch all of the webhook deliveries that were delivered since `lastWebhookRedeliveryTime`.
// It uses the `octokit.paginate.iterator()` method to iterate through paginated results. For more information, see "[AUTOTITLE](/rest/guides/scripting-with-the-rest-api-and-javascript#making-paginated-requests)."
//
// If a page of results includes deliveries that occurred before `lastWebhookRedeliveryTime`,
// it will store only the deliveries that occurred after `lastWebhookRedeliveryTime` and then stop.
// Otherwise, it will store all of the deliveries from the page and request the next page.
async function fetchWebhookDeliveriesSince({
lastWebhookRedeliveryTime,
repoOwner,
repoName,
hookId,
octokit,
}) {
const iterator = octokit.paginate.iterator(
"GET /repos/{owner}/{repo}/hooks/{hook_id}/deliveries",
{
owner: repoOwner,
repo: repoName,
hook_id: hookId,
per_page: 100,
headers: {
"x-github-api-version": "2022-11-28",
},
}
);
const deliveries = [];
for await (const { data } of iterator) {
const oldestDeliveryTimestamp = new Date(
data[data.length - 1].delivered_at
).getTime();
if (oldestDeliveryTimestamp < lastWebhookRedeliveryTime) {
for (const delivery of data) {
if (
new Date(delivery.delivered_at).getTime() > lastWebhookRedeliveryTime
) {
deliveries.push(delivery);
} else {
break;
}
}
break;
} else {
deliveries.push(...data);
}
}
return deliveries;
}
// This function will redeliver a failed webhook delivery.
async function redeliverWebhook({
deliveryId,
repoOwner,
repoName,
hookId,
octokit,
}) {
await octokit.request(
"POST /repos/{owner}/{repo}/hooks/{hook_id}/deliveries/{delivery_id}/attempts",
{
owner: repoOwner,
repo: repoName,
hook_id: hookId,
delivery_id: deliveryId,
}
);
}
// This function gets the value of a configuration variable.
// If the variable does not exist, the endpoint returns a 404 response and this function returns `undefined`.
async function getVariable({ variableName, repoOwner, repoName, octokit }) {
try {
const {
data: { value },
} = await octokit.request(
"GET /repos/{owner}/{repo}/actions/variables/{name}",
{
owner: repoOwner,
repo: repoName,
name: variableName,
}
);
return value;
} catch (error) {
if (error.status === 404) {
return undefined;
} else {
throw error;
}
}
}
// This function will update a configuration variable (or create the variable if it doesn't already exist). For more information, see "[AUTOTITLE](/actions/learn-github-actions/variables#defining-configuration-variables-for-multiple-workflows)."
async function updateVariable({
variableName,
value,
variableExists,
repoOwner,
repoName,
octokit,
}) {
if (variableExists) {
await octokit.request(
"PATCH /repos/{owner}/{repo}/actions/variables/{name}",
{
owner: repoOwner,
repo: repoName,
name: variableName,
value: value,
}
);
} else {
await octokit.request("POST /repos/{owner}/{repo}/actions/variables", {
owner: repoOwner,
repo: repoName,
name: variableName,
value: value,
});
}
}
// This will execute the `checkAndRedeliverWebhooks` function.
(async () => {
await checkAndRedeliverWebhooks();
})();
Testing the script
You can manually trigger your workflow to test the script. For more information, see Manually running a workflow and Using workflow run logs.
Alternative methods
This example used GitHub Actions to securely store credentials and to run the script on a schedule. However, if you prefer to run this script on your server that handles webhook deliveries, you can:
- Store the credentials in another secure manner, such as a secret manager like Azure key vault. You will also need to update the script to access the credentials from their new location.
- Run the script on a schedule on your server, for example by using a cron job or task scheduler.
- Update the script to store the last run time somewhere that your server can access and update. If you choose not to store the last run time as a GitHub Actions secret, you can remove the API calls to access and update the configuration variable.