Published on

Securely consume an external API on behalf of your users

Authors

Recently, while building Auto Page Indexer I was tasked with pulling all the latest websites and their respective sitemaps for each user from Google Search Console, which requires the use of OAuth2!

In Auto Page Indexer the users have a couple of options when synchronizing their Google Search Console data:

  1. They can pull all of the latest data and store it in Auto Page Indexer manually as many times as they'd like.
  2. Or they can set their websites to auto index which requires Auto Page Indexer to have the latest Sitemaps before each "indexing session"

For the second option we must store each user's respective Refresh Token since access tokens have a short expiry (usually 15 - 30 minutes).

  1. Access Tokens
  2. Refresh Tokens
  3. The Algorithm
  4. Consuming The Google Search Console API

Access Tokens

Every API request requires a valid Access Token and given that each Access Token only lasts up to 30 minutes we must request a new access token for each index session. Refresh Tokens on the other hand usually do not expire or are very long lived i.e. 3 months to a year which allows our users to only relogin to the application after an extended period of time. Imagine if users had to login to their account once a day just to run the app, 😱 that'd be a terrible user experience!

Refresh Tokens

Now the hard part about this entire process is how do we securely store the refresh token? Even though the Refresh Token only grants us access to a single and very limited scope googleapis.com/auth/webmasters we still need to securely store it in the event that the database becomes compromised by some black hat hackers. If the Refresh Tokens are encrypted at rest then even if the database is compromised the data is essentially meaningless as long as the encryption key is stored securely and the algorithm is difficult to brute force.

The algorithm

Node.js comes with built-in support for cryptography thanks to the crypto library. I decided to go with the AES 256 algorithm for several reasons:

  1. It is super simple to implement
  2. Extremely performant
  3. AES 256 is Unbreakable by Brute Force

Here is some example code that is closely related:


import crypto from 'crypto';
const algorithm = 'aes-256-cbc';
const ENC_KEY = "bf3c199c2470cb477d907b1e0917c17b"; // set random encryption key

const phrase = "who let the dogs out";

encrypt(text) {
    const iv = crypto.randomBytes(16);
    let cipher = crypto.createCipheriv(algorithm, Buffer.from(ENC_KEY), iv);
    let encrypted = cipher.update(text);
    encrypted = Buffer.concat([encrypted, cipher.final()]);
    return { iv: iv.toString('hex'), encryptedData: encrypted.toString('hex') };
  }

  // Decrypting text
  decrypt(text) {
    let iv = Buffer.from(text.iv, 'hex');
    let encryptedText = Buffer.from(text.encryptedData, 'hex');
    let decipher = crypto.createDecipheriv(algorithm, Buffer.from(ENC_KEY), iv);
    let decrypted = decipher.update(encryptedText);
    decrypted = Buffer.concat([decrypted, decipher.final()]);
    return decrypted.toString();
  }



const encryptedKey = encrypt(phrase);
const originalPhrase = decrypt(encryptedKey);

Above we simply return a JSON object after encryption and we store the IV with the encrypted data so that we can decrypt it once it is pulled from the database. As for the decrypt function we return a stringified JSON object so we must perform a JSON.parse on the data before we can use it.

Consuming the Google Search Console external API

Here we are taking advantage of the Google Provided NPM library: googleapis. This library contains pretty much every Google endpoint you'll ever need and all of the management around them. I highly recommend using it over manually hitting the API endpoints directly.

import { google } from 'googleapis';

// use our refresh token to get an Access Token
const oauth2Client = new google.auth.OAuth2(
  process.env.GOOGLE_CLIENT_ID,
  process.env.GOOGLE_CLIENT_SECRET,
  "https://www.googleapis.com/auth/webmasters",
);
oauth2Client.setCredentials({
  refresh_token: user.googleRefreshToken, // this is already decrypted
});
const accessToken = await oauth2Client.getAccessToken();

const searchConsole = google.searchconsole({
  version: "v1",
  headers: {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer ' + accessToken.token,
  }
});

const sites = (await searchConsole.sites.list())?.data;

Once we pull the sites we can iterate through each site and check for site-owner permissions as well as the sitemaps that belong to the sites so that we can synchronize the URLs with our local database.

That pretty much sums up a simple and effective way to use OAuth to securely consume 3rd party APIs on behalf of your users with node.js

Discuss on Twitter