Running Puppeteer on Vercel

Or a possible solution on how to run Chromium in a serverless environment.

Puppeteer and Chromium

Puppeteer is a Node.js library which provides high level API to control Chrome/Chromium over the DevTools Protocol. Chromium is basically the open-sourced version of Google Chrome. The main distinction is the special features Chrome has like login to the Google account at the browser level.

Why use Puppeteer?

As it is a Node.js library, it will provide with some browser capabilities in a backend that runs on a server. For example, as a tool to server side render or use the print functionality to create screenshot or PDF of a page. It can also be use for automated testing or capture of performance tracking.

So what is the issue here?

Problem arises when you are actually running this in some kind of cloud environment. Here, in my example, it is on Vercel but the same would apply to AWS Lambda or GCP functions even though the restriction will vary from a cloud provider to another. The issue is resource limitations. In Vercel’s case, the max size of an function.

Solutions

There are 2 solutions to this challenge that I can think of. Either get a smaller version of Chromium or load Chromium from a remote URL.

Smaller Chromium

I have looked for quite some times to find a good alternative and it seems like, as of today, @sparticuz/chromium looks like the best solution. This is a package that you need to install and will be part of your bundle. It is small enough (for now) to fit in Vercel and AWS lambdas so they did a great job. And it is quite easy to use:

import chromium from '@sparticuz/chromium'
...
chromium.setGraphicsMode = false
browser = await puppeteer.launch({
  args: chromium.args,
  defaultViewport: chromium.defaultViewport,
  executablePath: await chromium.executablePath(),
  headless: chromium.headless,
})

There are a couple of problems though…

First, it will not run on ARM64 so you cannot choose those lambdas and it won’t run on a Mac with silicon (M1, M2, M3) chip. It looks to be something they are looking into as this GitHub issue suggests. If you are try to do so, you will get: Error: spawn Unknown system error -8

Secondly, if you are running it in a NextJS application, you need to tell it that you are using some server components dependency. To do so, update the next.config.js and add:

const nextConfig = {
  //...
  experimental: {
    serverComponentsExternalPackages: ['puppeteer-core', '@sparticuz/chromium']
  }
  //...
}

If you forget this, you will see errors saying you are missing Error: The input directory “…/.next/server/bin” does not exist.or similar path.

Remote URL

I found a pretty simple one called browserless that has a free plan and can be set up with Puppeteer like so:

browser = await puppeteer.connect({
  browserWSEndpoint: `wss://chrome.browserless.io?token=${process.env.BLESS_TOKEN}`
})

As you can probably guess from this code snippet, you will need to register and create a token to use at browserless. It is quite straightforward to do so.

Conclusion

“Which solution to choose?” you might ask and, as often in computing, there is not one answer that would fit them all. The remote URL is way more simpler but will cost money if you want to pass the limitations of the free tier. It still might be a good solution for your case.

The smaller chromium is free but you might get some headaches making it work the first time. Here is my complete example to launch a browser for running it locally with installed browser and on serverless with the smaller print.

let browser: Browser | undefined | null
if (process.env.NODE_ENV !== 'development') {
    const chromium = require('@sparticuz/chromium')
    // Optional: If you'd like to disable webgl, true is the default.
    chromium.setGraphicsMode = false
    const puppeteer = require('puppeteer-core')
    browser = await puppeteer.launch({
        args: chromium.args,
        defaultViewport: chromium.defaultViewport,
        executablePath: await chromium.executablePath(),
        headless: chromium.headless,
    })
} else {
    const puppeteer = require('puppeteer')
    browser = await puppeteer.launch({headless: 'new'})
}

This requires puppeteer, puppeteer-core and @sparticuz/chromium installed with npm/yarn/pnpm and the update in the next.config.js mentionned above.

As always… thanks for reading.