Imagine this: You're documenting an unreleased feature, and your documentation requires screenshots. However, you're working in an internal environment that includes features you don't want to reveal to the public. What do you do?

We faced this exact situation during the API Summit release when we needed to capture screenshots of the new Kong Konnect Plus dashboard while our internal environment also showcased Mesh Manager features.

In days gone by, tackling such a challenge meant delving into browser dev tools, meticulously hiding elements each time, and redoing the process after refreshing the browser. Now, there’s a better way. Let me introduce you to shot-scraper.

## Kong ❤️ shot-scraper 

At Kong, we’ve adopted [shot-scraper](https://shot-scraper.datasette.io/en/stable/)shot-scraper, a tool for taking automated screenshots of websites, as part of our core workflow. We’ve even extended shot-scraper with [the capability to run macros](https://github.com/simonw/shot-scraper/pull/111)the capability to run macros.

Macros are predefined JavaScript functions that can be executed at specific moments, enhancing your ability to interact with the web page and set up conditions prior to capturing the screenshot. 

To solve the issue outlined above, I crafted a simple shot-scraper macro to remove elements from the page:

- type: function
  name: removeElementBySelector
  javascript: |
    function (selector) {
      const elementToDelete = document.querySelector(selector);
      if (elementToDelete) {
        elementToDelete.parentNode.removeChild(elementToDelete);
      }
    }

This macro was used in the following shot-scraper script to hide the Mesh Manager section in the sidebar:

- output: ../../app/assets/images/docs/konnect/konnect-control-plane-dashboard.png
  url: https://cloud.konghq.com/us/gateway-manager/
  wait: 4000
  width: 1920
  height: 800
  javascript: |
    removeElementBySelector('a.sidebar-item-link[href="/us/mesh-manager/"]');

Without shot-scraper, we would have had to edit the page manually using developer tools each time the page loaded:

## What else can shot-scraper do?

We use shot-scraper for a variety of different tasks:

1. Take a standard screenshot of a dashboard:

- output: ../../app/assets/images/docs/konnect/konnect-control-plane-dashboard.png
  url: https://cloud.konghq.com/us/gateway-manager/
  wait: 4000
  width: 1920
  height: 800

2. Update the page with random values (in this case the number of requests):

- output: ../../app/_assets/images/docs/konnect/konnect-dashboard.png
  url: https://cloud.konghq.com/us/overview/
  wait: 4000
  width: 1920
  height: 1080
  javascript: |
    async () => {
    updateDivWithRandomNumberInRange('div[data-v-ee980c4c][style="font-size: 22px;"]', 30000, 600000);
    }

3. Add annotations to specific elements on a page that you can reference in written documentation:

- output: ../../app/_assets/images/docs/konnect/konnect-vaults.png
  url: https://cloud.konghq.com/us/runtime-manager/988f1fa1-fd93-4ef7-a1f5-c481bd86e945/vaults
  wait: 4000
  width: 1920
  height: 1080
  javascript: |
    async () => {
      const sidebarItem = document.querySelector("#subnav-gateway-manager > li.sidebar-item-secondary.active > a");
      const prefixItem = getTestId('prefix');
      const addItem = getTestId('toolbar-add-vault');
      const menuItem = getTestId('overflow-actions-button');

      annotateNumber(sidebarItem, { number: 1, position: { left: "100px" } });
      annotateNumber(addItem, { number: 2, position: { right: "-50px" } });
      annotateNumber(prefixItem, { number: 3, position: { left: "-35px", top: "4px" } });
      annotateNumber(menuItem, { number: 4, position: { right: "-35px" } });
    }

4. Interact with the page like a user. Here, we configure a graph in our Analytics dashboard builder:

- output: ../../app/_assets/images/docs/konnect/custom-reports/api-usage-by-application.png
  url: https://cloud.konghq.com/us/analytics/reports/create
  wait: 4000
  width: 1920
  height: 1080
  javascript: |
    async () => {
    // Configure the report we want to see
    const form = document.querySelector(".report-form form");

    // Set the report title
    form.querySelector("h1").textContent = "API Usage by Application (last 30 days)";
    await sleep(500);

    // Select "Vertical Bar Chart" and wait for the UI to update
    await activateByTestId('chart-type-VerticalBar');

    // And the time duration
    await activateByTestId('k-datetime-picker-display');
    await activateByTestId('select-timeframe-30d');
    await activateByTestId('k-datetime-picker-submit');

    // Group by Service
    form.querySelector(".select-group-by .k-select-input").click();
    await sleep(200);
    await activateByTestId('k-select-item-Service');

    // Then by Application
    form.querySelector(".select-then-by .k-select-input").click();
    await sleep(200);
    await activateByTestId('k-select-item-Application');

    // Wait for the chart to render
    await sleep(1000);

    // The "selector" option didn't work well for some reason. This takes a nice screenshot without sidebar/header
    cleanupPage();
    }

Using shot-scraper, our docs team can write scripts once and have screenshots that are up-to-date, accurate, and reproducible by anyone on the team. 

## Why do you need automated screenshots for docs?

Screenshots in docs, in my experience, struggle with the following things: 

• - **Inaccuracy**: The UI evolves, but the screenshots remain static.
• - **Inconsistent appearances**: Screenshots vary in size, resolution, and quality.
• - **Resource consumption**: Keeping these elements synchronized with the rapid pace of UI development consumes valuable time.

So, what's the solution when you want accurate screenshots without spending hours on cropping and resizing? Automation!

This is the only way to ensure that you are insulated from the pace of UI development and design while still being able to provide screenshots in your documentation. 

For the Kong Docs team, shot-scraper is an invaluable tool that allows us to provide accurate documentation including screenshots to our readers. 

## Tips for automating screenshot workflows

If you're considering making the transition from manual to automated screenshot workflows, here are some practical tips:

• - **Prune existing screenshots**: Ensure that every screenshot serves a clear purpose. Screenshots are essential communication tools in documentation; they should convey key concepts, generate user interest, or showcase a visually appealing UI. However, they shouldn't compensate for poor UX design.
• - **Build a process**: Build a process around shot-scraper, or build shot-scraper into an existing process. Whether that’s a release process that concludes with running shot-scraper. Or a GitHub action that automates running shot-scraper. Creating a dozen scripts to never run them is just as much of a waste of resources as trying to configure your dev tools to take the same screenshot several times. 
• - **Advocate for your approach**: Advocating for this approach is a vital part of any docs-as-code strategy. Just like other aspects like version control, automated testing, and plain text documentation, it requires a collective effort. 

To successfully implement shot-scraper, everyone on your team should understand its value. Encourage your team members to embrace automation by showcasing the significant time savings it offers and its contribution to documentation accuracy. This approach can benefit not only the documentation team but also other departments, such as design teams, who can confidently make UI changes without burdening the documentation process. 

Shot-scraper as part of your docs arsenal can help bring efficiency, accuracy, and efficacy to your documentation.