Skip to main content

Search

Welcome to the Lumar Knowledge Base

Storing HTML and Screenshots for SEO Crawls

Andrew Levey
  • Updated

To help increase accuracy for page measurements and provide more information to investigate anomalies in the data—especially for important pages—you can store screenshots and the HTML for SEO crawls for 90 days from when the crawl is run. 

You can view a step-by-step guide on how to enable this feature, or read on to find out more.

Storing the HTML and/or screenshot can be enabled in the advanced settings in step 4 of the crawl setup process using the relevant checkboxes in the Report Settings section. 

Please note there is a cost associated to storing HTML and/or screenshots, so enabling will each use an additional 0.1 credits per URL (i.e. 0.2 credits if both are enabled). As mentioned above, the stored HTML and screenshot will also expire 90 days after the crawl is run.

Screenshot of the Report Setup section of Advanced Settings in Step 4 of the Example crawl setup process, with the option to check boxes for save html and save screenshot checked and highlighted.

Once you’ve enabled the option to save HTML and/or screenshot you can go ahead and run your crawl. Once the crawl completes, you’ll find the screenshot and HTML in the resource detail view. 

To do this, click on the URL in a report in Analyze:

Screenshot of the Lumar Analyze broken pages report, with the URL field highlighted.

This will take you to the Resource Detail view where you’ll find a number of metrics related to this specific page. At the top of the left-hand navigation, you’ll see an option for HTML which will show you the Static and Rendered HTML for the page. When you click into either the Static or Rendered HTML, you can also use Ctrl+F or Command+F to bring up the search option. You’ll also see the expiry date for the HTML just above the Rendered HTML option, on the right.

Screenshot of the stored HTML page in the resource detail view, with the HTML option in the left-hand navigation, find window in the static HTML, and expiry date of the HTML highlighted.

Underneath HTML in the left-hand navigation, you’ll also see the option to view the Screenshot. Again, on the right-hand side just above the screenshot, you’ll see its expiry date. The screenshot provided will match the user agent used in the crawl (i.e. mobile or desktop, etc.) and will capture the full viewport for the user agent, not just what's above the fold. 

Currently, images are not fetched by default during crawling to reduce network traffic costs, so screenshots will be represented without images. We'll be introducing an option to enable images in the project settings in the next few weeks, to allow more accurate screenshots.

Screenshot of the stored screenshot page in the resource detail view, with the Screenshot option in the left-hand navigation, and the expiry date of the HTML highlighted.

If you need to disable the HTML or Screenshot storage, simply go back to the Advanced Settings in the crawl setup, and uncheck the boxes.