Overview

Crawl URLs which have been crawled by Search Engines, and import the number of hits. URLs found in links on these pages will not be crawled unless they are found during the web crawl.

Logz.io

Logz.io is a company that provides log management and log analysis services. Please see this article for in-depth information on how to set up your crawls with the Logz.io source.

Splunk

By connecting your Splunk account to our Log File source, we will automatically import up-to-date bot crawling information every time we crawl your website. Contact your account manager to get started by clicking “Request Access”.

We run a query on your Splunk account to retrieve, for a specified date range, all of the URLs which received traffic from search engine crawlers as well as the count of hits (separated by desktop and mobile crawlers). This data is used to populate the “Log File” and “Bot Hit” reports.

Only Splunk versions which allow API access are compatible with Lumar. Currently this is any Splunk Cloud or Splunk Enterprise account. We are unable to connect to Splunk Lite accounts.

Manual Uploads

Upload a list of URLs on your site from your log analyser tool with hits metrics for a desktop crawler and an optional mobile crawler. (Up to 100MB).

Relative URLs will inherit the project’s Primary Domain, or you can set a different base domain for each upload in the Upload Settings.

View a list of supported Log File Summary files. If your upload format is not automatically recognised, you can reformat it into this example upload , or send it to support@lumar.io and we will see if it can be supported.