Site audit

What the audit does#

The site audit crawls your site the way a search engine does. It follows links from the home page, reads your robots.txt and sitemaps, and runs a list of checks on every page it reaches. When it finishes you get:

• a score from 0 to 100,
• the issues it found, as errors, warnings and notices,
• what is new and what was fixed since the audit before,
• and every issue ranked by the pageviews of the pages it touches, because Privatus Analytics already knows which pages people visit.

Every site is audited by default. The first audit starts when the site is added. Open Site audit in the site's sidebar to see the results.

Audited pages never count as events, and audits keep running while collection is paused at the monthly limit.

Schedule and limits#

Each plan sets how many pages one audit crawls and how often a site is audited. The numbers are the Site audit pages per audit and Site audit interval (days) rows of the table on the Plans page.

• The page allowance counts HTML pages on your site. Images, scripts, stylesheets and links to other sites are checked too and do not count.
• When a crawl stops at the allowance with pages left, the overview says how many pages were found and how many were audited.
• Run now starts an audit by hand. On a paid plan you can start one every 24 hours. On the Free plan a manual audit takes the place of the scheduled one, so there is one audit per interval.
• Only one audit runs per site at a time. Cancel audit stops it, and nothing of the unfinished crawl is kept.

Crawler#

The audit is run by our own crawler, PrivatusAuditBot. It sends this User-Agent:

Mozilla/5.0 (compatible; PrivatusAuditBot/1.0; +https://privatusanalytics.com/docs/features/site-audit#crawler)

It is built to be gentle with your server:

• It fetches one page at a time per site, over one kept-alive connection, and waits between requests.
• It reads robots.txt first and obeys it, following the rules for PrivatusAuditBot and then the rules for *.
• It honors Crawl-delay, up to 10 seconds.
• When your server answers 429 or 503 it backs off, obeys Retry-After, and gives up politely if the site keeps refusing. That audit ends as Partial with the reason shown.
• It only requests pages. It never submits forms, never logs in and never runs your JavaScript.

To slow the crawler down, add a crawl delay to robots.txt:

User-agent: PrivatusAuditBot
Crawl-delay: 5

To keep it out of part of the site:

User-agent: PrivatusAuditBot
Disallow: /internal/

To block it completely, use Disallow: /. The next audit then fails with a note that robots.txt does not allow the crawler. You can also turn the audit off in its settings.

If a firewall or bot protection blocks the crawler, allow the PrivatusAuditBot user agent there.

Scope and subdomains#

The crawl starts at the site's home page. If the home page redirects (for example from example.com to www.example.com), the crawler adopts the final address. The apex domain and www always count as the same site.

By default the crawler also follows links to subdomains of the site's domain, such as blog.example.com. Turn that off in the settings to stay on one host. Everything else is an external link. An external link is checked once with a light request to see whether it still works, and its page is never crawled.

Besides links, the crawler picks up URLs from your sitemaps and from the pages that got pageviews in your analytics, so pages that nothing links to are found too.

What is stored#

The audit stores metadata about your pages: the URL, status code, response time and size, title, meta description, first H1, canonical, robots directives, language, hreflang, structured data types, social tags, a short list of response headers, word count, link counts and the links between pages.

It never stores the text or the HTML of a page. Content is read in memory to count words and to detect duplicates with a hash, and then dropped. Nothing about your visitors is involved.

The pages, links and findings are kept for the two latest finished audits of a site. Older audits keep their score and their totals per check, so the trends still have their history.

Score#

The score is a public formula:

score = 100 x (1 - (E + 0.3 x W) / N) - 5 x S

• N is the number of pages crawled on your site.
• E is the number of pages with at least one error.
• W is the number of pages with warnings and no error.
• S is the number of site-level errors (such as an unreachable robots.txt), counted up to 4, so they cost at most 20 points.

Notices never change the score. A score of 90 or more is Good, 70 to 89 Needs improvement, and anything lower Poor. Muted checks are left out.

Overview#

The overview shows the latest finished audit:

• the score with its band and the change since the audit before,
• pages crawled, and errors, warnings and notices, each with what is new and what was fixed,
• the crawled pages split into healthy, with issues, redirects, broken and blocked,
• the score over time,
• the top issues, ordered by severity and then by the pageviews of the pages they affect,
• a card per category with its own score,
• status codes, click depth and response times,
• and what the crawler read in robots.txt and your sitemaps.

While an audit runs, the page shows how many pages were crawled so far and updates by itself. The results of the audit before stay visible below. A failed audit shows why it failed.

Issues#

Issues lists one row per check that found something: the number of URLs affected, how many are new, how many were fixed, and the pageviews those pages got in the 30 days before the audit. Filter by severity, by category, or to new issues only.

Open an issue to read why it matters and how to fix it, and to see every affected URL. Sort the URLs by pageviews to fix the pages people visit first. CSV downloads the list.

Pages#

Pages is the explorer of everything the crawler saw. Filter by status code, crawl state, indexable or not, in the sitemap or not, with or without issues, click depth, or text in the URL or title. Sort by pageviews, links in, depth, response time or word count. The Resources and External links tabs list the images, scripts, stylesheets and outbound links that were checked.

Open a page to see every stored field, its issues, the first 100 links in and out with their anchor text, and where it redirects to. Page analytics opens the same page in your analytics, and the analytics page detail links back to the audit.

History and compare#

History lists every audit of the site with its status, score and counts. Open a finished audit to see its overview.

Compare two audits shows, per check, the URLs affected before and after and what was added and fixed. For an audit and the one right before it these are the exact new and fixed URLs. For audits further apart they are the net change.

Muting#

Some findings are on purpose. Mute this check on an issue page hides the check for the whole site. Mute next to a URL hides the check for that URL only. Muted findings are still recorded, but they stay out of the counts, and out of the score from the next audit on. Muted lists every mute and lets you remove it.

Muting needs the content.write permission.

Email#

When an audit finishes, the workspace's owners and admins who can see the site get a summary by email: the score, what is new, what was fixed and the top issues, with a link to the audit.

To stop it for one site, use the unsubscribe link in the email or Turn off under Audit summary emails on the audit overview. To stop it for every site, untick Site audit summary emails on your account page.

Settings#

Settings on the audit page, and the Site audit tab of the site settings, hold the same form. Changing it needs the sites.manage permission.

Checks#

Every check the audit runs today:

API and MCP#

Everything on these screens is in the JSON API and the MCP server. Add .json to a page's address, or call the tool.

Reading needs analytics.read. Starting, canceling and muting need content.write. The settings need sites.manage. The full schemas are in the API reference.