Fetch status and hard-to-reach pages

Click Managing in the menu to see each URL’s fetch status — see Monitoring. Most pages fetch without trouble; this article covers what happens when one doesn’t.

Fetch status

On that screen, each URL shows a status — for example OK, not found, blocked, rate limited, or error. Pages that stay broken also appear under Broken links.

How pages are fetched

  • Most pages are fetched for you, so you can review the change instead of checking the site yourself.
  • PDF URLs use a separate download so the text can be compared. See PDF pages.

When a site blocks access

  • If a site returns 403 Forbidden, rate-limits checks (429), or shows a challenge page, the URL may stay blocked until access improves. On Scale and higher you can try Proxy for that page.
  • Challenge pages (for example “Just a moment…”) may be saved, so you can see that access was blocked rather than missing a change without knowing.
  • If the page is waiting on robots.txt, see Robots.txt before you try Proxy.

Getting help with a blocked page

On Scale and higher, click Managing in the menu and use Try Proxy under Status. Proxy explains slots and what you pay. If Proxy is not on your plan, or the page still will not load, contact us.

  • Proxy Turn on Proxy for a page that isn’t loading, on Scale and higher.
  • Broken links See which monitored pages no longer load, so you can decide which to keep.
  • Monitoring Browse every site and URL you track, check fetch status, and pause pages to free a slot.
  • Robots.txt What happens when robots.txt blocks a page, and how to let checks continue.
  • PDF pages Track changes in PDF files the same way you track webpages.
All help topics FAQ Create account