Prevent the Internet Archive Wayback Machine from archiving your website

A coworker asked me of a website that tracks different versions of websites. I forgot what it was (gasp) but my boss remembered that it’s the Internet Archive Wayback Machine.

I checked All Narfed Up to see if it was tracked, only to find out I blocked it eons ago. (See?)

If you don’t want its spiders to crawl and cache your website, create a robots.txt file and make sure the following is inside it:

User-agent: ia_archiver
Disallow: /


Published by

Bryan Villarin

Bryan is a Trust & Safety Wrangler at Automattic. He's also a photographer, card magician, and cat whisperer. (Thanks to my friend and colleague Steve Blythe for the sweet photo!)

Talk to me, Goose

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s