Prevent the Internet Archive Wayback Machine from archiving your website

A coworker asked me of a website that tracks different versions of websites. I forgot what it was (gasp) but my boss remembered that it’s the Internet Archive Wayback Machine.

I checked All Narfed Up to see if it was tracked, only to find out I blocked it eons ago. (See?)

If you don’t want its spiders to crawl and cache your website, create a robots.txt file and make sure the following is inside it:

User-agent: ia_archiver
Disallow: /


Published by

Bryan Villarin

Bryan works at Automattic. He's also a cat whisperer. Sometimes…a photographer, and card magician.

Talk to me, Goose

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s