Prevent the Internet Archive Wayback Machine from archiving your website

A coworker asked me of a website that tracks different versions of websites. I forgot what it was (gasp) but my boss remembered that it’s the Internet Archive Wayback Machine.

I checked All Narfed Up to see if it was tracked, only to find out I blocked it eons ago. (See?)

If you don’t want its spiders to crawl and cache your website, create a robots.txt file and make sure the following is inside it:

User-agent: ia_archiver
Disallow: /

Cheers!

Published by

Bryan Villarin

Bryan is a Community Guardian at Automattic. He's also a photographer, card magician, and cat whisperer. (Thanks to my friend and colleague Steve Blythe for the sweet photo!)

Talk to me, Goose

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s