A coworker asked me of a website that tracks different versions of websites. I forgot what it was (gasp) but my boss remembered that it’s the Internet Archive Wayback Machine.
I checked All Narfed Up to see if it was tracked, only to find out I blocked it eons ago. (See?)
If you don’t want its spiders to crawl and cache your website, create a robots.txt file and make sure the following is inside it:
User-agent: ia_archiver
Disallow: /
Cheers!
