Our Favorite Spider Emulators for SEO (2015)
Q: In your book you recommended using a spider emulator to see what a webpage looks like to search engine robots. What tool do you use when you’re doing this?
A: These days, we use three different methods for reviewing a site with the “spiders-eye view.” Here they are in order of simplest to most complicated:
* Rex Swains HTTP viewer at http://www.rexswain.com/httpview.html. Type in any URL and see what a typical client (browser or spider will see). This is a great tool for a quick and easy look at a page.