The Robot-Analyzer Professional Edition software (RA) monitors and analyzes search engine responses to your URL submissions by tracking ongoing search engine spider activities. In plain words, you tell the software where your log files are, on the web or your hard drive, and RA will process the log file(/spacer.gif) looking for all spider activity.

When completed, RA presents you with a list of spiders, the pages they visited and when, all in tree form. Want to see them by date or by pages visited? No problem, those options are also available. You can even export the spider visits, the spider logs and the spider IP addresses (for use in cloaking).

Here are some of the questions it can answer for you:

(a) Has my site been spidered?

(b) If so, who was it?

(c) When did the spiders visit?

(d) Which directories and pages did they index?

(e) Can I direct spiders towards content which is specific to their IP address?

(f) Are spiders indexing any inappropriate content?

(g) Are they getting everything they want or are they receiving error messages?

(h) I'm paying for an accelerated listing, was I spidered within the time agreed?

(i) If I'm paying for frequent re-indexing, is it happening?

Once a site has been indexed and is visible on the search engine listings, you need the following information:

(a) How frequently am I being re-spidered?

(b) By who?

(c) Which individual pages are constantly being re-spidered and are, therefore, critical to my search engine ranking?

(d) My site has changed – are the spiders still indexing my most critical pages?

(e) If not, why not?

(f) New spiders are constantly arriving – I need to add them to my spider list, set access rules and monitor their ongoing behavior.

Did you see some spiders whose behavior you didn't like? Just add them quickly to the main list of spiders included with RA. Then create a "Robots.txt" file in an easy 3-step process. When done, upload the file to your web site.

This easy-to-use software is the next must-have of every webmaster.

I wouldn't be without it now that I've seen who has been visiting my sites!

–Rhoda Schueller, Search Engine Marketing Instructor for the Academy of Web Specialists