How to keep bad robots, spiders and web crawlers away
Submitted by admin on Sat, 2010-11-13 10:57
Many so called webbots or web spiders are currently used for many different things on the Internet. Examples include search engines that use them to catalog the Internet, email marketing people that search for email addresses and many more. For a description of such robots check out The Web Robots Faq.
Table of contents
* Trapped and/or Banned Robots
* Definition of a bad robot
* How to identify bad robots
* Banning bad robots from the site
* The Robot Trap
* Download the traps
Visit the Website : ->:
Web Development: