 Explanation
of "Spiders" ("Robots")
(Also
see bookmarks)
What to do
--
Remove
all bookmarks that you have from search results for
property records, transfers, site search and tax
estimator.
Please
don't bookmark search results in the future.
Go
to http://www.medinacountyauditor.org/index.html to start your search
anew.
Search Engine Spiders
-- The search engines (Yahoo! Google & others) search
web sites for links to their other pages. For the main page and
the other pages, the spider then searches for keywords. This is
so that you can search the internet for places that might contain
the item for which you're looking.
The search engines will check for our file,
robots.txt, to discover what pages they have permission to
access.
Save Page Spiders -- Sometimes
the search engine spiders and other Internet Service Providers or
internet sites store the pages that they encounter at other web
sites in memory, or cache. This makes it faster for you to view
the contents of the web site.
Bookmark Spiders -- There
are many wonderful service providers offering many services to
their customers. When one bookmarks a page, a spider or robot is
sent out to retrieve that page and save it in "cache"
for quick retrieval for users.
For
our static web pages (those that end in "htm" or
"html"), we welcome spiders.
Nevertheless, for our search engines and programs this
can cause our computer to run overtime just for this retrieval.
We have had 10-15 "hits" by bookmark spiders at a time
on our search engines, and in April our Internet Service Provider
almost shutdown our "property search engine." (They did
shut us down temporarily!)
Some Providers have not been checking our
robots.txt file to discover what pages they have permission to
access.
An example might help:
A user searches for a property owned by
"Yankatovich," then bookmarks the result. Eveyday
thereafter, a bookmark spider hits our site, runs our programs to
find search results just for "Yankatovich."
This request is too specialized for us to honor. As
well, we have had 200-300 hits a day on our search engines in the
past which makes it more difficult to service "real
time" users.
|
