Despite the fact that it felt as though I was reading an advertisement for BrightPlanet, I found the statistics in this article fascinating. I had no idea that the deep web is so much larger than the surface web (400-550 times larger). For instance, just 60 sites are larger than the surface web by 40 times.
I think I remember reading in Scholarship in the Digital Age that most of the deep web is made up of databases, some of which are proprietary and not accessible.
I tried looking into using BrightPlanet, a "direct query engine", but you must get a subscription to use it, unlike traditional search engines which are free on the net. I wonder if we will ever see a day when BrightPlanet, or something like it, becomes free to use?
Hello Erin,
ReplyDeleteI'm glad to see I wasn't the only one feeling as though I was reading an advertisement for BrightPlanet, and I do have to agree that knowing how deep the web truly is is mind boggling.
I also remember reading something about the deep web consisting of databases, but wasn't there also a connection to other restricted access locations (such as an intra/extranet) lumped into it as well?
As for your thought regarding the direct query engine becoming a free resource, I have another question to offer: would a direct query engine searching the deep web be as effective as we would like it to be, as we have already concluded that a good portion of the deep web is proprietary in nature?
Just tossing a thought out there.
--Anthony