The Deep Web: Surfacing Hidden Value

Despite the fact that it felt as though I was reading an advertisement for BrightPlanet, I found the statistics in this article fascinating. I had no idea that the deep web is so much larger than the surface web (400-550 times larger). For instance, just 60 sites are larger than the surface web by 40 times.

I think I remember reading in Scholarship in the Digital Age that most of the deep web is made up of databases, some of which are proprietary and not accessible.

I tried looking into using BrightPlanet, a "direct query engine", but you must get a subscription to use it, unlike traditional search engines which are free on the net. I wonder if we will ever see a day when BrightPlanet, or something like it, becomes free to use?