DARPA's new search engine, Memex, combats human trafficking in Dark Web

CBS News' ''60 Minutes'' reported recently that  Defense Advanced Research Projects (DARPA), Agency, was working on a new search engine that would reveal the hidden secrets of the Dark Net or Deep Web, thewestsidestory.net.

Several researchers had suggested that the dark web hosted a number of illegal activities, including drugs, child pornography, human trafficking and more.

A search engine, Memex [derived from "memory" and "extender")] has been developed to help counter these activities.

Memex is an advanced browser analyses web data to present those search results that are often left out by the top search giants.

Google, Bing or Yahoo cannot display the search results that Memex can and the difference lies in the technology that has been used to develop it. While other search engines looked for popular content, Memex factored in other parameters such as advertisements.

The search engine, currently under development, is being used to search for the human traffickers hiding in the dark web. According to officials the technology could also be used to find missing people, disaster relief and much more.

According to DARPA, Memex would gain more importance as it revealed the part of the web that was still untouched.

''The Internet is much, much bigger than people think,'' DARPA program manager Chris White told ''60 Minutes.'' ''By some estimates Google, Microsoft Bing, and Yahoo only give us access to around 5 per cent of the content on the Web.''

Meanwhile, Computerworld reported that "Dark Web" search engine was in the spotlight this week for its use in combating human-trafficking activities, but it could play a role in business, too.

According to Jeff Schneider, a research professor within the Robotics Institute at Carnegie Mellon's School of Computer Science, there was huge potential for Memex. Earlier this year, Carnegie Mellon was awarded a $3.6 million contract to collaborate on Memex.

Schneider said, the programme right now was focused on human trafficking -- that was chosen as the target domain, but the technology could be applied to other fields. He added, any algorithms used to study human trafficking could easily be retargeted at other domains of people's interest.

The current "surface Web" -- that set of Web pages that got accessed when one  searched  through engines such as Google - is well known. The pages can be accessed as they are connected to other public pages and are widely visible to search engine crawlers, or spiders.

The so-called Dark Web, however, comprised the set of web pages that were not ordinarily accessible to web crawlers as they were too fleeting, for instance.