Wikipedia plans search engine to rival Google, Bing

Wikipedia may soon have its own search engine to rival the likes of Alphabet's Google and Microsoft's Bing. Its parent company, Wikimedia Foundation, was granted a sum of $250,000 late last year, an announcement which was only made available to the public this month.

Dubbed Knowledge Engine by Wikipedia, the San Francisco-based non-profit organisation is building a search engine to provide "a system for discovering reliable and trustworthy public information on the internet".

But for "the Internet's first transparent search engine" to come into being, it will have to overcome through some roadblocks.

The Wikimedia Foundation was awarded $250,000 in November last year from the John S and James L Knight Foundation. The search engine will allow users to discover material located on Wikipedia and its sister websites.

The Wikimedia Foundation insists that the Knowledge Engine will be open and transparent about how a piece of information originates and allow access to metadata. The Wikipedia's search engine will also protect user privacy, stay away from advertisements, and give emphasis to community building and sharing of information.

"Today, commercial search engines dominate search-engine use of the internet, and they're employing proprietary technologies to consolidate channels of access to the internet's knowledge and information," Wikimedia Foundation said. "Knowledge Engine by Wikipedia will democratise the discovery of media, news and information - it will make the internet's most relevant information more accessible and openly curated, and it will create an open data engine that's completely free of commercial interests."

While that sounds good, the issue with the search engine is the level of secrecy Wikimedia Foundation has surrounded it with. Volunteers who work hours on the project on a daily basis weren't aware of the issue until the document was made available to the public. The issue is seeing the Wikipedia community and Wikimedia Foundation drift apart from each other.

"There's been increasing alienation of the community from the foundation," William Beutler, a long-time Wikipedia editor, journalist for The Wikipedian blog, told Motherboard. "The community is this volunteer group that is made up of people who largely buy into Wikipedia for ideological reasons. Then you have the foundation, which has increasingly fewer people from the community and a larger Silicon Valley contingent that comes from a tech background.

''It seems like there's been a culture clash," he said. "And this is the most destructive manifestation of that culture clash."

For a media institution that puts the transparency over everything, the way it has handled information about its upcoming search engine shows the other side of the Wikimedia Foundation.

There's also no word on when we can expect the search engine to be ready.

'Out of the loop'
Wikimedia Foundation submitted a budget included in the grant announcement providing a total of $2,445,873, which is divided among 14 staff, hardware, and associated costs that include medical expenses and travel.

The comprehensive budget has eight engineers including programmers, two data analysts and four team leaders including a director and a vice president.

The biggest danger is that Google or Yahoo ''could suddenly devote resources to a similar project, which could reduce the success of the project'', mentioned Wikimedia documents submitted with the grant application.

Volunteers are surprised at Wikipedia's second attempt at a search engine

In the past too, Wikipedia has tried building a search engine. However, in 2009, the founder Jimmy Wales abandoned the Wiki Search project, saying it had ''not been enjoying the kind of success that we had hoped''.

At that time, he had said that he was deeply concerned about search.

''I will return again and again in my career to search, either as an investor, a contributor, a donor, or a cheerleader,'' he wrote in a blog post.

In direct contrast to the traditional search engine approach of matching key words, subjects and the reputation of web pages to search the most relevant result, Wikia Search had displayed short articles written by people on topics.

The information of the grant came as a surprise to volunteers within the Wikipedia community said Andreas Kolbe, who has written a story on the Knowledge Engine for the Wikipedia Signpost, a weekly online newspaper.

''Its gung-ho 'We're building a search engine!' content is a bit of a bombshell for the volunteer community,'' he told The Register.

''They were led to believe it was just about getting a central search function to find stuff spread out across the various Wikimedia sites, with OpenStreetMap thrown in perhaps.

''Volunteers feel WMF management has purposely kept them out of the loop.''