The IT behemoth that you might never have heard of

It’s a giant in online search, has also built smart speakers and a voice-activated personal assistant, and is testing a driverless car. But it’s not based in Silicon Valley – it’s Russian.

Earlier this year, I climbed into a self-driving car for the first time. I was one of hundreds who had waited patiently for their turn to take a spin. Some even got sunburnt while waiting, but felt the pain was worth it. In the end each of us got a pin saying “The first self-driving car’s passenger”.

The stunt took place during a conference held by one of the biggest internet companies in the world. It now works in such fields as artificial intelligence, self-driving cars, personal assistants, online-translating and dictionaries, online marketplaces and – most recently – smart speakers. But it wasn't Google. And it wasn’t even in Silicon Valley.

There are several web search engines in the world. The biggest are Google, Microsoft’s Bing, China's Baidu and the early pioneer Yahoo. Google is the most dominant, having become much more than simply the go-to search engine. But there’s another contender to add to this list that few people have heard of outside the country where it operates. That country is Russia. The search engine? Yandex.

You might also like:

Much like its Silicon Valley counterpart, the ‘Russian Google’ is more than just a search engine. It began life in the late 1990s as the Yandex.ru homepage. It has since developed into a sprawling IT giant.

“In the West they like to call us Google of Russia, but we have long been far more than that: we are Uber of Russia and Spotify of Russia and many more other things of Russia," the founder and the head of Yandex Arkadiy Volozh said in May 2017.

A USB flash drive containing 4Gb of memory costs just a few dollars in 2018. When Yandex was founded, a hard drive with that much memory could cost you thousands of dollars. That’s about how much memory Volozh had to purchase to set up the Yandex search engine in 1997.

At first it was just a project that the telecommunications company Comptek Volozh had founded almost a decade earlier. How then did it evolve into an IT giant that raised $1.3bn (£1bn) in at its IPO in 2011 – not far off Google’s own IPO of $1.7bn (£1.31bn)?

Yandex may be a mostly Russian enterprise, but it is starting to attract big players who have made their name outside Russia. There are former Google and Microsoft employees managing research departments. The company clearly considers Google to be one of its main rivals. But that may only be true when it comes to tech and ideas. Olga Maslikhova, a venture investor in IT, says Yandex has chosen not to try and rival Google on the world stage – it has decided to concentrate its resources on Russia.

“Yandex is a public company and it needs to boost its valuation. The choice is to scale to other regions or to develop new product lines in one exact region,” says Maslikhova. “Bearing in mind the Russian economic situation, it is complicated to expand internationally. So Yandex chooses the strategy of horizontal scaling to where they already operate.”

The talent is not necessarily Russian, however. David Talbot, a former Google languages researcher, is now the head of the Yandex.Translate service. Talbot joined a team led by Mikhail Bilenko, who had previously developed a platform for machine learning in Microsoft and now is the head of machine intelligence and research at Yandex.

There was some controversy when Talbot joined and Yandex’s online translator started using a neuronet system soon after Google Translate did. But Talbot told the BBC the architecture was different, which meant it could better deal with the particular task of translating English into Russian. 

“Most people are making what they do open source,” he said. “There’s no open source for exact implementation of the Google machine translation but many toolkits are available open source which allow you to experiment with these things. People have seen the benefits of not keeping their technology completely private. If you make it open source then many people will experiment with your software and they will find bugs and improvements.”

In mid-2017, the company opened its own machine-learning library. It has also just launched its cloud service aimed at helping companies to develop its products. This may end up including its own version of speech recognition technology.

Yandex is also at the forefront of bringing self-driving cars to real roads. For example, its self-autonomous taxi was launched as a test project in the suburbs of Kazan, capital of Russia’s Tatarstan region. This small town called Innopolis is known for its IT-cluster, and now its inhabitants can drive around hands-free. It’s just like Google’s Waymo, except it’s on the Russian steppes and not suburban Phoenix.

Yandex attracts many of the brightest young minds from the best of Russia's technical high schools, but as Maslikhova noted, while they may be hubs of promising tech specialists, Russia’s universities are not particularly business-minded. Building up the next generation of 'Yandexoids' (as Yandex employees call themselves) is also one of the company's aims.

Yandex operates its own lyceum to teach children basic programming basics. The Yandex School of Data Analysis has been offering free courses for senior students and postgraduates since 2007, and there are also joint programmes with selected Russian universities.

Yandex not only tries to teach young developers and analysts to deal with big data and AI, it also tries to get the rest of the population used to it as well. Yandex's Toloka platform gives everyone an opportunity to help the company teach its AI algorithms while earning some money. There are hundreds of tasks you can do on its website, each will earn you around 5c (4p) depending on your level of expertise. You only need working eyes, ears and a brain for some of the tasks – no high-school diploma is required to tell a machine what you hear on a recording (this is, after all, how you teach an algorithm to recognise human speech).

Yandex’s public engagement doesn't end there. This summer the company launched a mobile app for its project called Narodnaya Karta (“A public map” in English), which is essentially a Yandex.Maps editor (Yandex has a map app much like Google Maps). Narodnaya Karta is a tool in Yandex's Maps service that helps its users improve it. Everyone can suggest editing the current version of a map. Everyone can scan the world around them and upload new data so the map adapts. It is crowdsourcing cartography.

There may be even more ambitious projects ahead. Volozh proposed something ambitious this year at Yandex's ‘Yet Another Conference’ – one of the biggest tech conferences in Russia. It was the idea of making technologies dissolve around us.

How? By making it very easy to use and extremely natural. Want some pizza? No need to search for a pizzeria’s phone number, call it and place an order. Just push the button on your smartphone and ask your personal smart assistant – Alisa, Yandex’s equivalent to Apple’s Siri or Amazon’s Alexa – to order it for you. Papa John’s confirmed it has collaborated with Yandex to make this possible. A new platform called Yandex.Dialogues launched in May 2018; it’s essentially a platform that allows third-party developers to add ‘skills’ – particular functions – to Alisa.  

While seriously developing AI and helping international research groups to carry out scientific studies, the company still has time to subvert the stereotype of the stern-faced Russian. They’ve allowed Yandex.Maps’ users to label places with emoji (which turned into mass marking of almost all districts of Moscow with ‘poo’ emoji). Or when they launched Yandex.Autopoet – an algorithm that ‘writes’ poems imitating styles of famous poets. They’ve introduced translation to emoji and added the elvish language from Lord of the Rings to the Yandex.Translator a while ago.

Yandex’s AI can even write music: at Yet Another Conference in 2017 an orchestra performed an overture in the style of Alexander Scriabin written by software to celebrate the anniversary of the composer’s birth. The company also supported an online historical project shaped as social media called “1917 – Free History”. The project marked 100 years since the Russian revolution and aimed to reveal what was happening in Russian cities while the revolution took place.

Yandex is clearly aiming for the top – even if that means asking climbers to take a picture every step of the way to the top of the highest point on the planet, and make 360° panoramic photos on the Everest summit for Yandex.Maps. “We definitely need to hire a Head of Crazy Projects urgently now,” Volozh said recently. He's probably only half joking.

Yandex has gone way beyond search; it now operates in the arena of machine translation, it can sell and deliver stuff to you, it can serve you food, call you a cab, be your home cinema (thanks to its online film database Kinopoisk), help you educate yourself and catch your train. It is starting to involve itself into almost every aspect of life.

Many of us already have the feeling that our smartphones know us better than our loved ones. With so much involvement in the online activity of Russians, Yandex has access to huge amounts of data about different kinds of users’ behaviour. This is then ‘fed’ to the company's AI, allowing it to develop further and further.

Is Yandex going to become a monopoly? Iskander Giniyatullin, a partner at the tech venture fund Sistema VC, doesn’t think so. “Yandex merging with Uber Taxi can be seen as an outcome of these two companies’ competition. But what Yandex does is just getting into new areas that are now recognised as extremely promising worldwide,” he says.

Alexander Larianovsky, who spent seven years in Yandex as the head of international development, thinks the same: “We observe roughly the same story for every company anywhere else... It is a matter of either developing your business or cutting it, that’s it.”

In the past few months, Yandex has launched its taxi service in Lithuania. Earlier this year, they rolled out the same service in Latvia and Estonia. It doesn’t look unusual that the Russian IT giant works in countries that used to be part of the Soviet Union. But it is more surprising to see the level of its activities in its historically prickly neighbour Turkey.

This is seen as quite a logical move by Larianovsky though. He worked on launching yandex.com.tr in 2011. “It all starts with the school where we were taught that the young Soviet republic helped the young Turkish republic. We see each other not as enemies but as friends, historically.”

Yandex decided to enter the Turkish market when Turkey used to be more pro-Western than now, Larianovsky says. But it didn’t mean that the Russian IT company wanted to use it as a gateway to the West, he says. “No, it’s just been a first attempt to go somewhere non-Cyrillic.”

Volozh says his company won’t even try to compete with other search engines outside of Russia until competition laws change. But at the same time, it doesn’t look like it’s going to stop trying new things.

--

Join 900,000+ Future fans by liking us on Facebook, or follow us on Twitter or Instagram.

If you liked this story, sign up for the weekly bbc.com features newsletter, called “If You Only Read 6 Things This Week”. A handpicked selection of stories from BBC Future, Culture, Capital, and Travel, delivered to your inbox every Friday.