Coronavirus: The great contact-tracing apps mystery

Germany and Ireland have both trumpeted their success in rolling out contact-tracing apps.

But is there any evidence that they are doing what they are designed to do - warning people they could be infected with the virus?

Not yet - and the privacy-conscious way in which they are designed could mean we will never know how effective they have been.

Last month, the UK government announced that it was abandoning a centralised NHS contact-tracing app for England and switching to a decentralised version, based on the Apple-Google toolkit. This model has been favoured by privacy campaigners because the matching process takes place on users' smartphones rather on a central computer, providing a greater degree of anonymity.

But the government stressed that the new version would not arrive in a hurry, and Boris Johnson told the House of Commons that no country in the world had a functioning contact-tracing app.

The Leader of the Opposition, Labour's Keir Starmer, immediately pointed to Germany, as did many others who disputed the prime minister's claim.

Germany's Corona-Warn-App was rolled out nationwide in June. A few days ago, the Robert Koch Institute provided an update on its progress, celebrating the fact that it had now been installed by around 16 million people.

"A successful start that speaks for great interest and acceptance among the population," a statement from the app's developer said.

Bear in mind, however, that there are 83 million Germans, and it is thought more than half of the population need to have an app before it is truly effective.

"The app works" added the institute's president Prof Lothar Wieler. He went on to say that about 500 app users had tested positive for the virus and "had the opportunity to warn others via the app".

But he then said: "We cannot say exactly how many people were warned, because of the decentralised approach of the app."

In other words, we do not know whether the software is performing its key function.

To know how well your app is performing, it is also vital to understand whether it is producing a lot of false negatives or false positives - in bald terms, whether the software is failing to alert people who have been in close contact with an infected user, and sending warnings to those who haven't.

That's important to know, because studies have indicated that Bluetooth is an unreliable way to determine the distance between two people in some common situations.

The Robert Koch Institute (RKI) told us it too didn't know because, once again, the log history it would need to tap into "remains with the users, encrypted on their smartphones".

We wondered as a workaround whether Germany's manual contact tracers might provide RKI with a record of people who had come to their attention after being alerted by the app.

No, came the answer. Local municipalities run manual contact tracing and do not share this information.

RKI added that it hoped scientific tests would eventually assess the app's impact in the "medium to long term", but had yet to work out how this would be achieved.

Switzerland faces the same issue.

Its health department also pointed the finger at SwissCovid's use of Apple and Google's model.

"We hence have the same limitations in terms of statistics. We don't know - and have no way of finding out - the number of people warned by the app or any false positives/false negatives," a spokesman told the BBC.

No such caution from Cian Ó Maidín, whose firm Nearform is behind Ireland's app.

Covid Tracker app was launched two weeks ago and has rapidly attracted 1.3 million users, about a third of Ireland's smartphone users.

Nearform is also developing a very similar app for Northern Ireland, which decided to go it alone after the NHS project hit trouble. And Mr Ó Maidín suggested that other parts of the UK and major American states could also be customers soon.

"This is a solved problem," he told the BBC.

"We have a solution that works well, it's been tested, it's been verified. We can get a government up and running from the point at which we kick off to launch in a month."

Later he diluted this to "it may not be perfectly solved at this time" but continued to brush off concerns about effectiveness.

"We don't have a full data set yet but the early data looks really positive," he said, directing us to Ireland's health service HSE for a more detailed response.

We have actually been chasing them on this point for days, and will provide an update if we hear back.

As the Irish app is based on the same model as Germany and Switzerland's, it is not clear why it should generate more data.

Although there is one potential caveat: during the installation process, users are asked to agree to the collection of "anonymous metrics" about the "effectiveness of contact-tracing processes".

In any case, Mr Ó Maidín says we should not be overly concerned about missed or bad matches.

"The perfect should not be the enemy of the good," he says.

The problem is that with so little data exposed, can we even distinguish good from bad?

There have also been problems with contact-tracing apps in other countries: