CovidCard proactive release is out from DIA (h/t @marcdaalder): https://www.dia.govt.nz/diawebsite.nsf/Files/Proactive-releases/$file/Proactive-Release-Independent-CovidCard-Reports-FINAL.pdf

Still working my way through it, but so far... it looks like a lot more trials and data were needed to make the claims with confidence. There was still quite a lot of room for uncertainty.

Worth noting that the docs released today are from 13 May, 20 May, 5 June, 26 June, 24 July. So are a bit historical in that a lot has changed since then. We know more about the virus, other technologies have progressed further, and relationships between the groups have changed.

As such, the earlier documents contain assumptions and statements that were probably reasonable at the time, but with new information and developments are probably not true anymore. Goes to show how hard it can be do this stuff in the context of a fast-changing pandemic.

First document: sets out the original case for having technology help with symptom tracking and contact tracing. There isn't much new in here based on what we know now, although there is one interesting quote:

"The authors of this document recommend avoiding a haphazard or hurried approach to deploying digital technologies for contact tracing and apps in particular."

Which is arguably what govt has done. But others would say not fast enough...

Second document: University of Otago runs a trial in Nelson Hospital across six days, with 42 participants wearing them while at work. Significantly more close contacts picked up than what was self-reported in a written diary. Most participants say they'd be happy to wear a card.

Challenges include a lack of algorithm/thresholds for separating contacts from close contacts, difficulty establishing ground truth in the trial, and the ad hoc nature of the trial given it was organised quickly with decisions made in the field.

Images from this part of the report basically confirm that the card used in the trial is a Minew C10 card beacon: https://www.minew.com/products/c10-card-beacon.html

The same company now also has a "social distancing wristband" https://www.minew.com/product/B-Series/series.html

Report highlights challenges with firmware, which in some cases caused the cards to go into sleep mode, or to reset and lose data, and needed new firmware twice during the trial. This is what trials are for, but highlights how hard it would be to fix these issues after deployment

Overall, I think most of the recommendations and discussion from the report are about how to run a better trial in the future, rather than there being significant issues with the cards beyond improving the software and firmware. Which makes sense, the cards logically work.

Document 2b: a report from a company in Sydney called VT42, which I think is now Contact Harald: https://contactharald.com/about-us 

They provided the cards for the first trial, and the report details the technical issues experienced with the cards in the trial.

Third document is the key CovidCard proposal document dated 5 June, on which most of the media coverage at the time was based. As I've said before, there's some good bits in there but also some wrong assumptions that create an unfair case against other technology choices.

This is also the document that suggests the card would have to be quasi-mandatory and worn around the neck. Interestingly, parts of the doc about costs have been redacted, even though they have already been publicly released by journalists who were leaked the report.

Fourth document: CovidCard Validation Report dated 26 June, largely based on the second set of trials in the Waikato. These covered office, cafe, construction site, taxi, and house party scenarios, ranging from 3 to 21 cards in each scenario.

I think this is where some of the science of how the trials were run got separated from how it was covered and reported in the media. These are good trials to run, but they are very limited and indicative only, and the researchers themselves say larger open trials are needed.

This is quite different to how the Card was presented as a silver bullet that would solve our contact tracing challenges and that it was the "obvious" solution. Talk about how it would be useful in crowded scenarios like stadiums or public transport were not tested or proven.

In the scenarios where it was tested, the card had pretty good results in clean environments, but performed less well in difficult environments like cafes and construction sites. In my opinion (with limited IoT experience), the cards worked about as well as could be expected.

Some other people were worried that this was all vaporware and the country was being scammed. I did not share this concern because the cards do logically make sense and you'd expect them to work in standard environments. But they struggled in difficult environments.

What would make an environment difficult? Having a lot of cards in a relatively small space, or there being a lot of things that interfere with bluetooth signals (e.g. dust, water, metals). Again, the fact that the card performs less well in these environments is to be expected.

But I didn't see any media coverage or interview where this was explained. I saw claims that CovidCard had significantly lower error rates than Google/Apple, yet in difficult environments CovidCard probably had similar error rates to G/A ENF.

My main response to the data presented in the report is "more trials needed". At a larger scale with more devices in a range of difficult environments. Results were promising enough to justify further scientific study. But not enough to justify mass deployment to the population.

Would the CovidCard have been more accurate than phone-based Bluetooth (e.g. Google/Apple Exposure Notification Framework)? In some cases possibly, in other cases probably about the same - they haven't been compared side-by-side, so any comparison at this stage is limited.

The document then contains an interesting comparison of the effects on signal strength depending on where you wear the card, and concludes that around the neck on a lanyard is optimal. Pretty much anywhere else on the body or in fabric = reduced signal strength, more noise.

Fifth Document: the Defence Technology Agency (DTA) does a review of the CovidCard.

"In general, we were impressed with the quality of the effort put into the development of CovidCard, and in particular the attention paid to security and privacy considerations."

DTA assess the memory, battery capacity, and robustness to meet the targeted design lifetime of "at least one year". They think that it would be useful for contact tracing, but that the card would generate "an unknown and possibly significant" number of false positive contacts.

DTA also note the need for large-scale trials. They recommend that the card be run for more than 21 days to properly evaluate robustness and battery life, and that performance be evaluated in "alternative positions" such as in a shirt pocket as it may enhance "user acceptance".

DTA notes it is likely that there will be defective batteries in a mass rollout, which should be prepared for.

A fun little recommendation - in the cards tested, a red LED is on to indicate a good battery state. DTA recommended they switch it to green to "avoid user confusion".

DTA did some pretty robust robustness testing on the card:

"The impact test in section 5.2.4 of the Hardware Specification was carried out by dropping a 9.9 kg weight from a height of 1 m onto a 15 mm diameter bolt placed on top of the card. The card was deformed and no longer functioned."

DTA notes that a several groups of people wouldn't want to wear a lanyard, including machinery operators, surgeons and other health workers, and fitness instructors.

[It's also not entirely clear if the card is unidirectional, and whether one orientation is more effective.]

DTA concludes that the security side of things is probably fine, but that it should be checked by a professional security consultant too. It's noted that the central Card Registry Service is probably the most valuable target for attack, rather than any individual card.

Okay, so overall, what does this tell us?
- The cards are real and they probably work
- The trials were okay, but could have been better and were far from conclusive/decisive
- Do they work better than smartphones? In some cases, but further testing is needed.

When you put aside compulsion (because you could do that with apps too) and equity (because you could give out phones too), the key supposed benefit of the card is better accuracy than Google/Apple Exposure Notification on phones. And I don't think the evidence here proves that.

The data gives me a bit more comfort around the power budget. But it doesn't help me feel better about the locked-in hardware and firmware that would be almost impossible to update once deployed, or the potential missed opportunity around international interoperability. Or cost.

Is there a role for wearables to play in digital contact tracing? Yes, there could be. We might still get wearables in NZ in the near future.

Was it going to be this proposal? It would have been possible, but it would have been a lot of hard work to get there. /FIN

Disclaimers: I've tried my best to read the docs neutrally and pull out the relevant facts. I'm a researcher at the University of Auckland, and have no formal relationship with any digital contact tracing system/approach. I have had some chats with the Ministry of Health, unpaid.