Effectiveness

Introduction

There is a large body of evidence on how previous epidemics were combated - with a combination of painstaking disciplined manual contact tracing and aggregated location and movement maps.

However there is little evidence on effective measures addressing a pandemic at this scale as it is truly unprecedented in modern times. The previous pandemic at this scale, the Spanish Flu, was a hundred years ago in totally different circumstances.

Countries which are ahead of the curve, having gone through lockdown and seemingly successfully contained the spread of the virus can provide evidence. But there is no evidence on the sustainability of their measures and of the long-term success. And regarding the use of technology more specifically, it has always been part of a combination of measures.

Hence it is hard to know what the real impact (if any) was of using technology. Even in the Singapore case, which is often cited as an inspiration, the penetration was very low and the technology was explicitly linked to other measures.

We do have a fallback however, with evidence at a smaller scale which we can prudently try and extrapolate into hypotheses on how it could work at a bigger scale.

The challenge is how to adapt those recipes to the scale of this pandemic. Multiple patterns have emerged:

• For transmission tracing:
  • Human-powered contact tracing through contact tracers calling infected individuals, collecting information from their memory and following up on this information. We call this “Human Contact Tracing” or HCT.
  • Emulating the process through technology, e.g. by building a graph through ‘network effects’ of apps. We call this “Autonomous Contact Tracing” or ACT.
  • Assisting contact tracers with technology, e.g. by providing them with more information sources and tools
  • Assisting the citizen in helping the contact tracer (manual or automated)
  • Or any combination of the above. This combination could be a co-existing scheme (where the models are not coordinated or linked together, so there could be an overlap unknown to the participants) or a linked scheme (where information from both models are combined).
    • For informing policy making through mapping:
      • Correlating siloed sources of information, like maps of residence of infected individuals and movement information from cell phones or train and plane tickets
      • Aggregating health status information and movement information collected together through an app

Terminology

We need to clarify some terminology before proceeding:

• Transmission routes (see https://github.com/BDI-pathogens/covid-19_instant_tracing/blob/master/Manuscript%20-%20Modelling%20instantaneous%20digital%20contact%20tracing.pdf: transmission routes aligned to their implications for prevention):
  • Person-to-person: “transmission from one person to the other when the two persons are simultaneously present.” This happens through respiratory droplets and contact routes or fomites (infected objects) in the direct neighborhood of the infected person.
  • This transmission can be
    • Symptomatic: “direct transmission from a symptomatic individual, through a contact that can be readily recalled by the recipient”
    • Pre-symptomatic: “direct transmission from an individual that occurs before the source individual experiences noticeable symptoms. (Note that this definition may be context specific, for example based on whether it is the source or the recipient who is asked whether the symptoms were noticeable.)”
    • Asymptomatic: “direct transmission from individuals who never experience noticeable symptoms. This can only be established by follow-up, as single time-point observation cannot fully distinguish asymptomatic from pre-symptomatic individuals.”
  • Environmental: “transmission via contamination, and specifically in a way that would not typically be attributable to contact with the source in a contact survey (i.e. we exclude from this transmission pairs who were in extended close contact, but for whom in reality the infectious dose passed via the environment instead of more directly). These could be identified in an analysis of spatial movements.” We define this broadly as infection occurring without direct conscious contact between the infected person and the affected person (either through a fomite, through respiratory droplets carried further due to bad circumstances like deficient ventilation or high proximity contexts like a mass gathering, …) or as a proxy for unidentified infected persons in a place where such identification is not possible or non-users in the context of a software-based solution.

We acknowledge that boundaries between these categories may be blurred, but these broadly have different implications for prevention.

Effectiveness requirements

Following characteristics contribute to the effectiveness of the solution’s contribution to an exit strategy:

Accuracy

Accuracy can be defined as the number of correct predictions over the total number of predictions. Besides True Positives and True Negatives, predictions also include False Positives and False Negatives. The definition of Positives and Negatives is of course dependent on what the solution is attempting to predict. This might have multiple tiers, with one feature of the solution attempting to detect a contact between two individuals while another feature attempts to predict the transmission of the disease between those individuals in this contact.

• Accuracy is also determined by the timeliness of the information. As time passes, information can be either less or more useful for predictions. Also, less precise information (like non-confirmed symptoms) earlier can be used for valuable indicators.
• Completeness of information: the more contextual information available, the higher the accuracy. This might include whether a contact was inside or outside, its duration, whether the individuals were wearing face masks, the age of the fomite, …
• Inclusion of (para-)medical data (* see note) as required by the purpose of the solution. This might be:
  • Subjective information (reported by the user, like health log, pre-anamnesis, ...)
  • Objective information (reported by a medical doctor, like diagnosis, medical protocols, test results, …)

Speed

The speed of the process supported by the solution conditions its effectiveness. With pre-symptomatic and asymptomatic transmission at play, we are by definition already late. So any more delay increases the likeliness of further spread. Speed has multiple components:

• Speed of detection: time elapsed between a presumption of infection and the confirmation. This includes the time for gathering the information and connecting the dots, for making contact with the individual, but also the time for planning a test, executing the test, obtaining the confirmed results.
• Speed of detection for mapping the epidemic: time elapsed between a prediction of the development and spread of hotspots and its actual occurrence. (working with leading indicators like symptoms before MD consultation, location of contacts of index cases, … contribute to the shortening of this cycle)
• Speed of isolation: time elapsed between the presumption or the confirmation of an infection and the moment the individual is effectively isolated. This can be affected by preventative isolation. The ability to take preventive measures is conditioned by the speed of detection.
• Feedback loop length: at a higher level, the effectiveness of the exit strategy will be conditioned by the delay between measures taken by the authorities and the monitoring of its impact.

Adaptability

As the knowledge about the virus and its spread evolves, the solution needs to be adapted based on new insights. This could pertain to:

• Type of data being gathered
• Algorithms for data interpretation
• Aggregation levels

Insights in transmission routes

• in terms of transmission routes covered by the solution (cfr above: person-to-person, environmental, presymptomatic, asymptomatic)
• The ability to identify and understand the networks of transmission
• The ability to feed data into other parts of the process (eg non-confirmed symptoms into mapping)

Support of isolation and quarantining

encompassing the ability to exchange reliable and certified information with any measures that can encourage people to self-quarantine (social security, PPE provisioning, premises allocation, …)

Targeted measures

the ability to inspire targeted measures through granularity of the insights gained from the collected data, as measured by the granularity of the decisions that can be taken based on the solution

Efficient use of resources

the extent of support for avoiding overburdening the healthcare system and yield the maximal impact of available resources

Interoperability

solutions not only need to fill the requirements of the domain they belong to, they also need to interoperate in order to create a global optimum for the system as a whole.

Probability of side-effects

Probability of side-effects caused or aggravated by the solution. With infectious diseases in general and diseases leading to probable social isolation risks of stigmatization and discrimination increase. The solution might have an influence on this phenomenon.

Coverage

• Required adoption rate: the required rate of adoption for the solution to be effective. There is some debate on this aspect, as the assumptions behind and the expected impact of the number that has been circulating -60%- are not clearly understood and debate subsists on whether lower levels of adoption might have impact. (cfr [1]
• Expected adoption rate: the rate of adoption that can reasonably be expected. Factors influencing this:
  • The perception by the population of the urgency to adopt the technology and of the safety of using it.
  • The adoption of underlying technology supporting the solution, conditioned by the nature of that technology (hardware, operating system version, app downloads) and its availability. This availability might be limited physically (like the ownership of certain hardware) but also by decision (like Apple limiting background use of Bluetooth beacons).
  • The ease of sufficient use of the technology, which includes not only the user experience as such but also the inherent technical limitations, like battery capacity compared with energy consumption.
  • The scalability of the solution - the ability to serve the required number of users in the given time period.
  • Notwithstanding the level of the required adoption rate, it is important to note that the reliance on technology introduces a bias in the population covered by the solution. Technology is not evenly spread across various demographics and might even be less spread among the most vulnerable parts of the targeted population.
  • The adoption rate is also dependent on the availability of the apps in the distribution channels, i.e. the 2 dominating app stores as stated earlier. Apple and Google have released a joint Bluetooth protocol and mobile framework for Exposure Notification (read Automated Contact Tracing). One of the main reasons for this joint effort is to overcome the iOS limitations of using BLE in background apps .Now as the authors of that protocol they want to maintain control of its usage and will restrict any use outside of the parameters and context that they have stipulated (e.g. it cannot be combined with GPS location data). Combine that with their power over which apps are allowed on their app stores, there is a risk that this will hinder adoption or prevent the distribution outright
• Interoperability:
  • the ability to provide the features even when the involved individuals use different solutions. In contact tracing this might imply that the participants in the contact graph use different solutions.

Technical feasibility and time-to-market

• the feasibility of the proposed solution being developed in time might depend on prior similar solutions, on the dependence on break-through technology, on the availability of the expertise required.
• Speed of distribution: The digital solutions under consideration consist at the end user side of a mobile application. To get this app onto the user’s phone the world is reliant on the 2 dominating app distribution eco-systems, the App Store from Apple and the Play Store from Google. To be allowed to publish apps on those app stores, they need to go through a vetting phase (review) by these 2 tech giants. This process can take anywhere from a few days to a few weeks, depending on the functionality, complexity and other factors (like limitation of number of reviewers).

Technical dependability of the solution

• the degree to which the solution is expected to be available when needed at the required quality of service. This includes among others the availability of the components and their connectivity, the integrity of the information, the performance, ...

Evidence

about the effectiveness of the solution or its parts, based on prior experience

(*) Note on the medical information: In these, the accuracy of the medical data and of the link to the patient is crucial. This introduces some tension in the privacy requirements. Privacy can be improved through anonymization or at least pseudonymisation, which attempts to weaken the link with the natural person. For medical data however, the authentication of the patient and the veracity of the information is of utmost importance. Also, the privacy of medical data is protected through established patient-doctor relationships which also require authentication of the parties involved.