FAQs from Digital Health Innovators

From the webinar: Quality assure your digital health technology

On October 18th 2022 N!P and ORCHA organised the webinar: The Nordic Digital Health Evaluation Criteria, quality assure your digital health technology.

How is NordDEC related to national Health Technology Assessment processes such as Digi-HTA in Finland?

Thanks for your question! We are so excited to see such strong leadership in digital health across Europe and welcome all efforts that are focused on making digital health, healthy.

For the benefit of our audience it is worth stressing that we are not seeking to establish Health Technology Assessment rather we are focused on Digital Health Technology Assessment processes and these differ from processes such as IQWIG and AMNOG which are not digital HTA.

The NordDEC is not a stagnant assessment scheme and has been designed to evolve in line with emerging guidelines, standards and best practice both within the Nordics, but also Europe more widely. This includes the forthcoming ISO 82304-2 which we are very excited about and country specific approaches such as the Digi-HTA in Finland.

We fully anticipate that the NordDEC will exclude elements identified as important to individual jurisdictions. However, we recognise that jurisdiction specific criteria can be very focused on that population group. Therefore it may not be appropriate to include these components in baseline assessment criteria that have been designed to cross borders across the Nordics.

In this regard we anticipate the NordDEC as providing an efficient baseline assessment process that enables stakeholders across the Nordics to benefit from a cost efficient process.

Its use allows local evaluators to focus on jurisdiction specific requirements rather than baseline assessment requirements being requested multiple times across the Nordics and the rest of the world, to the frustration of digital health suppliers.

Our community of digital health suppliers continually reinforce the value of an approach that is consistent, avoids duplication of effort and prioritises deployment of technologies.

I have reviewed the NordDEC framework and the specific criteria. Why are you not referring to the international standards for usability, risk management that are used to demonstrate fulfilment of MDR?

Great question! In developing the NordDEC assessment we have focused on developing an objective assessment that focuses on yes/no criteria and in doing so promote an approach focused on objectivity. Our goal is to create an assessment framework that promotes consistency, objectivity and removes bias.

It is also a broad assessment framework that does not discriminate against technologies that do not classify themselves as medical devices.

It refers to various international standards for usability, including Web Content Accessibility Guidelines (WCAG 2.0 AA and WCAG 2.1 AA), as well as ISO9241, Apple HIG and Android App Quality Guidelines.

In the context of risk management, ISO14971 is the standard for the Application of Risk Management for Medical Devices. It is important to note however that not all apps we review are Medical Devices so they may have no need to have this ISO. Therefore, its inclusion would move to a narrower view of digital health technologies which we think leaves large swathes of the market not applicable and therefore unchecked. For those products that do have this ISO, they will pass our question set on Risk Management as they will have all the documentation we require in this space.

We are focused on ensuring that every digital technology that has the potential to be used by patients, citizens and healthcare professionals is safe, not just those covered by existing regulations and that there are safeguards in place to monitor technology, where there are delays to regulators.

How can you state there is no guidance? MDR provides an excellent framework.

We agree but we also know from our digital health suppliers that the MDR can be tricky to navigate, timely and costly. Our approach is not to replace existing guidelines, frameworks and regulations rather to enhance with a clear step-by-step process that promotes compliance.

As explained in the previous response we know that not all digital health technologies fall within this criteria and seek compliance against the regulations. The risk being that if they do not fall within this category they go largely unchecked.

Equally, for a product that started out as a non-medical device but over time has introduced new features and functionality, we provide an assessment that highlights necessary areas of compliance, including MDR, thus making the NordDEC supportive of these frameworks. It provides a collaborative approach that promotes and supports relevant compliance with relevant regulations.

Our processes also allow for continuous review and monitoring of product updates and upgrades to ensure ongoing compliance.

We are also focused on supporting those unfamiliar with MDR and similar requirements who have no specialist interest to still be able to distinguish between good and bad products and the NordDEC has been designed with this in mind.

How similar/different is the NordDEC assessments from the ORCHA assessment performed in UK?

The ORCHA Baseline Review as a foundation level assessment has been designed to reflect guidelines, best practice and regulations from across the world. This has therefore identified a range of common criteria that has fed into the NordDEC.

However, the NordDEC goes beyond the ORCHA assessment focusing on a range of additional enhanced assessment criteria that have been identified by Nordic stakeholders as important to their decision making process. This includes technical stability and security. In this regard it is similar to the NHS DTAC process in England.

How may we obtain copies of the slides presented today and also a copy of the webinar video?

Slides and a copy of the webinar video will be made available to all those who registered via email and the webinar recording is available by request here.

As part of addressing the semantic interoperability issue do you have standards for vocabularies for naming conventions for lab tests, units of measure and reference ranges, diagnostic test naming conventions and dictionary of medicines and devices as examples?

The NordDEC, powered by ORCHA, operates in International English for our assessments. As the Nordic data warehouse grows in volume our platform is flexible to ensure that the information presented to HCP’s is relevant and understandable to the context.

The MDR has well-defined requirements for cybersecurity (https://health.ec.europa.eu/system/files/2022-01/md_cybersecurity_en.pdf) but NordDEC doesn't refer to these; instead you have formulated your own, different requirements. Standards regarding clinical evidence (iso standards 63179 and 72704) are not referred to). NordDEC asks for clinical evidence to be reported, but there is no evaluation of its relevance in the result. Do you see any credibility problems that might arise with this?

There is a wide range of Digital Health Technologies, with a wide range of features, and setting a single bar risks setting it too high or too low. By using an adapted version of the NICE Evidence Standards Framework for Digital Technologies, we introduce proportionality. Of course, as part of that, the bar includes "traditional" clinical evidence in some cases (Tier 3a/b or Tier C).

When these are necessary, we do look at measures of significance, sample size, comparators used, and we ensure these are appropriate, so that if an app is going to be used as a treatment, we ensure there is evidence of its effectiveness. There are frameworks, such as GRADE which can go further into this, but they require specialist involvement.

We have passed through ORCHA certifications and it didn’t help us in NHS or any other region in UK, what benefits we could expect from NordDEC through the framework?

The ORCHA Baseline Review, provided by our partners ORCHA, has been designed as a global foundation level assessment that supports digital health products work towards compliance against a range of national assessment schemes including the NordDEC and the NHS DTAC. The OBR is a pre-requisite to entry on to the London Procurement Partnership Dynamic Purchasing System for Health and Care Apps and ORCHA provide libraries across 70% NHS regions, making products visible to health care systems, healthcare professionals, patients and citizens.

Similarly, in the Nordics the intention is that upon passing the NordDEC assessment criteria your product will be surfaced on the Nordic Formulary from which health care systems can draw down from. The intention is to support digital health suppliers to distinguish themselves and demonstrate to healthcare providers across the Nordics that their product meets quality assurance standards, meet technical requirements and provide tangible benefits to their healthcare system, healthcare professionals and citizens.

The assessment frameworks, similar to MDR, is not a substitute for sales and marketing activities while these frameworks are not linked to national reimbursement mechanisms. That is ultimately a policy level decision at national government level. It does however provide you with an opportunity to seek external quality assurance which governments, healthcare systems, patients and citizens welcome in making choices about which apps to use.

What is the expected timeline for implementing the evaluation process? And who is responsible for this in each of the Nordic countries?

Currently, the NordDEC is powered by our partners ORCHA who complete the assessments. The timeline is largely determined by the product owner’s readiness to submit evidence against each assessment domain however it is estimated to take up to three months to complete assessment, dependent on a product’s readiness.

The assessment framework is available for use by national bodies and healthcare systems across the Nordics to use as a vital first step in deciding whether to purchase and deploy these technologies.

Are the sponsorships for the first accreditations, to get things going without putting too much risk on the developers?

Yes, absolutely! We welcome interested parties to submit details to info@norddec.org and we can discuss with you potential opportunities for assessment.