PERFORMANCE EVALUATION, PERFORMANCE STUDIES AND POST-MARKET PERFORMANCE FOLLOW-UP
PART A
PERFORMANCE EVALUATION AND PERFORMANCE STUDIES
1.
PERFORMANCE EVALUATION
Performance evaluation of a device is a continuous process by which data are assessed and analysed to demonstrate the scientific validity, analytical performance and clinical performance of that device for its intended purpose as stated by the manufacturer. To plan, continuously conduct and document a performance evaluation, the manufacturer shall establish and update a performance evaluation plan. The performance evaluation plan shall specify the characteristics and the performance of the device and the process and criteria applied to generate the necessary clinical evidence.
The performance evaluation shall be thorough and objective, considering both favourable and unfavourable data.
Its depth and extent shall be proportionate and appropriate to the characteristics of the device including the risks, risk class, performance and its intended purpose.
1.1.
Performance evaluation plan
As a general rule, the performance evaluation plan shall include at least:
a specification of the intended purpose of the device;
a specification of the characteristics of the device as described in Section 9 of Chapter II of Annex I and in point (c) of Section 20.4.1. of Chapter III of Annex I;
a specification of the analyte or marker to be determined by the device;
a specification of the intended use of the device;
identification of certified reference materials or reference measurement procedures to allow for metrological traceability;
a clear identification of specified target patient groups with clear indications, limitations and contra-indications;
an identification of the general safety and performance requirements as laid down in Sections 1 to 9 of Annex I that require support from relevant scientific validity and analytical and clinical performance data;
a specification of methods, including the appropriate statistical tools, used for the examination of the analytical and clinical performance of the device and of the limitations of the device and information provided by it;
a description of the state of the art, including an identification of existing relevant standards, CS, guidance or best practices documents;
an indication and specification of parameters to be used to determine, based on the state of the art in medicine, the acceptability of the benefit-risk ratio for the intended purpose or purposes and for the analytical and clinical performance of the device;
for software qualified as a device, an identification and specification of reference databases and other sources of data used as the basis for its decision making;
an outline of the different development phases including the sequence and means of determination of the scientific validity, the analytical and clinical performance, including an indication of milestones and a description of potential acceptance criteria;
the PMPF planning as referred to in Part B of this Annex.
Where any of the above mentioned elements are not deemed appropriate in the Performance Evaluation Plan due to the specific device characteristics a justification shall be provided in the plan.
1.2.
Demonstration of the scientific validity and the analytical and clinical performance:
As a general methodological principle the manufacturer shall:
identify through a systematic scientific literature review the available data relevant to the device and its intended purpose and identify any remaining unaddressed issues or gaps in the data;
appraise all relevant data by evaluating their suitability for establishing the safety and performance of the device;
generate any new or additional data necessary to address outstanding issues.
1.2.1.
Demonstration of the scientific validity
The manufacturer shall demonstrate the scientific validity based on one or a combination of the following sources:
relevant information on the scientific validity of devices measuring the same analyte or marker;
scientific (peer-reviewed) literature;
consensus expert opinions/positions from relevant professional associations;
results from proof of concept studies;
results from clinical performance studies.
The scientific validity of the analyte or marker shall be demonstrated and documented in the scientific validity report.
1.2.2.
Demonstration of the analytical performance
The manufacturer shall demonstrate the analytical performance of the device in relation to all the parameters described in point (a) of Section 9.1 of Annex I, unless any omission can be justified as not applicable.
As a general rule, the analytical performance shall always be demonstrated on the basis of analytical performance studies.
For novel markers or other markers without available certified reference materials or reference measurement procedures, it may not be possible to demonstrate trueness. If there are no comparative methods, different approaches may be used if demonstrated to be appropriate, such as comparison to some other well-documented methods or the composite reference standard. In the absence of such approaches, a clinical performance study comparing performance of the novel device to the current clinical standard practice is required.
Analytical performance shall be demonstrated and documented in the analytical performance report.
1.2.3.
Demonstration of the clinical performance
The manufacturer shall demonstrate the clinical performance of the device in relation to all the parameters described in point (b) of Section 9.1. of Annex I, unless any omission can be justified as not applicable.
Demonstration of the clinical performance of a device shall be based on one or a combination of the following sources:
clinical performance studies;
scientific peer-reviewed literature;
published experience gained by routine diagnostic testing.
Clinical performance studies shall be performed unless due justification is provided for relying on other sources of clinical performance data.
Clinical performance shall be demonstrated and documented in the clinical performance report.
1.3.
Clinical evidence and performance evaluation report
1.3.1.
The manufacturer shall assess all relevant scientific validity, analytical and clinical performance data to verify the conformity of its device with the general safety and performance requirements as referred to in Annex I. The amount and quality of that data shall allow the manufacturer to make a qualified assessment whether the device will achieve the intended clinical benefit or benefits and safety, when used as intended by the manufacturer. The data and conclusions drawn from this assessment shall constitute the clinical evidence for the device. The clinical evidence shall scientifically demonstrate that the intended clinical benefit or benefits and safety will be achieved according to the state of the art in medicine.
1.3.2.
Performance evaluation report
The clinical evidence shall be documented in a performance evaluation report. This report shall include the scientific validity report, the analytical performance report, the clinical performance report and an assessment of those reports allowing demonstration of the clinical evidence.
The performance evaluation report shall in particular include:
the justification for the approach taken to gather the clinical evidence;
the literature search methodology and the literature search protocol and literature search report of a literature review;
the technology on which the device is based, the intended purpose of the device and any claims made about the device's performance or safety;
the nature and extent of the scientific validity and the analytical and clinical performance data that has been evaluated;
the clinical evidence as the acceptable performances against the state of the art in medicine;
any new conclusions derived from PMPF reports in accordance with Part B of this Annex.
1.3.3.
The clinical evidence and its assessment in the performance evaluation report shall be updated throughout the life cycle of the device concerned with data obtained from the implementation of the manufacturer's PMPF plan in accordance with Part B of this Annex, as part of the performance evaluation and the post-market surveillance system referred to in Article 10(9). The performance evaluation report shall be part of the technical documentation. Both favourable and unfavourable data considered in the performance evaluation shall be included in the technical documentation.
2.
CLINICAL PERFORMANCE STUDIES
2.1.
Purpose of clinical performance studies
The purpose of clinical performance studies is to establish or confirm aspects of device performance which cannot be determined by analytical performance studies, literature and/or previous experience gained by routine diagnostic testing. This information is used to demonstrate compliance with the relevant general safety and performance requirements with respect to clinical performance. When clinical performance studies are conducted, the data obtained shall be used in the performance evaluation process and be part of the clinical evidence for the device.
2.2.
Ethical considerations for clinical performance studies
Each step in the clinical performance study, from the initial consideration of the need for and justification of the study to the publication of the results, shall be carried out in accordance with recognised ethical principles.
2.3.
Methods for clinical performance studies
2.3.1.
Clinical performance study design type
Clinical performance studies shall be designed in such a way as to maximize the relevance of the data while minimising potential bias.
2.3.2.
Clinical performance study plan
Clinical performance studies shall be performed on the basis of a clinical performance study plan (CPSP).
The CPSP shall define the rationale, objectives, design and proposed analysis, methodology, monitoring, conduct and record-keeping of the clinical performance study. It shall contain in particular the following information:
(a)
the single identification number of the clinical performance study, as referred to in Article 66(1);
(b)
identification of the sponsor, including the name, address of the registered place of business and contact details of the sponsor and, if applicable, the name, address of the registered place of business and contact details of its contact person or legal representative pursuant to Article 58(4) established in the Union;
(c)
information on the investigator or investigators, namely principal, coordinating or other investigator; qualifications; contact details, and investigation site or sites, such as number, qualification, contact details and, in the case of devices for self-testing, the location and number of lay persons involved;
(d)
the starting date and scheduled duration for the clinical performance study;
(e)
identification and description of the device, its intended purpose, the analyte or analytes or marker or markers, the metrological traceability, and the manufacturer;
(f)
information about the type of specimens under investigation;
(g)
overall synopsis of the clinical performance study, its design type, such as observational, interventional, together with the objectives and hypotheses of the study, reference to the current state of the art in diagnosis and/or medicine;
(h)
a description of the expected risks and benefits of the device and of the clinical performance study in the context of the state of the art in clinical practice, and with the exception of studies using left-over samples, the medical procedures involved and patient management;
(i)
the instructions for use of the device or test protocol, the necessary training and experience of the user, the appropriate calibration procedures and means of control, the indication of any other devices, medical devices, medicinal product or other articles to be included or excluded and the specifications on any comparator or comparative method used as reference;
(j)
description of and justification for the design of the clinical performance study, its scientific robustness and validity, including the statistical design, and details of measures to be taken to minimise bias, such as randomisation, and management of potential confounding factors;
(k)
the analytical performance in accordance with point (a) of Section 9.1 of Chapter I of Annex I with justification for any omission;
(l)
parameters of clinical performance in accordance with point (b) of Section 9.1 of Annex I to be determined, with justification for any omission; and with the exception of studies using left-over samples the specified clinical outcomes/endpoints (primary/secondary) used with a justification and the potential implications for individual health and/or public health management decisions;
(m)
information on the performance study population: specifications of the subjects, selection criteria, size of performance study population, representativity of target population and, if applicable, information on vulnerable subjects involved, such as children, pregnant women, immuno-compromised or elderly subjects;
(n)
information on use of data out of left over specimens banks, genetic or tissue banks, patient or disease registries etc. with description of reliability and representativity and statistical analysis approach; assurance of relevant method for determining the true clinical status of patient specimens;
(o)
monitoring plan;
(p)
data management;
(q)
decision algorithms;
(r)
policy regarding any amendments, including those in accordance with Article 71, to or deviations from the CPSP, with a clear prohibition of use of waivers from the CPSP;
(s)
accountability regarding the device, in particular control of access to the device, follow-up in relation to the device used in the clinical performance study and the return of unused, expired or malfunctioning devices;
(t)
statement of compliance with the recognised ethical principles for medical research involving humans and the principles of good clinical practice in the field of clinical performance studies as well as with the applicable regulatory requirements;
(u)
description of the informed consent process, including a copy of the patient information sheet and consent forms;
(v)
procedures for safety recording and reporting, including definitions of recordable and reportable events, and procedures and timelines for reporting;
(w)
criteria and procedures for suspension or early termination of the clinical performance study;
(x)
criteria and procedures for follow up of subjects following completion of a performance study, procedures for follow up of subjects in the case of suspension or early termination, procedures for follow up of subjects who have withdrawn their consent and procedures for subjects lost to follow up;
(y)
procedures for communication of test results outside the study, including communication of test results to the performance study subjects;
(z)
policy as regards the establishment of the clinical performance study report and publication of results in accordance with the legal requirements and the ethical principles referred to in Section 2.2;
(aa)
list of the technical and functional features of the device indicating those that are covered by the performance study;
(ab)
bibliography.
If part of the information referred to in the second paragraph is submitted in a separate document, it shall be referenced in the CPSP. For studies using left-over samples, points (u), (x), (y) and (z) shall not apply.
Where any of the elements referred to in the second paragraph are not deemed appropriate for inclusion in the CPSP due to the specific study design chosen, such as use of left-over samples versus interventional clinical performance studies, a justification shall be provided.
2.3.3.
Clinical performance study report
A clinical performance study report, signed by a medical practitioner or any other authorised person responsible, shall contain documented information on the clinical performance study protocol plan, results and conclusions of the clinical performance study, including negative findings. The results and conclusions shall be transparent, free of bias and clinically relevant. The report shall contain sufficient information to enable it to be understood by an independent party without reference to other documents. The report shall also include as appropriate any protocol amendments or deviations, and data exclusions with the appropriate rationale.
3.
OTHER PERFORMANCE STUDIES
By analogy, the performance study plan referred to in Section 2.3.2, and the performance study report, referred to in Section 2.3.3, shall be documented for other performance studies than clinical performance studies.
PART B
POST-MARKET PERFORMANCE FOLLOW-UP
4.
PMPF shall be understood to be a continuous process that updates the performance evaluation referred to in Article 56 and Part A of this Annex and shall be specifically addressed in the manufacturer's post-market surveillance plan. When conducting PMPF, the manufacturer shall proactively collect and evaluate performance and relevant scientific data from the use of a device which bears the CE marking and is placed on the market or put into service within its intended purpose as referred to in the relevant conformity assessment procedure, with the aim of confirming the safety, performance and scientific validity throughout the expected lifetime of the device, of ensuring the continued acceptability of the benefit-risk ratio and of detecting emerging risks on the basis of factual evidence.
5.
PMPF shall be performed pursuant to a documented method laid down in a PMPF plan.
5.1.
The PMPF plan shall specify the methods and procedures for proactively collecting and evaluating safety, performance and scientific data with the aim of:
(a)
confirming the safety and performance of the device throughout its expected lifetime,
(b)
identifying previously unknown risks or limits to performance and contra-indications,
(c)
identifying and analysing emergent risks on the basis of factual evidence,
(d)
ensuring the continued acceptability of the clinical evidence and of the benefit-risk ratio referred to in Sections 1 and 8 of Chapter I of Annex I, and
(e)
identifying possible systematic misuse.
5.2.
The PMPF plan shall include at least:
(a)
the general methods and procedures of the PMPF to be applied, such as gathering of clinical experience gained, feedback from users, screening of scientific literature and of other sources of performance or scientific data;
(b)
the specific methods and procedures of PMPF to be applied, such as ring trials and other quality assurance activities, epidemiological studies, evaluation of suitable patient or disease registers, genetic databanks or post-market clinical performance studies;
(c)
a rationale for the appropriateness of the methods and procedures referred to in points (a) and (b);
(d)
a reference to the relevant parts of the performance evaluation report referred to in Section 1.3 of this Annex and to the risk management referred to in Section 3 of Annex I;
(e)
the specific objectives to be addressed by the PMPF;
(f)
an evaluation of the performance data relating to equivalent or similar devices, and the current state of the art;
(g)
reference to any relevant CS, harmonised standards when used by the manufacturer, and relevant guidance on PMPF, and;
(h)
a detailed and adequately justified time schedule for PMPF activities, such as analysis of PMPF data and reporting, to be undertaken by the manufacturer.
6.
The manufacturer shall analyse the findings of the PMPF and document the results in a PMPF evaluation report that shall update the performance evaluation report and be part of the technical documentation.
7.
The conclusions of the PMPF evaluation report shall be taken into account for the performance evaluation referred to in Article 56 and Part A of this Annex and in the risk management referred to in Section 3 of Annex I. If, through the PMPF, the need for preventive and/or corrective measures has been identified, the manufacturer shall implement them.
8.
If PMPF is not deemed appropriate for a specific device then a justification shall be provided and documented within the performance evaluation report.