§ 170.405 Real World Testing

Revision History

Version #	Description of Change	Version Date
1.0	Initial publication	06-15-2020
1.1	Updated date in regulation text based on correction notice	08-07-2020
1.2	Updated compliance date per the Interim Final Rule with Comment Period (IFR), Information Blocking and the ONC Health IT Certification Program: Extension of Compliance Dates and Timeframes in Response to the COVID-19 Public Health Emergency; removed (f)(5) from C-CDA related requirements	11-02-2020
1.3	Updated to provide additional clarity on the Attestations Condition and Maintenance of Certification requirements.	03-12-2021
1.4	Updates to reflect changes outlined in Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing (HTI-1) Final Rule	03-11-2024

Regulation Text

Regulation Text

§ 170.405 Real World Testing.

1. Condition of Certification requirement. A health IT developer with one or more Health IT Module(s) certified to any one or more of the ONC Certification Criteria for Health IT in § 170.315(b), (c)(1) through (3), (e)(1), (f), (g)(7) through (10), and (h) must successfully test the real world use of those Health IT Module(s) for interoperability (as defined in 42 U.S.C. 300jj(9) and § 170.102) in the type of setting in which such Health IT Module(s) would be/is marketed.
2. Maintenance of Certification requirements-
   1. Real world testing plan submission. A health IT developer with Health IT Module(s) certified to any one or more of the criteria referenced in paragraph (a) of this section must submit to its ONC-ACB an annual real world testing plan addressing each of those certified Health IT Modules by a date determined by the ONC-ACB that enables the ONC-ACB to publish a publicly available hyperlink to the plan on CHPL no later than December 15  of each calendar year.
      1. The plan must be approved by a health IT developer authorized representative capable of binding the health IT developer for execution of the plan and include the representative’s contact information.
      2. The plan must include all health IT certified to any one or more of the criteria referenced in paragraph (a) as of August 31 of the year in which the plan is submitted, and address the real world testing to be conducted in the calendar year immediately following plan submission.
      3. The plan must address the following for each of the certification criterion identified in paragraph (a) of this section that are included in each Health IT Module’s scope of certification:
         1. The testing method(s)/methodology(ies) that will be used to demonstrate real world interoperability and conformance to the full scope of the certification criterion’s requirements, including scenario- and use case-focused testing;
         2. The care setting(s) that will be tested for real world interoperability and an explanation for the health IT developer’s choice of care setting(s) to test;
         3. For any standards and implementation specifications referenced by the criterion that the developer has chosen to certify to National Coordinator-approved newer versions pursuant to paragraph (b)(8) or (9) of this section, a description of how the developer will test and demonstrate conformance to all requirements of the criterion using all versions of the adopted standards to which each Health IT Module was certified as of August 31 of the year in which the real world testing plan is due.
         4. A schedule of key real world testing milestones;
         5. A description of the expected outcomes of real world testing;
         6. At least one measurement/metric associated with the real world testing; and
         7. A justification for the health IT developer’s real world testing approach.
   2. Real world testing results reporting.
      1. If in the course of conducting real world testing the developer discovers one or more non-conformities with the full scope of any certification criterion under the Program, the developer must report that non-conformity to the ONC-ACB within 30 days.
      2. For real world testing activities conducted during the immediately preceding calendar year, a health IT developer must submit to its ONC-ACB an annual real world testing results report addressing each of its certified Health IT Modules that include certification criteria referenced in paragraph (a) of this section by a date determined by the ONC-ACB that enables the ONC-ACB to publish a publicly available hyperlink to the results report on CHPL no later than March 15 of each calendar year, beginning in 2023. For certified Health IT Modules included in paragraph (a) of this section that are updated using Inherited Certified Status after August 31 of the year in which the plan is submitted, a health IT developer must include the newer version of the certified Health IT Module(s) in its annual real world testing results report. The real world testing results must report the following for each of the certification criteria identified in paragraph (a) of this section that are included in the Health IT Module’s scope of certification:
         1. The method(s) that was used to demonstrate real world interoperability;
         2. The care setting(s) that was tested for real world interoperability;
         3. The voluntary updates to standards and implementation specifications that the National Coordinator has approved through the Standards Version Advancement Process.
         4. A list of the key milestones met during real world testing;
         5. The outcomes of real world testing including a description of any challenges encountered during real world testing; and
         6. At least one measurement/metric associated with the real world testing.
   3. [Reserved]
   4. [Reserved]
   5. [Reserved]
   6. [Reserved]
   7. [Reserved]
   8. Standards Version Advancement Process – voluntary updates of certified health IT to newer versions of standards and implementation specifications. A health IT developer subject to paragraph (b) of this section is permitted to update Health IT Module(s) certified to any one or more of the certification criteria referenced in paragraph (a) of this section to a newer version of any adopted standard or implementation specification included in the criterion, provided that newer version is approved by the National Coordinator for use in certifications issued under the ONC Health IT Certification Program. A developer that pursues such updates of its certified Health IT Module(s) must:
      8. Provide advance notice to all affected customers and its ONC-ACB-
         8. Expressing its intent to update the certified Health IT Module(s) to the National Coordinator-approved advanced version of the standard implementation specification;
         9. The developer’s expectations for how the update will affect real world interoperability for the Health IT Module(s);
         10. Whether the developer intends to continue to support the certificate(s) for the existing certified Health IT Module(s) version(s) for some period of time and how long or if the existing certified Health IT Module(s) version(s) will be deprecated; and
      9. Successfully demonstrate conformance with approved more recent versions of the standard(s) or implementation specification(s) included in each certification criterion under which the developer chooses to update its certified Health IT Module(s).
      10. Maintain the updated certified Health IT Module(s) in full conformance with all applicable Program requirements.
   9. Standards Version Advancement Process - voluntary certification to newer versions of standards and implementation specifications. A Health IT developer is permitted to seek certification for its Health IT Module(s) to any one or more of the certification criteria referenced in paragraph (a) of this section using a newer version of any adopted standard(s) or implementation specification(s) included in the criterion without first obtaining certification to the version of that adopted standard or implementation specification that is incorporated by reference in § 170.299, provided that the newer version is approved by the National Coordinator for use in certifications issued under the ONC Health IT Certification Program. Developers may, for each standard and implementation specification included in each criterion, choose on an itemized basis whether to seek certification to the version incorporated by reference in § 170.299, or to one or more newer version(s) approved by the National Coordinator for use in Health IT Module(s) certifications issued pursuant to section 3001(c)(5) of the Public Health Service Act, or to both.

Standard(s) Referenced

Standards Referenced

None.

Certification Companion Guide: Real World Testing

Real World Testing Resources