MCERTS: standards for environmental data acquisition and handling systems

Question 1

DAHS to which this performance standard applies

Accepted Answer

DAHS that can undergo the validation process are typically:

PC-hosted data management applications that perform functions including data monitoring, data storage and archiving, displays and reports
embedded applications hosted on measurement instruments or sensors that perform data management functions
distributed control systems (DCS)
supervision, control, and data acquisition (SCADA) systems

Some measurement instruments perform a certain amount of data manipulation, such as averaging, sampling, smoothing and calibration, which could fall within the remit of this standard. However, if these instruments provide data to DAHS, it would only be necessary for an instrument supplier to submit a data management application running as embedded software (programs and data that permanently in read only memory or flash memory) in the instrument if at least one of the following conditions applied:

the instrument generates official reports directly without the participation of a DAHS
the data manipulation is of a sophistication that raises questions as to whether the software is performing hidden manipulations on the data
there are doubts about the provenance of the software, for example, its origin or maintainability is uncertain
the supplier is uncertain about the present status of the software in terms of its lifecycle, change control and other quality issues

Question 2

Background to this standard

Accepted Answer

Here we provide developers of DAHS with an overview of the DAHS certification process.

Quality assurance and auditing

The following principles apply to auditing of your DAHS.

Quality management system

You must have a quality management system (QMS) that complies with the requirements of ISO 9001 ‘Quality Management Systems’. You can demonstrate conformance by certification of the QMS to ISO 9001. If the QMS is not certified, the MCERTS assessor will decide the suitability of the QMS during the product certification, surveillance, and recertification audits.

Evidence of testing and other verification and validation methods

To gain an MCERTS certificate, an MCERTS assessor must audit the DAHS. The audit report provides the evidence to the certification body to support the award of the certificate.

Audit check lists

The certification body will give you checklists that will help you supply the necessary evidence to support the issue of an MCERTS certificate.

Issue of concessions during an audit

During an audit you may request that the assessor grants a concession for a particular clause or sub-clause. A concession can be permanent for the term of the certificate or temporary. The certification body will review a permanent concession at each recertification audit.

A temporary concession is time limited. You must make good the non-compliance for the concession within this time. You must inform the certification body as soon as you make good the non-compliance, so they can update the information supporting the certificate. The certification body will review temporary concessions at every surveillance audit.

In either of the above cases, the criteria for applying a concession must make sure:

it does not reduce the integrity or security of any data
you have used an alternative means of fulfilling a requirement that delivers all or most of the required functionality
no alternative exists, but you can show the DAHS software has no need to comply with the clause or sub-clause

The assessor will identify the concessions in their report that supports the certificate.

Continuing development

You must make use of software metrics tools to build a quantitative justification for claims of maintainability of the software.

Manufacturer’s published documentation

You must make available a technical manual and an operating manual to the users of the DAHS. This documentation should be part of a ‘help’ system if the host computer has the appropriate human-machine interface (HMI). This information must also be available to the MCERTS assessor involved in the certification process.

Background to part A of the standard – Generic software quality

The DAHS may have already been certified to standards, such as:

ISO/IEC 12207 and ISO/IEC 15288 ‘Software Lifecycle Processes’
ISO/IEC 15504 Information technology – Process assessment

This means that the DAHS has already undergone assessment by an independent expert, whose report provides evidence of compliance. The MCERTS assessor will review the report and any resultant certificate by comparing it with Part A of this standard.

In the absence of any pre-existing quality assessment, it will be necessary to audit the software against Part A of this standard.

Many DAHS are hosted on PCs, which make use of a variety of software components to handle functions, such as graphical displays and real time databases. Often, these applications are written by following a ‘rapid development’ or ‘agile’ software lifecycle model, where subsets of the functionality are successively developed and acceptance tested, each during a short period of time. Such approaches are acceptable.

You must support and maintain your DAHS effectively in the long term if it is to keep its MCERTS certificate. An audit will require the company to answer questions about its ability to withstand the loss of key staff, withdrawal of support for key software components and other threats to the viability of the product.

Background to parts B and C of the standard – Data management

Relevance

You must make the aim, scope, and application of any DAHS clear and refer to the environmental regulations to which it applies. An MCERTS assessor will confirm that the evidence exists to support a ‘fully compliant’ result for each relevant criterion. You must mark those criteria that are not relevant as ‘N/A’ (not applicable).

Your responsibilities

It is your responsibility to fix defects, decertify defective versions, and to certify new ones. Before the certification body issues a certificate, you must remedy defects in the software lifecycle and in the DAHS that the MCERTS assessment finds.

Transparency

You must state the methods and data you use. They must be auditable. It is generally not acceptable to submit data in response to regulatory requirements, which has been subject to hidden calculations, assumptions, and policies. You must share manipulations to data with MCERTS assessors. The MCERTS assessor will use this for purposes of assessment only and will keep it in commercial confidence.

Data integrity

You must make sure that:

data is stored and managed without becoming corrupted
appropriate calculations are correctly implemented
reports correctly represent raw or processed data
measurement uncertainty is being addressed and propagated to the degree required for the data and decisions which are being supported
you clearly state limitations and qualifications

You must produce a mathematical specification that shows that you use appropriate algorithms and arithmetic. An appraisal of the algorithms with sample calculations must be available, so that it is possible to prove the robustness of the algorithms across the full range of data. Additionally, you must justify the appropriateness of manipulations, such as smoothing or filtering techniques, if you use them.

Completeness

You must make it clear how the DAHS fulfils the requirements that:

data and other contextual information are stored and managed properly within the system
data are available to the operator and used in the preparation of reports
where data has the potential to affect decision-making (such as outlying data points and details of sampling conditions) then this information is used correctly in the generation of interim and final reports
in treating outliers, the DAHS software distinguishes between when it treats them as legitimate data, and when it deems them to be in error
the DAHS software makes the treatment of outliers clear

Security

You must include:

provisions to protect against loss or corruption of data
prevention of infection by viruses and all other forms of malware

Documentation

You must include a guide to the DAHS that enables its operator to use it properly. It must also discourage improper use.

Embedded data

The data that you include must be from an approved source, such as a reference document. Where you must keep such data up to date, you must confirm that you apply updated versions correctly. Equally, embedded data must be correctly accessed and used only for approved and appropriate purposes.

Embedded methods, policies, and assumptions

Methods and assumptions must be technically valid, reasonable, clear, and agreed with or approved by one or more Industrial bodies. For example, the Joint Environmental Programme for the power industry has published Electricity Supply Industry – IED Compliance Protocol for Utility Boilers and Gas Turbines (LCP BREF Update 2022).

Hardware and complementary software requirements

You must specify computer architecture, operating systems, application program platforms (for example, Microsoft Excel, Access) and supplementary software elements. You should include a rationale of the choice of any software components that you incorporate into the DAHS software.

Continuity of certification

You can withdraw old and defective versions of software, while automatically self-certifying new versions by applying the following safeguards:

a test specification that is part of the DAHS documentation that was audited against the MCERTS criteria for data management
automated installation of the new version’s software components
automatic validation, using the audited test specification, of new versions using audited dummy data sets or equivalent realistic test data sets
distinguishing between ‘major’ changes that may require external assessment by an MCERTS assessor and ‘minor’ changes that do not
surveillance audits by the certification body of the software supplier’s continued adherence to the MCERTS standard

You must validate any change that effects the integrity of reports or displays of regulatory information. Your validation must be at least as rigorous as the MCERTS compliant validation process audited during the certification process.

Question 3

Part A of the performance requirements – generic software quality

Accepted Answer

The following paragraphs state the generic software quality requirements of the standard on the developer of the DAHS.

A1 Software lifecycle definition

A1.1 You must justify the choice of the software quality standard used in Part A. You need to confirm whether the choice of Part A standard is appropriate for your DAHS software. You must risk assess each application to establish whether other standards that are more rigorous than this one should apply. If you prefer, you may use another standard of at least equal rigour to Part A of this one.

You must justify alternatives to Part A to the assessor.

A1.2 If the software quality of the DAHS is already certified then you must supply a copy of the certificate and the existing report to the MCERTS assessor. Evidence referenced in the supplied report must also be available to the MCERTS assessor. You must make sure that the previously certified version of the software is the version that you will use in the MCERTS certification and that no unauthorised changes have been made since that previous certification.

A1.3 You must have a software quality plan (SQP), which includes a definition of the software lifecycle that applies to the DAHS software. The SQP must also include a definition of responsibilities as stated in section A1.14.

A1.4 You must list all documents produced and used in the development of the DAHS. The list must include reference numbers and version numbers.

This identifies the documents you need to produce and whether you need to maintain them as live documents throughout the lifecycle.

A1.5 You must apply a change control procedure throughout the DAHS software lifecycle.

A1.6 You must hold all DAHS software, reference data and other objects that constitute a release of the DAHS software under software configuration management (SCM). You must use an SCM tool (or tools) to automate this process.

You must use an SCM tool for source code and software components. You should use SCM tools, document management systems, or other facilities for holding other data, software tools, and documents. If you do not do this, you must justify why to the MCERTS assessor.

A1.7 You must define the version of all software components and packages used.

A package is a body of software, such as SQL Server or a SCADA package. You must make sure that you maintain these packages and that the suppliers’ bug fixes and updates are available, so that the DAHS makes use of up-to-date versions of packages and components.

You need not keep software such as graphics components that you have integrated into the DAHS up to date. You should judge whether the version in use performs consistently and reliably.

A1.8 You must maintain a system for reporting, logging, and tracking software defects and other alerts.

You should log and track defect reports either in a manual system or a database tool.

A1.9 The software defect reporting system must enable you to produce accurate release notes with each release of the product. Each release of a new version must have release notes that include:

a list of software defects that the new version fixes
a list of new features and facilities included in the new version
a statement as to the level of testing performed during the validation of the software
a list of unfixed software defects that could have significant impact on the user and calculated or reported results

A1.10 You must define the means used to verify the software at each stage of its development in the SQP.

Verification will typically be a combination of reviews and testing. You must identify the degree of testing. MCERTS audits require evidence that verification has taken place so you must archive the minutes of reviews and the results of tests and make them available for inspection. You can log reviews in lab notebooks, copy and file them manually, or maintain them as an email thread. You must hold test results under SCM.

A1.11 You must list in the SQP all software tools and any commercial off the shelf hardware and software packages used in the software lifecycle. You must also identify the version number of each software item.

You must identify special in-house utility programs, spreadsheets, VBA (Visual Basic for Application) plug-ins running within Microsoft Office applications, and other specialised tools. You must maintain these tools under SCM and support any platform applications such as Microsoft Office. You must identify their version numbers.

A1.12 You must backup all software and documentation. The backup procedure must include:

procedure for regular on-site backups
procedure for regular off-site backups
procedure for restoring files
procedure for regularly testing the backup arrangements

A1.13 You must password protect access to the DAHS software development environment.

A1.14 You must define the responsibilities of everyone working on the project.

This identifies how you log communications such as emails, who is to produce each document and code module, and who is responsible for reviewing it.

Small companies may not have sufficient staff to enable independent code and document reviews to be undertaken. In this case, you must explain this in the SQP. You should use an external reviewer during the development. If you do not have evidence that reviews have taken place, then that evidence must be produced as an addition to an MCERTS audit.

A2 DAHS requirement specification

A2.1 You must define the functions that the DAHS needs to perform, the interfaces it will use, and the constraints under which the DAHS must run.

You must provide evidence that you have specified the functions and interfaces to the extent that you can specify validation tests for them. For example:

if a user interface includes menus, then functions are defined for each menu item and exception conditions and user input errors are trapped
for each communications interface, you have defined the behaviour of the software for normal and faulty operation

A2.2 The functions, interfaces and reports that have been specified by prototyping must be reviewed by the specifiers and revised according to the results of the review. You must archive the results of the review in your documentation.

A2.3 As soon as you have delivered the DAHS to a user, it must come under change control.

You must use SCM throughout the development, including during prototyping.

A2.4 You must produce a technical manual for the DAHS. It must be a live document that describes the functions, interfaces, and reports of the DAHS.

A3 Software detailed design

A3.1 You must make all design information available to the MCERTS assessor.

You may use any design method, provided it meets the DAHS’ requirements and the tools and components used to build it. If the application makes use of a database, then its data dictionary must be available and must include appropriate measures for making sure of data integrity. If you use a SCADA package in the application, then the points or tags database containing the measurements used by the SCADA package must be defined and available to the MCERTS assessor.

A3.2 You must maintain sufficient design information to make sure the software does not decay, and you support it over the long term. You must justify to the MCERTS assessor the steps you have taken to make sure you can maintain the software. You must include your justification in the detailed design document.

A4 Software coding and unit testing

A4.1 You must use a coding standard for all the languages used to develop the software. The coding standard must define guidelines and conventions for naming, modularity, module complexity limits, testing, and ease of software maintenance. You must also define guidelines and conventions for defensive programming that protects programs against the effects of receiving bad data from diverse sources, including systems, users, and other programs.

If you use scripting languages, such as SQL and JavaScript, then there must be a coding standard for each one.

A4.2 Your coding standard and all other guidelines used must be in the MCERTS assessor’s audit.

A4.3 You must makes sure that a competent person, other than its author, reviews all code.

You should use software tools to support the reviewing process. You can focus reviews on parts of the code that are the more critical than others. The justification for selective levels of code review must be available to the MCERTS assessor, typically in the SQP.

A4.4 You must subject all code to static analysis, if static analysis tools are available.

You should apply static analysis tools before code review to enable the review to concentrate on deeper design issues. You must provide a justification for not using static analysis tools in the SQP. If static analysis tools are not available or cannot be used for justifiable reasons, you must use desk checking for data flow errors, such as the use of un-initialised variables.

A4.5 When tool support for gathering code metrics is available at reasonable cost, then your static analysis must also include the gathering of source code metrics. This applies to the programming languages used to implement the software. You must gather these at both the function and file (unit or module) level. The metrics must include code size (lines of code), comment size, comment density and complexity (number of decision points per function). Also, if possible, it should include the number of individual calls in and out of each function. If you do not use code metrics in the SQP, you must justify why.

DAHS may consist of hundreds of thousands of lines of source code. This means the measurement and control of complexity is essential to safeguard the long-term maintainability of the software.

A4.6 You must follow the principles and methods of defensive programming. Code reviews will ascertain that you follow these methods.

A4.7 You must justify the choice of the level of unit (module) testing in the SQP. Unit testing is a test procedure used to validate that individual modules or other units of source code are working properly. Unit testing must include some or all the following techniques:

structure testing including testing of significant branches
boundary value testing this is a testing technique using input values at, just below, and just above, the defined limits of an input range. The input values cause outputs to be at, just below, and just above, the defined limits of an output range

Structure testing must exercise at least both sides of every branch. Boundary value testing involves values just inside or just outside the specified limits for each input (‘off-by-one values’). It includes special cases such as empty arrays, empty strings, and zero values.

You may not need to do unit testing for functions of low complexity because static analysis and code review may provide sufficient verification. If you use this selective approach to unit testing, then justify it in the SQP.

A4.8 You must use regression testing. You must repeat tests for each new version of a code module before using it in a new DAHS software build.

A5 DAHS software integration and validation

A5.1 You must validate the DAHS software against an acceptance test specification (ATS). Validation must include functional and performance tests.

Where you use automated test tools and harnesses, you can specify the ATS as a set of scripts or similar means used to drive the tests.

A5.2 Some software modules are more critical than others. You may use more testing for critical modules and less for lower criticality ones, such as those performing offline reporting. For offline reporting, you should have a set of standard input files and a set of standard output files against which you can compare the results from the standard inputs.

The ATS must specify tests including some or all the following, depending on the criticality of the part of the DAHS software under test:

realistic tests that represent the likely values encountered when using the DAHS
boundary value testing
unusual combinations of inputs, including physically unlikely input values
error handling

These include negative values where positive is expected, out-of-range inputs, missing files and bad path names, and tests that cause modules or functions within the DAHS to return an error.

User interface tests should include cancelling dialogue boxes, pressing inappropriate buttons, aborting the software when it is running and dragging forms.

Stress tests assess the DAHS software under extreme operating conditions. They should include exposing the software to maximum data rates, writing large disk files, and operating the software with high network traffic.

A5.3 You must validate each release of the DAHS software against the ATS. You may use a subset of the ATS to conduct partial regression testing when the changes are minor. You must include a justification of the level of testing performed in the release notes for that version.

A5.4 You must review the ATS to make sure you have effective coverage. The review must supply evidence that the set of tests achieves an appropriate level of validation.

A6 Commercial status of the supplier

A6.1 You must supply evidence that you can provide technical support for the DAHS.

A6.2 You must make arrangements for the ongoing support of the DAHS software in the event of being unable to continue that service.

You may place the DAHS software source code and its design information on a web file storage service, so that it is available to others. If you do this, you must provide evidence for this arrangement and instructions for retrieving the source and design information to the certification body. If you are unable to support the software, the certification body must give customers of the software, access to the source and design information on request.

Alternatively, you may declare the DAHS software source code and its design information as saleable intellectual property that you can make available to others commercially in a sale of assets.

You should put each new valid release as a set of zipped files into a separate directory on the backup service. You will delete releases from the backup service when you withdraw them. These files can be password protected. You must inform the certification body about how this information can be accessed.

Question 4

Part B of the performance requirements – data management

Accepted Answer

The following paragraphs state the data management requirements of the standard on the developer of the DAHS.

B1 Introduction

Part B covers the general aspects managing the environmental measurement data. You must include reference data and the data that describes the origin of the sources of the data in the scope of the system.

B2 DAHS documentation

B2.1 You must accompany every installation with release notes.

This applies to changes in functionality and changes to any constant data that are in the software, such as tables and conversion factors.

B2.2 You must make sure that an operations manual is available. This must include guidance to novice users, except in the case where the DAHS is only used by previously trained staff.

B2.3 You must supply a technical manual. It must include the input measurements, data processing algorithms, data storage and databases, outputs to external interfaces, reports, and other functions and interfaces of the DAHS.

B2.4 You must state requirements for any training in the operating manual.

B2.5 If the functionality of the DAHS can be extended at the source code level by the customer by adding a source code plug in, then you must make the necessary interfacing details available in the technical manual. If plug-ins result in a significant change to functionality and source code, they will be included in the MCERTS assessment.

The MCERTS certificate for a product containing plug-ins will include a list of the certified plug-ins.

B3 Mathematical specification and the processing of measurement data

B3.1 You must produce a mathematical specification for DAHS.

You must specify the steps in the processing of measurement data and identify their associated sources of error. You may use a simple mathematical specification that only includes averaging.

B3.2 You must include the following in your mathematical specification:

specification of numerical inputs and their ranges, their validation, and the handling of outliers
definition of all formulae and specification of the algorithms used in the DAHS, including their limits of validity and evidence of their suitability in numerical analytic terms
use of stable formulae whose solution is not vulnerable to rounding errors
validation of any non-trivial algorithms by means of an offline implementation using representative test data

If relevant to your DAHS, the mathematical specification must show:

the appropriate use of any floating-point arithmetic and the avoidance of practices such as cancellation, division by very small quantities, exact comparison of floating-point numbers
the use of well-conditioned formulae where small changes in the input data do not cause relatively large changes in the solution
appropriate use of any scaled integer arithmetic

During DAHS software validation, use the results of the above offline validation of non-trivial algorithms to corroborate the results from the online version. You must make sure that any test data set and test software have the same level of design and control as the main DAHS software.

B3.3 When available, you must use instrument status information to show the validity of measurements. The mathematical specification must specify how to use the status information.

You must make sure that the DAHS makes use of the following data, where available:

measurement uncertainty and device statuses returned by measurement devices
indications of the validity of measurements based on the status of the measurement devices and measurement uncertainties computed within the DAHS

B3.4 You must make sure that all input data is verified before it is used in calculations, reports, displays or other outputs:

You must never use unverified data in any outputs from the DAHS.

If the DAHS receives data from another system that has already verified the data and marked it as such then there is no need for you to repeat the verification. You must identify this data in the mathematical specification and its non-verification justified.

Unverified data is for example data that has not been range checked or had its status code accepted.

B3.5 You must make sure that each measured value is accompanied by an indication of its validity. If the measured value is invalid, then you must record the reason for this. You must record with the measured values, all conditions that can make one or more measured values invalid. Your method of recording must allow the identification of affected data, together with the reason for their invalidity.

You must make sure that the data processing uses the indication of validity to decide whether measured values are eligible for inclusion in the calculated values and reports.

Each indication of validity does not have to be in the same file as its corresponding measured value. However, you must make sure, that there is an unambiguous and robust relationship between each measured value and its indication of validity.

You must make sure that the documentation makes it clear how the DAHS manages invalid values. You can do this by holding the last valid value, by substituting a default value or exclusion of the data.

B3.6 You must make sure that the DAHS is able to ignore input data acquired during times and conditions when measurements do not have to be reported.

B3.7 The DAHS must be able to process the measurement data without capping the peak measurement values.

B4 Traceability and auditability of data

B4.1 You must make sure that the measurement data records are traceable to:

the physical location
time of the measurement
the measurement devices used
the status of the devices

B4.2 You should use coordinated universal time (UTC) for the time and date of the measurement. If you do not do this, you must justify the reason in the SQP or DAHS design documentation.

You should use an independent source for the time and date measurement. If you do not do this, you must justify the reason in the DAHS design.

B4.3 You must make sure that the status history of measurement devices is held so that measurement data can be traced to the operational status of the device (if available) at the time of the measurement.

If the calibration of the device is traceable to a standard, then include a reference to that information, or a similar indication, in the operational status.

B4.4 You must make sure that all measurement data records are auditable so that the original input data and all later modifications can be identified.

You may use the following ways to store data for the first time:

store the original sample data in a database for raw data and store the average data in separate average tables in the database
you only need to store the average data – you can buffer the raw values temporarily for the purposes of computing averages, but you do not have to store them in the long term

You must make sure that all data that enters long term storage must have all changes to it logged in an audit trail.

B4.5 You must make sure that all manual changes to data are auditable and traceable to the date, time, and operator responsible for the change.

You must make sure there is no hidden manipulation of data.

You must protect a measurement database from attempts to export data into a spreadsheet application, to change the data, and then to import the changed data back into the measurement database. If such an operation is performed on database records, then this fact must be logged and must appear in any audit trail related to the custody of the measurement data.

In cases where a database holds the measurement data and averages in various tables, you must be sure to prevent access to the data via unauthorised SQL scripts, typically run from high privilege accounts.

B4.6 You must make sure that changes to configurable DAHS includes an audit trail.

You must make sure that DAHS that are set up with a configuration tool that adds, removes, and edits the definition of data inputs and calculations has those changes recorded. This enables the configuration at a particular time to be readily available.

B4.7 You must make sure it is possible to find the version of the software used to log an item of data.

As a minimum, this must include a record of the dates and times during which each version of the software was deployed.

B5 Storage and security of data

B5.1 You must make sure access to the DAHS and all its data is password protected. You must use separate protected passwords, where access to data must be segregated between different user roles.

B5.2 You must make sure that the DAHS can log all attempts at unauthorised access. You must also make sure that the DAHS user or administrator are made aware of any attempts at unauthorised access.

You should consider encrypting access log files to prevent a security weakness arising from the unencrypted logging of failed accesses, where a password has been entered into a username field.

B5.3 You must make sure data is encoded and stored in a form that cannot be changed manually.

Typically, to do this, you should not use human readable data representations.

B5.4 You must make sure all access to the data is checked against corruption or loss of data. In cases where these checks are not possible, you must justify their exclusion to the MCERTS assessor.

You must make sure that databases and other means used to store data must apply data integrity checks when each record is accessed. You must use error checking methods, such as cyclic redundancy codes or other means of detecting corruption and inconsistency on data records.

B5.5 You must make sure that the software can check databases and other repositories of data at regular intervals for the loss or corruption of data.

B5.6 You must make sure the DAHS includes functions that facilitate the auditing of its data and the tracing of data in terms of its origin and path, modification, and other manipulations.

These facilities will aid auditability and traceability. It should enable the user of the software to view a report and establish that data that contributes to the report has been through processes such as smoothing, averaging, and manual editing.

B5.7 You must make sure that the origin and path of the data includes the physical location of the raw measurement or the point of sampling, the sample stream (if relevant), the instrument or sensor, the input channel and network node (if relevant).

B5.8 You must make sure that the DAHS and its data is backed up, so that it is capable of restoration without undetected loss of data.

If hardware failures or operating system failures cause the loss of stored data or measurements not to be made, then you must advise users of the gaps in the data in any reports generated from the incomplete data.

B5.9 You must make sure that calibration data, constants, operating parameters, and all other supporting data is auditable, and that the history of all modifications is recorded. The supporting data must include the value, units, period of validity, and origin. If the end user chooses, history files can be archived offline after a period of greater than 3 months, or a longer period if a relevant sector standard in Part C requires this.

Where calculations use supporting data, you must be able to show whether, for example, an instrument has been recalibrated during a reporting period. This clause applies to data entered manually, data uploaded from instruments, or from other sources such as reference tables.

B5.10 You must set up an audit trail for each set of supporting data, with the time and date of each change, the name and role of the modifier.

You must make sure that the data and the assumptions used for calibration curves are traceable.

B5.11 The DAHS must support the display and reporting of audit trail information on demand. You must have an audit trail that can go back at least 6 years.

B5.12 You must design the DAHS to minimise the manual transcription of data and other avoidable sources of error.

B5.13 You must make sure that the operating system for the PC conforms to a suitable security level, if the DAHS runs on a PC on which other applications can be loaded or is on a network that requires protective security measures.

B5.14 You must make sure that it is possible to interrogate the stored data at any time without interrupting the continuous data logging mechanism.

You must make sure that the DAHS has the capability to store data for the length of time specified by the applicable regulatory requirements.

B6 Interfaces to measurement devices

B6.1 You must make sure that the DAHS:

logs failures leading to loss of data from each measurement channel
logs the reason for failures, if available
makes failures known to users

B6.2 Where a group of measurements must be synchronised within a time window, the DAHS must log any loss of synchronisation, and the corresponding measurements marked to that effect.

B6.3 You must make sure that the DAHS time-based measurements are normally synchronised with a calendar and clock. This means that normally a measurement made every 15 minutes takes place at HH.00, HH.15, HH.30, and so on.

If you need to defer any time-based measurements to allow a fixed number of time units, you must justify this in the technical manual.

B6.4 You must make sure that time and date formats avoid representations that lead to potential errors within a 100-year period beginning at the earliest in 1980.

You must state in the technical manual the limits on the time and date handling of the DAHS.

B7 Report generation and the display and presentation of data

B7.1 You must make sure that all reports and displays show whether any measurement data or calculated results are based on invalid or incomplete data.

Incomplete data would be for example, an average measured value taken over fewer points than normal because an interface has been temporarily out of use.

B7.2 You must make sure that the calculation and presentation of data in displays and reports complies with relevant regulatory requirements. You must make sure that references to the relevant regulatory requirements are available to the user and listed in the DAHS help section and in the technical manual.

B7.3 You must make sure that reports and displays are labelled with a specific title.

B7.4 You can only mark reports generated by MCERTS certified DAHS as MCERTS compliant.

You must make sure that MCERTS compliant reports are identified by their official title in the header and marked with the MCERTS logo.

In some circumstances a report may have to include information based on data from both MCERTS certified and non-certified sources. You must make sure that the MCERTS data is identifiable to the user.

B7.5 You must make sure that the export of reports and displays do not cause the loss of validity and invalidity information, or the incorrect application of MCERTS logo.

B7.6 You must mark any displays or reports that contain test data.

B7.7 You must make sure the DAHS can generate reports automatically or on request.

B8 Confirmation of the certification of new versions and withdrawal of defective versions

B8.1 You must make sure that new versions of the DAHS remain compliant with this standard.

Once the MCERTS certificate is issued you must make sure that the software is appropriately maintained using the audited software lifecycle.

B8.2 You must make sure that each release of the DAHS software has a version number of the form ‘major-minor’.

A major release requires recertification, a minor release does not. When you decide a change is minor you must justify the decision by listing the affected modules and showing that the change is very localised. You can do this automatically by using a code navigation tool that shows module dependency by cross-referencing.

A minor release can have one of the following characteristics:

a localised change within the code to fix a software defect, to reformat a display or report, or to add a feature
a small modification for a site-specific feature that you must introduce to meet the end user’s requirements
a change to include a new edition of reference tables
a rebuild to take advantage of a new release of a function library

A major release entails, for example:

addition of significant functionality, for example, a new class of input devices
a port to a different operating system or database package
a set of changes that affect many modules in the DAHS and which require substantial modifications to several functions of high complexity

You may use alternative numbering formats so long as the major-minor component remains obvious.

B8.3 In the event of a major release you must provide supporting evidence to the certification body who will advise whether recertification is necessary.

B8.4 You must accompany each release with release notes, which must include a justification as to why the release is major or minor. You must support the justification by reference to the number of source files changed or added, and the number of lines of code that you change or add.

B8.5 An MCERTS certificate is valid for 5 years. When the period is close to expiry, or when there has been a succession of ‘minor’ changes, you must apply for an extension to the certificate by sending the release notes (the change history) for the minor changes to the certifying body.

The set of release notes will show whether the succession of ‘minor’ changes amounts to a significant change that requires recertification.

B8.6 The certification body carries out an annual surveillance check of the software development process.

B8.7 Where a DAHS is sold as a member of a family of DAHS, and each member is a subset of the main DAHS, then it is permissible to certify the main DAHS only. You may say that the subset is also certified, provided that the subset is clearly a subset of a single build of the main DAHS software code.

B8.8 You can only withdraw a version of a DAHS when there is evidence that continuing use of that version will compromise the quality of the data. You must make this evidence available to all purchasers of the DAHS. It is permissible to have different versions of a DAHS in use at the same time if each has MCERTS certification.

B9 Software installation and acceptance

B9.1 You must make sure that the installation of the DAHS software and its associated files is automated.

B9.2 You must make sure that de-installation and quarantining of earlier versions of the DAHS software and its associated files is automated.

B9.3 You must make sure that you accompany each release of a DAHS with release notes and is uniquely identified by a version number of the form ‘major-minor’ (in accordance with the software configuration management requirements and section B8).

B9.4 You must make sure that the user can test each release of the DAHS. If the user requires, you must be able to demonstrate that the algorithms used in the DAHS are applied correctly to audited sets of test data.

Where possible, once you have completed installation of the DAHS it must be possible for the user to run several test cases automatically through the DAHS to show that the installation has been successful.

B9.5 You must make the release notes available online. They must include a list of all:

cleared software defects
outstanding software defects
fixed data files and the provenance of their data
constants, for example physical constants and conversion factors (these must be displayed whenever requested)

You must make sure the major components, and any reference data of the DAHS are available to the user (for example, through the ‘Help’ – ‘About’ menu).

B10 Software security and prevention of malware attacks

B10.1 Prior to its installation you must show that the DAHS software and its associated files are virus free.

B10.2 If you supply computer hardware then you must show that all computers and their associated operating systems and middleware are virus free and have the capability of remaining so.

B10.3 If you supply computer hardware, all USB ports, and other means of introducing malware must be accessible to authorised personnel only.

B10.4 If you install a DAHS with Internet links you must provide protection and separation of networks.

Question 5

Part C of the performance requirements – standards for specific sectors

Accepted Answer

Here we specify sector specific requirements.

C1 Continuous emissions monitoring systems

This section outlines the performance standards for DAHS used in getting data from continuous emissions monitoring systems (CEMS) that must meet the requirements of EN 14181:2014 Stationary source emissions – Quality assurance of automated monitoring systems.

DAHS reporting must be able to be configured to meet applicable legislation required by the local competent authority. For example, the approach to reporting and measurement uncertainty may vary.

You will require the following standards for the testing of DAHS under MCERTS:

EN 17255-1:2019 Stationary source emissions – Data acquisition and handling systems – Part 1 specification of requirements for the handling and reporting of data
EN 17255-2:2020 Stationary source emissions – Part 2: Specification of requirements on data acquisition and handling systems
EN 17255-3: Stationary source emissions – Part 3: Specification requirements for the performance test of data acquisition and handling systems

EN 17255-1 includes an annex on data capping (annex C). Do not apply this annex C to MCERTS DAHS because data capping of peak measurement values must not take place.

EN 17255-2 specifies that you must keep data in permanent data storage for at least 5 calendar years. In the UK you must keep data fora minimum of 6 calendar years because this aligns with the time required for storage of information by sites regulated under the UK’s Environmental Permitting Regulations.

Where required, such as when switching between duty and standby CEMS, you may apply the QAL2 calibration function to the 1 minute first level data (FLD), rather than to the STA (short term average, for example 30-minute data). This means that fewer STAs will be invalidated. This approach is reflected in Note 2 of section 8.3 of EN 17255-1, which states that ‘Applying the QAL2 calibration function to the FLD first and then averaging to form the STA produces an equivalent result’. This applies to both the pollutant and peripheral measurements. The STA for each is used for the correction of the pollutant to reference conditions.

When determining mass emissions, you may apply the calibration function and conversion to standard conditions directly to the FLD (that is to 1-minute averages). This data set must not be used for compliance assessment with concentration-based emission limit values.

C2 Electronic verification tools for closed pipe electro-magnetic flowmeters

Operators, or their appointed contractors, use electronic verification tools to verify the operation of some electromagnetic flowmeters used to obtain regulatory flow data. The electromagnetic flowmeter manufacturers produce the verification tools, and they are specific to certain models within that manufacturer’s product range. You can obtain further details from the certification body.

Question 6

MCERTS certification process

Accepted Answer

Product certification is made up of the following phases:

part A audit, which you can replace by an equivalent
part B and C audit
review of the audit report
issue of the MCERTS certificate

The audit can cover all three parts of the standard in the same process.

Instruments that include a DAHS must still have their instrument functions certificated to the appropriate MCERTS standard.

Certification committee

The role of the certification body is to assess and certify compliance with the MCERTS standard.

An independent, competent person or group of persons referred to as the certification committee will take any certification decision based on technical judgement of the standard.

Auditing and surveillance

The certification body will conduct an audit of the DAHS development process to confirm that you have followed this standard.

Subsequent surveillance audits of the DAHS software development process are normally conducted every 5 years to make sure that the DAHS is being maintained appropriately.

Certificate validity

MCERTS certificates for the DAHS software are valid for 5 years. After this time, the certification body will review the certification and identify any necessary auditing to maintain the certification. They will carry out assessment for this recertification against the standards current at the time of recertification.

Modifications to certified DAHS software

You can modify certified DAHS software, provided you demonstrate to the certification body that these changes have the following characteristics:

they are minor changes, as defined in the standard
you adhere to the certified DAHS software development process
the cumulative effect of a sequence of minor software changes does not amount to a major change from the originally certified DAHS software

You must keep detailed records of all design changes to the DAHS and have provisions for design verification and review to make sure that the software still meets the required performance standards. You must make these records available for verification, so that the certification body can decide whether the scope of any changes is either major or minor.

See the requirements for a recertification audit and a surveillance audit later in this guide.

Question 7

Sample headings for a DAHS software quality plan

Accepted Answer

Here we give guidance on the information that you should include in the quality plan.

Typical headings are as follows.

Introduction

Summary of project or product.

List of deliverables.

Risk assessment

Including choice of the software quality standard used if Part A of this standard is not used.

Quality system

Statement as to whether a QMS exists, including a copy of any certificate for the company’s QMS, if one is available.

Project organisation and control

Roles and responsibilities for all those involved in the project, including subcontractors.

Outline plan and milestones.

Communications between members of the project.

Policy on recording decisions.

Policy on storing emails and other correspondence.

Policy for meetings minutes, recording of reviews and so on.

Change control.

Software configuration management.

Progress meetings.

Records and archiving.

Training requirements.

Software configuration management (SCM)

The use of software configuration management tools is mandatory for source code and strongly encouraged for documents. It is perfectly acceptable to use different methods of SCM for documents and source code.

The SQP must define how you organise the SCM for the project.

Software lifecycle

Selection and justification of the software lifecycle that you will use.

Selection and justification of the tools and methods you will use, including a list of all tools and components.

List of software standards you will use, including the coding standard.

Procedure for handling problems (software defect reports) and non-conformances.

Procedures for verification of the software, typically testing and reviews.

Backups and security.

List of reviews to be performed including:

independence matrix (table showing that authors do not review their own work)
identification of the more critical parts that require more detailed review

Design phase

Definition of the contents of each of the design documents, depending on the methods used:

list of prototype releases and their functionality
mathematical specification
software design documents
software acceptance test specification
software design verification report and code reviews

Delivery phase

Acceptance test policy, testing and sign off.

Software release policy – release documents, method of delivery.

Support or warranty commitments for the software.

Question 8

Information about the MCERTS recertification audit

Accepted Answer

Normally, an MCERTS certificate is valid for 5 years. Towards the end of that time, you must contact the certification body to arrange the recertification audit before they update the certificate for its next period of validity.

The audit will include a review of the original assessment report and the check list tables to upgrade the report so that it reflects the current state of the certified DAHS software and the latest version of the standard. This will include an update to the documentation baseline so that the latest titles and version numbers of the documents are in the baseline as a table in the MCERTS audit report.

There will be an examination and discussion of the release notes and fault history for the software, in particular:

assessment of the scope of the ‘minor’ changes made during the life of the certificate (certification of major changes takes place on their introduction)
analysis of any fault reports whose consequences could have led to the loss of data integrity and the production of incorrect reports

There will be a root cause analysis of any serious bugs or other incidents where data integrity was lost or impacted. Or, if you have already done the analysis, the assessor will examine the evidence that you conducted the root cause analysis effectively and that you have made any necessary improvements.

The audit will include a review of the latest source code metrics to ascertain:

the continued maintainability of the code
the extent of any changes both in the DAHS software code and major software components, and the tools used to produce it

There will be a check of your capability with respect to the ongoing development and maintenance of the software.

Question 9

Information about the MCERTS surveillance audit

Accepted Answer

The certification body will conduct annual surveillance audits to make sure ongoing compliance of the certified product. These will be remote surveillance audits (RSA). This is a questionnaire that you must complete and submit to the certification body within 4 weeks of the due date. Following satisfactory review, the certification body issue a summary report that will confirm ongoing compliance for the current year of the certification cycle.

Completion of annual surveillance audits will mean that the certifying body can reduce the scale of the recertification audit because of routine assessments over the certification period.

If a remote surveillance questionnaire is unsatisfactory then the certification body will follow up by videoconferencing or site visit to confirm ongoing compliance against the following points:

discussion of questionable areas in the initial questionnaire
determination of root cause analysis
evidence checks of root cause analysis
discussion and agreement of any necessary corrective actions
evidence checks of corrective actions
assuming a satisfactory outcome, issuing of a single page report confirming ongoing compliance
if there is an unsatisfactory outcome, the certification will be revoked, and the Environment Agency will be informed

Question 10

If you have questions

Accepted Answer

The Environment Agency appointed CSA Group Testing UK as the certification body to operate this scheme on their behalf. Here is their guide to the certification of products under the Environment Agency’s MCERTS scheme.

If you have any questions about the certification process, or would like further information on how to make an application, please contact:

CSA Group Testing UK Ltd

Telephone: +44 (0) 1244 670900

Email: mcerts@csagroup.org

If you have any comments about this publication, please contact our National Customer Contact Centre.

General enquiries

National Customer Contact Centre
PO Box 544
Rotherham
S60 1BY

Email enquiries@environment-agency.gov.uk

Telephone 03708 506 506

Telephone from outside the UK (Monday to Friday, 8am to 6pm GMT) +44 (0) 114 282 5312

Monday to Friday, 8am to 6pm.

Cookies on GOV.UK

Applies to England