Basic guide for XML software developers
Information for developing and testing of software for HMRC forms, returns and messages.
Introduction
The Software Developer Support Team (SDST) provides support to software developers during development and testing of their product. Our role includes:
- publishing technical information packs
- updating developers of changes to the testing environment, informing of test service availability and providing testing credentials to use the test services
- giving appropriate advice and feedback
- helping developers complete the recognition process
- consulting on new services and proposed changes to existing services
- representing developers views, issues and requirements across HMRC
Register as a software developer
To use the test services you’ll need a Vendor ID and the relevant test credentials. You can get these by registering as a software developer.
Technical packs
The technical packs contain information needed to develop software. These documents are aimed at software developers only.
Software development - an overview
Development for online services is typically done in 3 stages:
1. Transaction Engine
The Transaction Engine handles the requests and responses for most of HMRC Services. It’s important to read the Transaction Engine: Document Submission Protocol before developing a product.
2. Submission (message body)
XML – online submissions need to be sent in XML (eXtensible Markup Language). The schema sets out the format, structure and data types that HMRC accepts for a particular online service and can be integrated in to your product to help validate submissions. The schemas are found in the relevant technical pack for the service you’re developing for.
Information on XML and schema can be found on w3.org.
Business Rules Validation (BVR) Document – most technical packs include some kind of business rules document. These specify the additional validation rules that are not captured by the schema, particularly where cross-field validation applies.
Rules and Interface Management (RIM) – the RIM process machine generates artefacts to ensure consistency. For RIM online services, the BVRs (schematron) can also be incorporated into your software alongside the schema to validate data prior to submission. However, not all services support RIM – you should check the technical pack for the online service that you are developing for.
Information on schematron can be found at xfront.com and schematron.com
3. The generic IRmark (or HMRCmark) technical pack
Legislation lays the burden of proof on the submitter of a form. An IRmark or HMRCmark for online internet submissions is calculated on the body of the message submitted. If there is dispute that the form received by HMRC is not the same as the one they sent, this will help the submitter prove this return or message.
HMRC performs an identical calculation on the body of the message received to check that the content is the same.
The IRmark and HMRCmark consists of:
- client-side generation and server-side validation at HMRC of a secure hash (IRmark / HMRCmark)
- generation of a digital receipt from HMRC that is returned to the submitter
The digital receipt confirms acceptance of the submission and the validity of the submitted IRmark / HMRCmark string. It also contains HMRC’s digital certificate so the signature can be verified by a third party, and a service specific message.
Test services
The test services vary depending on the online service. Usually there’ll be either:
- an online test service – Third Party Validation Service (TPVS)
- a downloadable offline testing tool – Local Test Service (LTS)
Technical packs for the service that you’re developing are available. Most technical packs have a ‘How to Use the Test Service’ document to get you started.
Offline testing tools - Local Test Service (LTS) and Desktop Checker (CIS only)
Some online services also provide an LTS or Desktop Checker. These are downloadable tools to allow you to test your software products offline and provide a similar function to TPVS. However, there is no connectivity to the External Test Service, so only the body of the message can be checked and not the Transaction Engine functionality.
Third Party Validation Service (TPVS)
Test directly to TPVS to validate the ‘Body’ of the text XML submissions only.
The URL for TPVS can be found in the relevant technical pack.
External Test Service (ETS)
The message handling and credentials authentication testing is performed by ETS.
When you submit XML to the Transaction Engine a ‘conversation’ should take place between your application and the Transaction Engine enabling your message to be routed to HMRC.
The submitter’s credentials (such as their user name, password, tax reference and so on) are also verified at this stage. This is detailed in the Transaction Engine: Document Submission Protocol Document.
ETS is a combination of the transaction engine test environment and TPVS and mimics, as far as possible, full end-to-end testing. ETS validates the GovTalk Header within the XML against the GovTalk schema, verifies the test credentials and handles the messaging process. The test submission is automatically routed to the TPVS service where the <Body> of the XML submission will be validated against the schema and the business rules or Quality Standard. A response message (pass or fail) will then be returned as part of the messaging process.
The URL for submissions to the ETS service is:
https://test-transaction-engine.tax.service.gov.uk/submission
The URL for the Poll messages to the ETS service is:
https://test-transaction-engine.tax.service.gov.uk/poll
Test in Live (TIL)
For some online services you can send a test submission to the ‘live’ environment. To send a ‘Test in Live’ submission you should use the appropriate < Class>, for example for the Corporation Tax online service this would be HMRC-CT-CT600-TIL.
This will perform a validation check on the return including the user’s live credentials, but no data will be sent to the HMRC back-end systems.
See Appendix A for more information on what’s included for TIL and live submissions.
Software recognition
Once you have successfully completed your own testing you can apply for HMRC recognition for your product.
Details of products that have successfully completed the recognition process will be published on GOV.UK.
The recognition process involves successfully submitting a number of test scenarios provided by the SDS team to the appropriate test service that you are developing for. You’ll need to send the test file XML outputs to the SDS team. We’ll review the files and advice appropriately.
Each technical pack provides full details of the recognition process.
We aim to complete (where possible) recognition applications within 10 working days of receipt of your documentation.
Our service commitment
Contact the SDS team by email: [email protected].
Where possible we aim to respond to your query in full within 2 working days. However, there may be occasions where we need to seek guidance from our business or IT partners in answering your query.
You can find out more about general customer and user enquiries relating to the live internet services from the online services helpdesk.
Information on the availability of live HMRC services is published at HMRC service availability and issues.
You may wish to receive the latest HMRC updates via email alerts or your feed reader. Please use the links on the HMRC page on GOV.UK to do this.
Test service support
The test services are supported Monday to Friday, 9am to 5pm.
It may be necessary to close the test and live services to allow updates to existing and introduce new services. Where possible these updates will take place outside of normal working hours.
Appendix A
Submission Information for Test, Live and Test in Live environments
Software Developer test environments: TPVS / ETS
| Message | To test your product’s Test in Live’ functionality prior to release (only if the online service or your software supports ‘Test in Live’ functionality for the End User) | To test your product’s ‘Live’ functionality prior to release |
|---|---|---|
| <Class> | Include the ‘-TIL’ suffix | Without ‘-TIL’ suffix |
| <GatewayTest> | 1 (or omit this element) | 1 (or omit this element) |
| <SenderID> | Test credentials supplied to developer by SDST | Test credentials supplied to developer by SDST |
| <Value> | Test credentials supplied to developer by SDST | Test credentials supplied to developer by SDST |
| URL endpoint | https://test-transaction-engine.tax.service.gov.uk/submission | https://test-transaction-engine.tax.service.gov.uk/submission |
End Users: Transaction Engine live (production) environment
| Message | For End User ‘Test in Live’ submissions (only if the online service or your software supports ‘Test in Live’ functionality for the End User) | For genuine Live submissions to HMRC |
|---|---|---|
| <Class> | Include the ‘-TIL’ suffix | Without ‘-TIL’ suffix |
| <GatewayTest> | 0 (or omit this element) | 0 (or omit this element) |
| <SenderID> | The ‘UserName’ obtained by End User when registered for HMRC online services | The ‘UserName’ obtained by End User when registered for HMRC online services |
| <Value> | The ‘Password’ obtained by End User when registered for HMRC online services | The ‘Password’ obtained by End User when registered for HMRC online services |
| URL endpoint | https://transaction-engine.tax.service.gov.uk/submission | https://transaction-engine.tax.service.gov.uk/submission |