Conformance Test Suite: version history and scenarios

The following tables list each version of the Conformance Test Suite (CTS) commencing at v2.0.0 providing detail on the version of the Consumer Data Standards and CDR Register Design Reference it aligns with, the release date, the test scenarios included, and the high level scenario changes between versions.  When new test scenarios are added, they are added to the existing list and identified with an asterisk.

More information on these scenarios is available in the CTS Guidance Materials article.

CTS Version: 3.4.0

Released on: 09/09/2021

CDS	Register Design	# Scenarios	Version changes
1.10.0	1.5.0	Data Holders - 17 scenarios 1.     Discovery Document Validation 2.      Dynamic Client Registration (DCR) 3.      Concurrent Consent 4.      DH Initiated Revocation 5.      DR Initiated Revocation 6.      Removed Software Product 7.      Reactivate Software Product 8.      Replace Existing Consent with PAR Scenario 9.      DR Initiate Token Revocation 10.   Register PUT GET 11.   Get Software Product Status Register Polling 12.   Get Data Recipients Register Polling 13.   Ensure Client Assertion Data In Token Request 14.   Amending Account for An Existing Consent Scenario with PAR 15.   Ensure Holder of Key 16.   Ensure Infosec Endpoints Using MTLS Authentication with X509 17.   Consent Software Statement Assertion with Sector Identifier uri	Released on 09-Sep-2021 No New Scenarios CTS Test Scenarios have been updated to validate the usage of standardised Error Codes, where Participants have started adopting the Standardised Error Handling requirements, as per CDS 1.10+. Where the Participant has adopted the Standardised Error Handling Requirements, CTS will validate that their responses for specific CTS induced failure conditions (e.g. invalid Software Product, Invalid Consent etc.) conform to CDS 1.10+ requirements - which includes validation of HTTP Status Codes, Error schema and the usage of correct Standard Error code itself. Where Participants have not started using the standardised Error codes in their responses, the CTS validation will revert to the existing logic as per Test Plan 3.3.
		Data Recipients - 9 scenarios 1.     Dynamic Client Registration (DCR) 2.      Once-Off Consent (Get Accounts) 3.      Ongoing Consent (Get Accounts) 4.      Once-Off Consent (Get Transactions) 5.      Ongoing Consent (Get Transactions) 6.      Revoke Consent Arrangement (DR to DH) 7.      DH Initiated Revoke Consent Arrangement (DH to DR) 8.      Token Revocation (DR to DH) 9.      PAR Extend Consent	Released on 09-Sep-2021 No New Scenarios CTS Test Scenarios have been updated to validate the usage of standardised Error Codes, where Participants have adopted  the Standardised Error Handling requirements, as per CDS 1.10+. Where the Participant has adopted the Standardised Error Handling Requirements, CTS will validate that their responses for specific CTS induced failure conditions (e.g. InvalidArrangement etc.) conform to CDS 1.10+ requirements - which includes validation of HTTP Status Codes, Error schema and the usage of correct Standard Error code itself. Where Participants have not started using the standardised Error codes in their responses, the CTS validation will revert to the existing logic as per Test Plan 3.3.

CTS Version: 3.3.0

Released on: 30/07/2021

CDS	Register Design	# Scenarios	Version changes
1.10.0	1.5.0	Data Holders - 17 scenarios Discovery Document Validation Dynamic Client Registration (DCR) Concurrent Consent DH Initiated Revocation DR Initiated Revocation Removed Software Product Reactivate Software Product Replace Existing Consent with PAR Scenario DR Initiate Token Revocation Register PUT GET Get Software Product Status Register Polling Get Data Recipients Register Polling Ensure Client Assertion Data In Token Request Amending Account for An Existing Consent Scenario with PAR Ensure Holder of Key Ensure Infosec Endpoints Using MTLS Authentication with X509 Consent Software Statement Assertion with Sector Identifier uri	No new scenarios Each scenario has been modified to support November 2021 obligations for Phase 2 for non-major ADIs, including registration requests, which will be made using phase 1 and 2 scopes: openid   profile  bank:accounts.basic:read  bank:accounts.detail:read  bank:transactions:read  bank:regular_payments:read  bank:payees:read  common:customer.basic:read  common:customer.detail:read  cdr:registration  The CTS will confirm the non-disclosure of phase 2 consumer data* to inactive software products The CTS will confirm the successful disclosure of phase 2 consumer data* to an authorised client The CTS will accept CDR Standardised Errors for early adopters, but will not validate their contents. The CTS still supports application-specific errors *Phase 2 consumer data will be sourced from the following end points: Get Account Detail, Get Direct Debits for Account, Get Bulk Direct Debits, Get Direct Debits for Specific Accounts, Get Scheduled Payments for Account, Get Scheduled Payments Bulk, Get Scheduled Payments for Specific Accounts, Get Payees, Get Payee Detail, Get Customer Detail
		Data Recipients - 9 scenarios Dynamic Client Registration (DCR) Once-Off Consent (Get Accounts) Ongoing Consent (Get Accounts) Once-Off Consent (Get Transactions) Ongoing Consent (Get Transactions) Revoke Consent Arrangement (DR to DH) DH Initiated Revoke Consent Arrangement (DH to DR) Token Revocation (DR to DH) PAR Extend Consent	No new scenarios Registration can be made with the CTS DH with phase 2 scopes (noting that the CTS DH doesn’t support most resource end points at this stage) The CTS will accept CDR Standardised Errors for early adopters, but will not validate their contents. The CTS still supports application-specific errors New ADR Flexible Testing beta test plan – contact [e-mail] to express your interest in participating in the beta, or for more information

CTS Version: 3.2.0

Released on: 17/06/2021-01/07/2021¹

CDS	Register Design	# Scenarios	Version changes
1.8.0	1.5.0	Data Holders - 17 scenarios Discovery Document Validation Dynamic Client Registration (DCR) Concurrent Consent DH Initiated Revocation DR Initiated Revocation Removed Software Product Reactivate Software Product Replace Existing Consent with PAR Scenario DR Initiate Token Revocation Register PUT GET Get Software Product Status Register Polling Get Data Recipients Register Polling Ensure Client Assertion Data In Token Request Amending Account for An Existing Consent Scenario with PAR Ensure Holder of Key Ensure Infosec Endpoints Using MTLS Authentication with X509 Consent Software Statement Assertion with Sector Identifier uri	1Released on 17/06/2021 No new scenarios. Each scenario has been modified to be compliant with the following two Register design changes described in Release notes v1.5.0: Added the definition that profile is part of the registration SSA scope Added the definition that software_roles is now optionally part of the registration response Testing related to the “request_object_signing_alg” may be considered at a future date. One user to many brands The CTS implemented a feature which will allow users with the ‘Authorised CTS Tester’ role in the RAAP to access all CTS instances under their organisation(s) e.g. brands. When the user logs into the CTS participant UI, they will be able to pick a brand or software product from the CTS user interface, and they will be navigated to that instance. 2
		Data Recipients - 9 scenarios Dynamic Client Registration (DCR) Once-Off Consent (Get Accounts) Ongoing Consent (Get Accounts) Once-Off Consent (Get Transactions) Ongoing Consent (Get Transactions) Revoke Consent Arrangement (DR to DH) DH Initiated Revoke Consent Arrangement (DH to DR) Token Revocation (DR to DH) PAR Extend Consent	1Released on 01/07/2021 No additional scenarios Modifications have been made to be compliant with the following two Register design changes described in Release notes v1.5.0: Added “profile” to the registration SSA response Added “profile” and “request_object_signing_alg” to the registration response provided from CTS back to the ADR One user to many software products The CTS implemented a feature which will allow users with the ‘Authorised CTS Tester’ role in the RAAP to access all CTS instances under their organisation(s) e.g. brands. When the user logs into the CTS participant UI, they will be able to pick a brand or software product from the CTS user interface, and they will be navigated to that instance. 2

² NOTE: the 'Authorised CTS Tester' role will have access to all CTS instances for that organisation, and any other organisations they are assigned to. Access is not limited by brand. This aligns to the user roles and permissions matrix available in the participant portal user guide.

CTS Version: 3.1.0 - OBSOLETE

Released on: 29/04/2021

CDS	Register Design	# Scenarios	Version changes
1.8.0	1.4.0	Data Holders - 17 scenarios Discovery Document Validation Dynamic Client Registration (DCR) Concurrent Consent DH Initiated Revocation DR Initiated Revocation Removed Software Product Reactivate Software Product Replace Existing Consent with PAR Scenario DR Initiate Token Revocation Register PUT GET Get Software Product Status Register Polling Get Data Recipients Register Polling Ensure Client Assertion Data In Token Request Amending Account for An Existing Consent Scenario with PAR Ensure Holder of Key* Ensure Infosec Endpoints Using MTLS Authentication with X509* Consent Software Statement Assertion with Sector Identifier uri*	Three new scenarios*
		Data Recipients - 9 scenarios Dynamic Client Registration (DCR) Once-Off Consent (Get Accounts) Ongoing Consent (Get Accounts) Once-Off Consent (Get Transactions) Ongoing Consent (Get Transactions) Revoke Consent Arrangement (DR to DH) DH Initiated Revoke Consent Arrangement (DH to DR) Token Revocation (DR to DH) PAR Extend Consent	No additional scenarios Support for Sector Identifier uri (optional field from Register design)

CTS Version: 3.0.0 - OBSOLETE

Released on: 08/04/2021

CDS	Register Design	# Scenarios	Version changes
1.7.0	1.3.0	Data Holders - 14 scenarios Discovery Document Validation Dynamic Client Registration (DCR) Concurrent Consent DH Initiated Revocation DR Initiated Revocation Removed Software Product Reactivate Software Product Replace Existing Consent with PAR Scenario DR Initiate Token Revocation Register PUT GET Get Software Product Status Register Polling Get Data Recipients Register Polling Ensure Client Assertion Data In Token Request*^ Amending Account for an Existing Consent Scenario with PAR*	Two new scenarios* ^Conducts suite of bad requests against the token endpoint, to test error returned from DH
		Data Recipients - 9 scenarios Dynamic Client Registration (DCR) Once-Off Consent (Get Accounts) Ongoing Consent (Get Accounts) Once-Off Consent (Get Transactions) Ongoing Consent (Get Transactions) Revoke Consent Arrangement (DR to DH) DH Initiated Revoke Consent Arrangement (DH to DR) Token Revocation (DR to DH) PAR Extend Consent	No additional scenarios

CTS Version: 2.1.0 - OBSOLETE

Released on: 18/03/2021

CDS	Register Design	# Scenarios	Version changes
1.7.0	1.3.0	Data Holders - 12 scenarios Discovery Document Validation Dynamic Client Registration (DCR) Concurrent Consent DH Initiated Revocation DR Initiated Revocation Removed Software Product Reactivate Software Product Replace Existing Consent with PAR Token Revocation Register PUT GET Get Software Product Status Register Polling Get Data Recipients Register Polling	No new scenarios However support for Register design 1.3.0 (new optional fields in the registration request, plus additional new OpenID scope)
		Data Recipients - 9 scenarios Dynamic Client Registration (DCR) Once-Off Consent (Get Accounts) Ongoing Consent (Get Accounts) Once-Off Consent (Get Transactions) Ongoing Consent (Get Transactions) Revoke Consent Arrangement (DR to DH) DH Initiated Revoke Consent Arrangement (DH to DR) Token Revocation (DR to DH) PAR Extend Consent	No new scenarios However support for Register design 1.3.0 (new optional fields in the registration request, plus additional new OpenID scope)

CTS Version: 2.0.0 - OBSOLETE

Released on: 18/02/2021

CDS	Register Design	# Scenarios	Version changes
1.6.0	1.2.3	Data Holders - 12 scenarios Discovery Document Validation Dynamic Client Registration (DCR) Concurrent Consent DH Initiated Revocation DR Initiated Revocation Removed Software Product Reactivate Software Product Replace Existing Consent with PAR Token Revocation Register PUT GET Get Software Product Status Register Polling Get Data Recipients Register Polling	Initial release for Non-Major ADIs
		Data Recipients - 9 scenarios Dynamic Client Registration (DCR) Once-Off Consent (Get Accounts) Ongoing Consent (Get Accounts) Once-Off Consent (Get Transactions) Ongoing Consent (Get Transactions) Revoke Consent Arrangement (DR to DH) DH Initiated Revoke Consent Arrangement (DH to DR) Token Revocation (DR to DH) PAR Extend Consent	Initial release