May 31, 2021
Consumer Data Right (CDR) — Lessons Learned from a Data Holder’s Conformance Test Suite (CTS) experience -Part 1
Manglu Balasubramanian
This article is part of the series of articles on Consumer Data Right.
Note: Refer to the CDR Series section (at the bottom of this page) for links to other articles that are part of this series.
During the course of the last 12+ months , the CDR Support team has published a number of knowledge articles. The articles about the Production Incidents from the July 2020 release allowed us (the non-major ADIs ) to learn from the experiences of the CDR Pioneers last year.
The goal of this article series (Lessons learned from CTS experience) is to provide similar insights in the Conformance Test Suite (CTS) space with the hope of paving a way forward to an awesome CTS experience for the future Providers in the CDR ecosystem.
1st July 2021 is a key milestone date for the CDR ecosystem. Currently 5 banks (the Big 4 Banks and Regional Australia Bank) are the only active Data holders in the CDR regime. From the 1st of July 2021, all the Non-major Authorized Deposit-taking Institutions (ADI) are expected to become an active Data holder (at least for the Phase 1 products). Refer to the CDR Phasing table below for the planned rollout dates.
Before a CDR Participant (will refer to them as Provider in this article) can become active in the CDR ecosystem, they must successfully pass the Conformance Test Suite (CTS). The non-major ADIs have been working on their CTS test executions during the last few weeks/months.
What is CTS?
CTS is a system that simulates the CDR ecosystem. Two versions of the CTS are available for participants to connect to, depending on the nature of their participation in the ecosystem, i.e. whether they are an ADR or a Data Holder.
This article series focusses on the lessons learned from our CTS experience as a Data Holder.
What role does CTS play?
CTS aims to give consumers confidence that participants:
- can enter the Consumer Data Right (CDR) ecosystem without disruption to other participants; and
- are capable of conforming with the Consumer Data Standards and CDR Register Design.
There are a number of articles that provide information about CTS. The readers are advised to look at the references section for articles that provide detailed information about CTS, it’s role, benefits, and it how it fits in the on-boarding process.
There are several test scenarios that are executed during the CTS phase, and this would vary from one Data Holder to another depending on the phase that a Data Holder supports. For e.g., Non-major ADI Data Holder may approach the 1st of July 2021 to support only Phase 1 Products while another Data Holder may be willing to go early with support for Phase 2 Products.
Note: This article is not meant to be a criticism on anyone (individuals and/or organizations). The primary intent is to highlight areas that we as a CDR community can work on to improve the overall CTS experience for all the parties involved.
Lesson #1: Avoid assumptions (and conjectures)
The CTS Connection Data Sheet document provides the information on CTS Register URLs which needs to be configured in our environment as a preparatory step for CTS test executions. The URLs (shown below) have a conformanceId which the providers receive as part of the CTS enrolment activity
https://api.cts.cdr.gov.au/cts/{conformanceId}/register/cdr-register/v1/banking/data-recipients
https://api.cts.cdr.gov.au/cts/{conformanceId}/register/.well-known/openid-configuration/jwks
A natural thing for us to do before embedding any config information in our solution is to see if these URLs are good/functional.
Our initial impression was that we had an incorrect “Conformance Id” as the error response did not make sense. The HTTP Status code 400 typically represents a bad request (a client-side error).
We tried to see what happens if we use any random UUID value for the conformance ID in the URL to see if the behaviour is any different. We tried a few different UUIDs, and we always got a HTTP 200 OK response with a valid JSON Payload (shown in the figures below)
This behaviour left us absolutely confused. Any UUID value for the conformance ID except the one given by the CTS team returned a meaningful response!
The results of a using a non UUID value for Conformance Id is shown in Figure 6. It does appear to perform some validation to check that the conformanceId specified in the URL is an UUID.
What could we have done better ?
To be honest , there is very little we could have done better. I would always recommend doing a sanity/shake out test on any information before we start using them.
The best recourse is to seek clarity from the CTS team on the expected responses for these URLs and how the URLs are meant to be used.
What could the CTS Team have done better ?
(a) CTS team should have communicated this information to the CDR Providers that accessing the URLs outside of an active scenario will not return a valid response.
(b) CTS team should return an error response if a random UUID value is used for the conformance ID in these URLs.
Lesson #2: Check (and recheck) your configuration
Numerous studies have shown that Configuration errors are among the dominant causes of system failures.
We failed the very first test scenario and there were a few reasons behind this.
There were a couple of configuration error(s) one of which is shown below:
The InfosecBaseUri was configured as https://cts.infosec.api.dataholder.com.au/
Our solution (in the Authorization Server) used the “Context root” of identity.
Looking back, it is easy to state that the right configuration (during the CTS enrolment ) should have been: https://cts.infosec.api.dataholder.com.au/identity
The internal testing used the complete URL and as a result enough attention to detail was not paid while filling up the CTS enrolment form.
It is important to note that this configuration error would be visible only when all the pieces come together (CDR Register, ADR Software Product and a Data Holder Solution). Testing the Data Holder in isolation would not surface this issue in an earlier test phase.
What could we have done better ?
(a) We took the effort to build a sample ADR app to test our Data Holder solution however we did take a few shortcuts (in this example: Using the complete URLs of our Data Holder Solution) and caught this configuration issue during the CTS Phase.
The quote “Every shortcut has a price usually greater than the reward.” resonated with the team when we encountered this error during our CTS execution.
What could the CTS Team have done better ?
(a) CTS Portal should display the key configuration used for the tests (in this case all BaseURIs) . We should not be reliant on other sources such as the enrolment form. We raised the change to the InfoSecBaseURI via an email request to the CTS team resulting in the config information now fragmented in multiple places (email and the form) . The CTS team has visibility to this configuration however the provider’s team do not have a single simplified view of this configuration (BaseURIs).
(b) The CTS Portal should show the request and response messages and supplementary verbose information from their log files (related to our CTS executions). This allows the provider teams to troubleshoot things quickly. Logs are a foundational component of the debugging and troubleshooting process. Having access to only our log files without any insight to the other side (CTS) can only increase the defect resolution time.
Lesson #3: Additional Planning and Preparation for CTS
Standard planning is adequate when we are building and testing “run of the mill” solutions. The CDR solution certainly does not fall in this category. It is complex and relatively new. CTS is an even newer proposition.
The standard SDLC test phases (such as SIT, UAT etc.) typically involves our delivery teams (employees, contractors, and our partners). The teams have likely worked together for months and years and have a well-established working rhythm and process. As a result, our prior experiences gives us a predictable and meaningful plan and approach to execute these test phases.
The CTS test phase, on the other hand, is a different ball game all together. CTS is being executed for the first time (by the Providers) and a successful outcome is heavily dependent on good collaboration and communication between the Provider teams and the CTS teams.
Planning a time frame for CTS is an interesting and challenging exercise.
Is it a week, 2 weeks, 6 weeks, or couple of months-long activity?
Do we need a regular cadence with the CTS team (such as online meetings x number of hours a week) to augment the support process that is put in place by CTS?
Have the other Providers shared their CTS experience so far with the community (in a private or public forum)?
Has the CTS team shared any lessons from their interactions with the providers who have been successful with their CTS executions?
These are some of the questions that were in our heads, however we did not find clear answers. The team planned for a 2–3 weeks CTS execution and hoped for the best.
When we started our CTS phase, the team did not have accounts created in the Service Management Portal which is used to raise support tickets for assistance and troubleshooting. The ticketing systems are naturally a far better medium than emails.
What could we have done better ?
(a) Ensured that our entry criteria for CTS included the verification of the support process and steps established by the CTS team (in this instance, access for the team to use the CTS Service Management Portal)
(b) We should have reached out to the CTS team for dedicated timeslots for troubleshooting that is pre-arranged say 30 mins sessions every 2 days to supplement the emails and ticket-based support process.
What could the CTS Team have done better ?
(a) CTS team should publish the time taken (e.g., 3–4 weeks) by both Data Holders and Data Receipients from start to a successful finish of their CTS activities. This helps the other providers with their planning activity.
(b) CTS team should have published the typical errors and incidents from their experiences with other providers as knowledge sharing articles, guides, or tutorials. This would reduce the workload as well as time taken to come through the CTS phase successfully.
(c) The CTS team should have streamlined the access to the Service Management Portal and ensured that the teams had accounts set up and validated before the Providers are allowed to execute their CTS scenarios.
Conclusion
There’s no use talking about the problem unless you talk about the solution
In the true spirit of the quote above (from Betty Williams) this CTS article series focusses on both the problems and potential solutions (suggestions/ recommendations where an obvious solutions doesn’t exist) for an improved CTS experience for the CDR ecosystem players.
I hope this helps other providers with inputs to plan and better execute their CTS activities.
I would be keen to hear the experiences (not just the lessons) of the other Data Holders and Data Recipients who are going through (or have successfully been through) their CTS test executions.
Acknowledgements
Sincere thanks to my colleagues who are actively involved with our CDR solution.
I would like to thank the ACCC CTS team for their support and assistance during our CTS activities.
What’s next?
The next article in this CTS series will share more lessons from our Data Holder CTS experience. See you soon!!
Glossary:
Authorised deposit-taking institution (ADI): A financial institution licensed by the Australian Prudential Regulatory Authority (APRA) to carry on banking business, including accepting deposits from the public.
Big 4 Banks: The four major banks in Australia :
Australia and New Zealand Banking Group Limited (ANZ); Commonwealth Bank of Australia (CBA); National Australia Bank Limited (NAB); Westpac Banking Corporation (Westpac). They are also referred to as Major ADIs as well as Initial Data Holders in the CDR ecosystem.
CDR Participant: Is a data holder, or an accredited data recipient, of the CDR data
Non-major ADIs : ADIs other than the Major ADIs.
Providers : Generic term that is used to refer to both Data Holders and Data Recipients. The CDR documentation also uses the term “CDR Participant” to refer to Providers.
SDLC: Software Development Life Cycle.
SIT: System Integration Test.
UAT: User Acceptance Test.
UUID: Universally Unique IDentifier. They are also known as GUID(Globally Unique IDentifier).
References:
- CDR Conformance Test Suite
- Conformance Test Suite Update (Oct 2020)
- Official CDR Website
- CDR Support Portal
- July 2020 CDR Ecosystem Production Incidents
- CDR Participant Test Strategy
- The Conformance Test Suite (CTS) of Australian Consumer Data Rights (CDR) ecosystem
- An Empirical Study on Configuration Errors in Commercial and Open Source Systems
