This knowledge article has been prepared to address questions relating to the Get Metrics API within the Consumer Data Right regime.
Background on Get Metrics API
The Get Metrics API as defined by the Consumer Data Standards is a binding data standard that allows the ACCC to obtain operational statistics from Data Holders on the operation of their CDR implementation. As such, Data Holders are required to make the Get Metrics API available for the CDR Register to call and must ensure the data returned in response to this API call is complete and accurate.
Some of the statistics obtainable from Get Metrics are determined by the non-functional requirements (NFRs) of the CDR regime. The NFRs are not yet binding. It is intended that analysis of actual usage in production is conducted to validate the specified NFRs are appropriate prior to them becoming a binding data standard.
The ACCC intends to use the current Get Metrics dataset to monitor adoption of the CDR and interaction with the CDR ecosystem, such as the number of customers and accredited data recipients with active authorisations. The ACCC also intends to use the Get Metrics data to monitor and assess the performance of Data Holders in the ecosystem. This includes monitoring average response times and the availability of the Data Holders’ systems.
Frequently Asked Questions
What version of Get Metrics should non-major ADIs be implementing?
V2. The ACCC and DSB recognise that V2 of the Get Metrics API must be supported from no later than 31 July 2021. We also appreciate that this change to the data standards will occur only four weeks after non-major ADIs are required to implement the Get Metrics API under the data standards (i.e. 1 July 2021), and that there are additional build costs in seeking to implement V1 of the Get Metrics API for this short amount of time.
As the CDR Register is expected to support V2 of the Get Metrics API by the end of the first quarter of 2021, we accept that it is reasonable for non-major ADIs to solely implement V2 of the Get Metrics API from the 1 July 2021 compliance date.
What time zone should the metrics be collected in?
It is important that the metrics published on the Get Metrics end point map to consistent dates/times for each Data Holder. Metrics are expected to represent the date and time of the Data Holder’s organisation, irrespective of the locations of their servers.
How accurate does the data need to be?
The ACCC expects Data Holders to populate the Get Metrics endpoints with data which is as accurate as possible. However it is understood that outages may affect accuracy of the data, therefore tolerances of +/- 1-2% are deemed acceptable.
How often will the ACCC be collecting this data?
The CDR Register currently calls the Data Holders’ Get Metrics endpoints at 5:00AM AEST (or AEDT in the summer months) on a daily basis, This enables any gaps in the data, which may have occurred due to Data Holder availability, to be corrected. Metrics published on the Get Metrics end point are expected to represent the latest values and any previously retrieved values will be overwritten. This enables Data Holders to aggregate their data more than once a day removing any concern that overwritten values may have already been collected.
How old can the data be?
The ACCC understands metrics data may be distributed between different systems in a Data Holder’s implementation and that collection and consolidation may be a batch process. This type of process makes it difficult to guarantee the data collected represents the latest and most up to date without detailed coordination of collection. Therefore, it is deemed acceptable for the data presented to be up to 24 hours old.
What happens if there is a gap in our data due to outages?
The ACCC appreciates that system outages may create gaps in data collection. As mentioned in “How often will the ACCC be collecting this data?” any gaps in the data over the previous 7 days can be corrected as each daily call collects one week of data. The ACCC accepts that gaps may occur by exception and will investigate gaps which appear to be systemic.
Will we be portrayed negatively if low consumer data sharing utilisation skews our performance metrics?
The ACCC understands there are circumstances that may skew performance data and will discuss results with relevant Data Holders should there be any concerns.
Is there a plan to make API performance metrics publicly available?
The ACCC does not have any immediate plans to publish API performance metrics. In the future, we may look at making some of the Get Metrics data publicly available. This may include publishing aggregated statistics about overall Data Holder performance in the system as well as data illustrating the take up and usage of the CDR more broadly.
Is latency between the client and the server part of the performance metrics?
How should metrics be published against white-label brands?
Metrics collection by the CDR Register will be measuring how brands perform within the Consumer Data Right ecosystem irrespective of their white label relationship. Therefore, white-label service providers will need to provide separate GetMetrics endpoints for all brands they service.