Introduction
Get Metrics Version 4 and Version 5 add further granularity to metrics. The following notes clarify obligations and features of these versions.
Version 3
V3 obligations
Data Holders currently supporting Get Metrics V3 are required to continue supporting V3. There is no deprecation date, until a date to be set by the ACCC in a forthcoming urgent change to standards. New Data Holders can support V4 or V5 and are not required to support V3.
Version 4
V4 obligations
Support for Get Metrics V4 is required from 1st November 2023. V4 is a subset of V5, and can be deprecated as soon as V5 is supported.
V4 changes
Get Metrics V4 adds granularity to the NFRs. For example, reporting of site-wide Transactions Per Second (TPS) is broken down into authenticated and unauthenticated categories.
See:
V4 introduces new metrics, including AuthorisationMetrics. AuthorisationMetrics specifies several authorisation measures including abandonedConsentFlowCount, which captures how many consents have been started but not ended. Abandoned consent flows are counted for the course of the current day so far, and also for up to seven previous days.
The start of a consent flow, from the DH perspective, is the point when the ADR calls the DH authorise endpoint.
The end of a consent flow is the point at which the DH exchanges tokens with the ADR.
See:
- Get Metrics V4 AuthorisationMetrics
Version 5
V5 obligations
Support for Get Metrics V5 is required from 13th May 2024.
V5 changes
Get Metrics V5 contains all the changes in Get Metrics V4, and some additional metrics granularity.
In V5, AuthorisationMetricsV2 adds abandonmentsByStage to the abandonedConsentFlowCount. For each abandoned consent flow, authorisation metrics captures the stage of authorisation at which the consent was abandoned:
- preIdentification - consumer identification was not complete
- preAuthentication - identification complete but consumer authentication not complete
- preAccountSelection - authentication complete but the consumer did not complete selection of accounts
- preAuthorisation - accounts selected but the consumer did not complete authorisation of the consent
- failedTokenExchange - authorisation complete but tokens were not exchanged
For each abandoned consent, the count for only one stage is incremented. Accordingly the sum of counts for all phases is equal to the count for abandoned consents.
As with V4, counts for abandoned consents, and for all stages, are recorded for the current day so far, and for up to seven previous days.
See:
ErrorMetricsV2
In the ErrorMetricsV2 schema, the aggregate
field is a carry forward of the previous Get Metrics v3 errors field, so should include only server errors and not client errors.
The currentDay
object in ErrorMetricsV2 includes client and server errors, in an object format similar to the following:
"errors":
"aggregate": {
"currentDay": 305,
"previousDays": [245, 345, 543]
},
"authenticated": {
"currentDay": {
"500": 12,
"406": 83
},
...
}
Comments
2 comments
The example for ErrorMetricsV2 above suggests client errors (406) are expected to be included as part of the currentDay object whereas the description states "include only server errors and not client errors". Can this be clarified please?
Hi Abhideep Das
The description states that the `aggregate` field provides a continuation of data from the previous version and should only include server errors to match the previous requirement.
The new fields provided as `unauthenticated` and `authenticated` types should include all errors by error code, allowing client or server side to be distinguished.
A minor clarification to reflect this is expected to be made in the next version of the Standards, as noted in this Maintenance comment - https://github.com/ConsumerDataStandardsAustralia/standards-maintenance/issues/612#issuecomment-1822108820
Please sign in to leave a comment.