Guidance on Get Metrics V3, V4 and V5

Introduction

Get Metrics Version 4 and Version 5 add further granularity to metrics. The following notes clarify obligations and features of these versions.

Version 3

V3 obligations

Data Holders currently supporting Get Metrics V3 are required to continue supporting V3. There is no deprecation date, until a date to be set by the ACCC in a forthcoming urgent change to standards. New Data Holders can support V4 or V5 and are not required to support V3.

Version 4

V4 obligations

Support for Get Metrics V4 is required from 1st November 2023. V4 is a subset of V5, and can be deprecated as soon as V5 is supported. 

V4 changes

Get Metrics V4 adds granularity to the NFRs. For example, reporting of site-wide Transactions Per Second (TPS) is broken down into authenticated and unauthenticated categories. 
See:

• AverageTPSMetricsV2
• PeakTPSMetricsV2

V4 introduces new metrics, including AuthorisationMetrics. AuthorisationMetrics specifies several authorisation measures including abandonedConsentFlowCount, which captures how many consents have been started but not ended. Abandoned consent flows are counted for the course of the current day so far, and also for up to seven previous days. 
The start of a consent flow, from the DH perspective, is the point when the ADR calls the DH authorise endpoint. 
The end of a consent flow is the point at which the DH exchanges tokens with the ADR.

See:

• Get Metrics V4 AuthorisationMetrics

Version 5

V5 obligations

Support for Get Metrics V5 is required from 13th May 2024. 

V5 changes

Get Metrics V5 contains all the changes in Get Metrics V4, and some additional metrics granularity. 

In V5, AuthorisationMetricsV2 adds abandonmentsByStage to the abandonedConsentFlowCount. For each abandoned consent flow, authorisation metrics captures the stage of authorisation at which the consent was abandoned:

• preIdentification - consumer identification was not complete
• preAuthentication - identification complete but consumer authentication not complete
• preAccountSelection - authentication complete but the consumer did not complete selection of accounts
• preAuthorisation - accounts selected but the consumer did not complete authorisation of the consent
• failedTokenExchange -  authorisation complete but tokens were not exchanged

For each abandoned consent, the count for only one stage is incremented. Accordingly the sum of counts for all phases is equal to the count for abandoned consents. 
As with V4, counts for abandoned consents, and for all stages, are recorded for the current day so far, and for up to seven previous days. 

See:

• AuthorisationMetricsV2

ErrorMetricsV2

In the ErrorMetricsV2 schema, the aggregate field is a carry forward of the previous Get Metrics v3 errors field, so should include only server errors and not client errors.

The currentDay object in ErrorMetricsV2 includes client and server errors, in an object format similar to the following:

"errors":
    "aggregate": {
        "currentDay": 305,
        "previousDays": [245, 345, 543]
    },
    "authenticated": {
        "currentDay": {
            "500": 12,
            "406": 83
         },
    ...
    }