Drinking Water Inspectorate Logo

DWI/OFWAT
Joint Serviceability Study

Review of Drinking Water Quality Aspects of Serviceability
Relating to Infrastructure and Non-Infrastructure Assets

 

Phase 2 - Report

 

May 2001

 

 

CONTENTS

 

FOREWORD

SUMMARY

1: INTRODUCTION

2: FORM OF SERVICEABILITY MEASURES

3: INFORMATION REVIEW

4: CURRENT SERVICEABILITY MEASURES

5: SPATIAL DEFINITIONS

6: STATISTICAL ANALYSIS

7: RISK

8: THE IMPACT OF COMPANY ACTIVITY

9: CONCLUSIONS

10: POSSIBLE ARCHETYPES FOR SERVICEABILITY MEASURES

11: COMBINING MEASURES

Appendix A

Appendix B

Appendix C

Appendix D

FOREWORD

We are pleased to publish this report of the jointly commissioned work on serviceability in relation to water supply. The report by W S Atkins covers the Phase 2 work of a review of drinking water aspects of serviceability. The objectives of the Phase 2 work were:

We are seeking an approach to serviceability indicators that can be developed on an evolving basis as better information on the performance and operation of assets becomes available. For each Periodic Review we seek an improved set of serviceability indicators whose historic trends will demonstrate with more confidence whether the systems are being maintained fit for purpose in a sustainable way.

We wish to establish how much can be achieved in the development of indicators in time for use in the next Periodic Review in 2004. Over the coming months a number of water companies will be working with us on the development of promising approaches, using real asset performance and operational data. The next stage will be completed this coming autumn so that water companies can report on any new indicators in the 2002 June returns.

We wish the process to be open and welcome constructive input at each stage, beginning with comments on the work to date. We are also pleased that OFWAT and DWI are involved in the separate study being carried out collectively by the water companies and wish to see that approach as complementary to our project.

We expect to be able to publish a report on Phase 3 of our study towards the end of this calendar year.

DWI and Ofwat will continue to work closely together on these issues to inform the next and future periodic reviews.

Philip Fletcher
Director General, OFWAT		 Michael Rouse
Chief Inspector, DWI

 

SUMMARY

This document is a report on the second phase of an assignment carried out for the Drinking Water Inspectorate and Ofwat by WS Atkins. The assignment comprised a review of the drinking water aspects of serviceability relating to infrastructure and non-infrastructure assets.

The report is a consultation document. It addresses some of the issues surrounding serviceability and identifies a basket of possible serviceability measures, some of which could be implemented in time for the next periodic review. The views and ideas set out in the document are not necessarily those of the Regulators. The ‘conclusions’ have been reached by WS Atkins alone.

The study which has lead to this report was restricted to water supply and does not touch on sewerage. However, the principles and archetypes identified in the report could be extended into the sewerage sector without difficulty.

The study has considered the definition and measurement of risk. It has also considered the influences of operational and capital expenditure. However, the specific requirement in MD161 to demonstrate how the flow of services to customers can be maintained at least cost in terms of both capital maintenance and operating costs whilst recognising the trade off between cost and risk has not been addressed.

The concept of serviceability

The key concept of a serviceability measure is that it should comprise a number of indicators of asset performance together with the undertaker’s effectiveness in operating them. The overall serviceability measure is an indication of the adequacy of the whole system to deliver a stable, satisfactory service to customers and the environment.

Principles for constructing serviceability measures

A basket of possible archetype serviceability measures included in Section 10 of this report is set out under categories. The points, which separate ‘pre-AMP4’ measures from ‘post-AMP4’ measures, have not been defined. Those will depend on testing and consultation.

The possible measures are grouped into: the existing measures, those involving development of existing measures and entirely new measures. They are sub-divided into asset and operational performance measures. A total of 18 measures are discussed and presented for consideration beyond those in current use.

This basket of archetypes provides a basis for discussion on how to proceed towards AMP4 and beyond. The final selection of measures to be assessed when considering the companies’ maintenance of overall serviceability to customers must consider a balance of what is appropriate and what can be achieved within the available time.

Our main conclusions are that there should be sufficient information held by companies to derive asset and operational performance measures at treatment works and zone level. Information should be available for previous years, although the extent of dis-aggregation needs to be tested.

Water quality data is available within companies to apply to measures of asset performance, although there may be shortfalls in the number of samples and gaps in time series data. Access to operational data would help the development of a methodology for water treatment works. For distribution, we may have to rely on regulatory information where there should be adequate data points for broad statistical analysis.

We conclude that a large volume of past and present water quality data exists in a useful format. The data is of a relatively high quality. Some companies already analyse the data to provide information about the performance of their assets and management systems. We believe that, after trials and consultation, standard methods of analysis should be applied to relevant parameter determinations.

We conclude that risk is an important element in the assessment of serviceability, building on the various measures of asset performance and operational performance. It is a forward-looking measure that has been applied implicitly by Regulators in the agreement of programmes of work. A methodology can be developed to reflect the likelihood and consequence of occurrences.

Brief consideration is given to the potential of combining weighted measures and the construction of an asset/operational performance matrix. It is clearly not possible to move directly to such approaches, but they are worthy of consideration in the long term. Aspirations to a long-term integrated approach could influence the shape of the short and medium term approaches.

 

1: INTRODUCTION

This document is the deliverable for Phase 2 of an assignment carried out by WS Atkins Consultants Ltd for the Drinking Water Inspectorate and Ofwat. The contract comprises a review of the drinking water aspects of serviceability relating to infrastructure and non-infrastructure assets.

The document has been produced with a view to using it as a basis for consultation on the issue of serviceability by the Regulators with the water companies in England and Wales.

The text is written mainly in the third person and does not attribute policies, views or ideas to a particular source. The use of the first person - principally in 'conclusions' - denotes the views and opinions of WS Atkins, which are not necessarily those of the Regulators.

WS Atkins would like to acknowledge the valuable assistance provided by the Inspectorate, Ofwat and the two companies visited during the preparation of this report.

The document is structured to form the basis of a consultation involving all sides of the water industry.

Sections 1-5 cover various facets of the study findings on the existing situation and the principles involved in moving towards more robust serviceability measures. Section 6 considers the approaches to statistical analysis of company data and Section 7 looks at risk. The impact of company activity on serviceability measures is discussed in Section 9.

While presenting conclusions on the study to date in Section 9, there remains a considerable amount of work before the 18 possible archetypes for serviceability put forward in Section 10 can be developed to implementation.

1.1 Objectives and Scope

The initial remit and objectives of the project were defined at a meeting involving the Drinking Water Inspectorate (DWI), the Environment Agency (EA), Ofwat and WS Atkins Consultants Ltd (WS Atkins) on 14 November 2000. This report covers the following objectives:

  1. To improve understanding of links between drinking water quality and the operation and maintenance of treatment and distribution system assets in maintaining serviceability to customers.
  2. To review critically current methods of measuring serviceability of water supply assets.
  3. To investigate the options available for improvements to current indicators of serviceability for water supply assets.

1.2 Limitations

The breadth of this study, encompassing the interests of the DWI, Ofwat and the EA, leads to serviceability measures that are based on very detailed and comprehensive information. This introduces constraints on the implementation timetable. By contrast, the depth of the study does not reach as far as the economic aspects of risk and the capex/opex balance. This tends to constrain the development of a viable single serviceability measure. In particular:

  1. The DWI has an interest in measuring serviceability at the level of individual assets or, at least, of related asset groups. The Inspectorate makes most of its decisions and gives opinions at that level. Ofwat, on the other hand, whilst interested in individual assets, makes most of its judgements and comparisons at company level.
  2. It is recognised that comprehensive measures of serviceability will take time to test and adjust. Only some of the measures can be expected to be available in time for use in the next periodic review. It may still be unrealistic to apply overall serviceability measures at that time.
  3. The requirement on companies in MD161 to demonstrate how the flow of services to customers can be maintained at least cost in terms of both capital maintenance and opex whilst recognising the trade off between cost and risk is not addressed in this study.

1.3 Aspects that Contribute to Serviceability

There is general agreement that serviceability should encompass measures of asset and operational performance related to four key areas:

1.4 Definitions

For clarity, definitions of key terms have been established and are applied through this report. These terms form the basis of the methodology and development of serviceability measures and need to be considered by all stakeholders, so a common understanding of the terms can be taken forward.

Serviceability to customers: a long run approach which considers the ability of the water (and sewerage) systems to maintain a standard of service to customers and the environment. A high level measure which generally reports exceedances or non-compliance against standards. (Ofwat PR99 Glossary)

Thus the assessment of serviceability to customers and the environment requires a number of a indicators of the state of the assets in terms of asset performance, combined with the undertaker’s effectiveness in operation of the assets in terms of operational performance. The overall serviceability measure is an indication of the adequacy of the whole system to deliver a stable, satisfactory service to customers and the environment.

To clarify any ambiguities, serviceability is a measure of the service customers (or the environment) directly receive from a company. These measures reflect water quality compliance, availability, reliability and customer care.

A robust measure of serviceability requires a detailed methodology and framework taking into account appropriate measures and sufficiency of information for both asset and operational performance.

The methodology then requires the definition of further terms for the components contributing to serviceability as set out below, to provide consistency across companies and clarity between regulators and utilities.

Output Quantity:a measure of performance against a defined standard of service to customers or the environment.

Asset Performance Measure: a quantitative measure that demonstrates how inherently fit-for-purpose the asset is to carry out its duty or process.

Asset Condition Measure: a quantitative (or qualitative) measure that demonstrates the physical state of the assets.

Operational Performance Measure: a quantitative measure which represents the operating outcomes of the assets.

Operational Practice: a qualitative measure reflecting the management practice and inputs applied in operating assets.

The definitions of asset performance and condition are similar to those used in Scotland by the Water Industry Commissioner.

Consistency in approach and clarity between Regulators and water companies is important. The first step is to set out clear and concise definitions of key terms, as above. Should these terms differ from any local practice or understanding by regulators and companies, the differences need to be identified and discussed.

Fundamental to the successful conclusion of the review process is a clear, universally understood and agreed set of definitions.

2: FORM OF SERVICEABILITY MEASURES

There are differing views on the merits of combining individual indicators to form an overall serviceability indicator. Currently, Ofwat tracks individual indicators at company level, as in Ofwat Information Notes 35A & 35B, to inform judgement on the needs of capital maintenance. (Ofwat does use a weighted index assessment when assessing overall performance, as in the final determinations.) Views are invited on the value of aggregated measures set out in this section.

Some general requirements were agreed as a basis for the investigation, review and development of aggregated serviceability measures. They are derived from the remit and objectives and recognise the limitations outlined in Section 1.2, above.

2.1 Key Principles

Discussions between the DWI, Ofwat and WS Atkins elicited the key principles of an aggregated measure of serviceability and how it should be applied. Serviceability should be:

  1. derived from measures at individual treatment works, service reservoirs and in zones within companies. The framework could be extended to other assets;
  2. a combination of asset and operational performance measures;
  3. sufficiently flexible to reflect each company’s circumstances but within a robust framework to ensure consistency;
  4. scaled from unacceptable (non-serviceable) to good (fully serviceable) using an appropriate scale;
  5. able to reflect the level of risk taken by companies.
  6. suitable for aggregation to a company level using weightings based on population or volume;
  7. able to recognise the relative importance of its components;
  8. forward looking, through projection of component measures;
  9. rearward looking to recognise the influence of trends and historic inputs;
  10. useful as a comparator within and across companies.

2.2 Method of Reporting

Serviceability measures should be practical to avoid burdening Regulators with unnecessary detail or complex methods of analysis. The onus should be on companies to carry out data analysis and report at high level. At the same time, Regulators are interested in any material poor performance. The key elements of a reporting structure are therefore:

  1. high level, aggregated and weighted measures of performance;
  2. reporting of poor performance by exception;
  3. data held by companies on secure systems and available for inspection as required;
  4. methodologies developed by companies to suit their own systems and management structures within a framework of guidelines issued by the Regulators;
  5. information subject to audit by the appointed DWI Inspector/ Technical Assessor or Ofwat Reporter.

2.3 Combination of Asset and Operational Measures

Measures may be combined into overall serviceability scores or indices. Further discussion and evaluation is needed to consider whether just one serviceability measure is appropriate. Individual measures are not necessarily independent of others so combination should be approached with caution. Weighting is about individual judgements of importance. The possible serviceability measures should be evaluated and tested in some detail before weightings are decided. Ultimately, the judgements will rest with the Regulators.

2.4 Links to Capital and Operating Expenditures

It is assumed that some capital expenditure is an input to provide improvement in asset performance and condition. Thus a unit of capital expenditure will provide an improvement in one or more asset performance measures. Similarly, it is assumed that some increases in operating costs are inputs which would provide improvements in operational performance.

2.5 Deriving Long Term Expenditure Requirements

As well as reflecting the standards of service received directly by customers, serviceability measures should be able to inform the assessment of future expenditure requirements. They should also inform the nature of that expenditure - whether long-term, major capital investment in maintenance, minor capex or on-going operating expenditure.

2.6 Avoiding a 3 Approach

A serviceability measure should not be so mechanistic that the methodology might lead to a ‘shopping list’ of schemes and expenditure which may be greater than that needed to maintain serviceability. One safeguard against this possibility might be to require companies to demonstrate after the event a statistically significant improvement in benefit, or performance measure, from the level of expenditure proposed. This approach has been followed with some success in the Section 19 Undertaking work.

 

3 INFORMATION REVIEW

3.1 Water Quality Information

The main sources of water quality information are summarised in Appendix B with references to statutory requirements and DWI Information Letters.

Water quality information is available in a number of areas including:

Regulatory water quality information for treatment works is derived from samples taken at the works and at other sampling points. The obligatory treatment works’ samples are restricted bacteriological parameters and residual disinfectant, although turbidity is to be measured under new Regulations. Some 15 other parameters that are sampled at other points in the system are deemed to be unaffected by location and can be taken to be representative of the quality of water leaving the works.

The extent of sampling and testing is laid down in the Water Supply (Water Quality) Regulations 1989, and the new regulations, SI No. 3184:2000.

3.2 Non-Quality Information

The main sources of non-quality information are summarised in Appendix B with references to Annual Return and Periodic Review submissions to Ofwat and Returns to the Environment Agency.

The main sources of information include:

Annual (June) returns to Ofwat reporting actual output quantities and activities carried out in the report year, including measures of low pressure, interruptions, mains bursts, leakage and operating costs but all reported at company level.

3.3 Water Quality Incidents

Companies are required by Information Direction 1998 to report all ‘events’ to the Inspectorate. Some of these events may be classified as ‘incidents’, according to criteria set out in Information Letter 13/99, and these are investigated by the Inspectorate.. An Inspector’s report is prepared on each incident.

A review of reported incidents in 1998, 1999 and 2000 revealed that the databases contain detailed information of the company, incident name, service area, nature of each incident, and its impact in terms of population affected. Examination of a sample of Inspectors’ reports provided detailed assessments of the cause of incidents.

About 80% of incidents related to distribution, 10% to 12% related to treatment, 1% to 6% to service reservoirs and a similar figure for resources. Many of incidents related to planned work, burst mains and deposits in the distribution system and microbiological/ disinfection failures related to treatment works. The incidence of non-infrastructure asset failures is low. As an example, the analysis of the 1999 incidents is shown in Figure 1.

 

Figure 1: 1999 Water Quality Incidents Pie-Chart

The system is satisfactory for what it is designed to do. There is a wealth of information on incidents within the Inspectorate which can contribute to the assessment of operational and asset performance. It would be possible to develop the incident score as a measure of mainly operational performance, and compare performance across companies. Closer linkage of incidents to zones and treatment works would assist this assessment.

3.4 Visits to Water Companies

Two large water companies were visited to understand the extent of regulatory information held by them, the methodology for reporting and the nature of the systems used to store and report information.

The visits confirmed that robust data storage systems were available for both regulatory and private operational data, that data could be retrieved easily and that both bespoke and ad hoc procedures were used for processing reports. The two companies concerned were also very helpful in demonstrating how the data could be used for internal management reporting.

3.5 The Approach of Other Regulators and Utilities to Serviceability

Reports and the current practice applied by some other utility organisations have been reviewed and are summarised below.

Scotland

The Water Industry Commissioner for Scotland has developed further asset performance and condition assessments based on the Ofwat model. A measure of risk is also included, reflecting the likelihood and consequence of any event. In addition, a methodology has been developed to roll forward asset condition and performance to provide a revised statement of asset performance condition and risk each year, based on new assets provided and any information to reflect deterioration of assets. The concept of combining performance measures to give a serviceability measure has not been followed.

Overseas

Water utilities in Australia and New Zealand follow similar performance measures of interruptions, pressure, water quality and measurements of customer satisfaction. Many of these measures are similar to Ofwat’s June Return Information Requirements. Water quality standards for some parameters differ from UK; the main difference is that the standards are guidelines rather than statutory.

Electricity

In the electricity sector, the Regulator, Ofgem is developing an ‘Information and Incentives Project’ (IIP) with distribution companies. This takes a wide look at the balance between costs and quality (performance). The methodology has identified three key measures to be incentivised: the number of customers interrupted (customers interrupted per 100 customers per year), the duration of the interruptions (as customer minutes lost) and the response that customers receive when they contact the company (measured by the speed and quality of the response). Incentives are limited to up to 2% of regulated revenue per annum. While this is a specific development for the electricity sector, there are commonalities in terms of combining and normalising costs and performance measures as a framework for comparisons.

Other

Approaches to serviceability developed by some England and Wales companies for their Periodic Review submissions are available within Ofwat but could not be made available for this phase of the study.

3.6 Literature

Search

A literature search exercise was carried out as part of this phase of the project. This made use of the powerful search engines on the Internet to identify publications that might contain information that would provide new directions and ideas for consideration. Examples of the publications highlighted are included as Appendix C. These covered work carried out in the United Kingdom, North America, Australasia, West Indies and Europe, but the search was confined to English language publications.

The exercise identified considerable numbers of documents with appropriate key words and phrases, but few worthy of detailed examination. Many were based on existing Ofwat measures as benchmarks; others gave very superficial consideration to this very complex subject.

Some abstracts indicated they might be of interest to the extent of justifying their purchase. Detailed examination of these documents and a widening of the search to include non-English language documents may be useful.

UKWIR

UKWIR has produced a report ‘Quantification of Serviceability’ dated 1998. The report is comprehensive and reaches firm conclusions. It identifies three broad categories of serviceability measure, which are:

The report identifies some useful criteria for the effectiveness of the measures and emphasizes the need for ‘leading’, as opposed to ‘lagging’, indicators. There is an attempt to separate out the influence of "factors outside a company’s control" but it recognises the need to measure the ability of assets and systems to deal with extreme events. The report emphasizes customer and social values and incorporates them in its suggested measures. The measures include costs, both forward and rearward, and risk assessments. Risk includes three elements - probability, extent and importance. The report does not recognise the constraints on the supply of information.

 

4: CURRENT SERVICEABILITY MEASURES

4.1 Introduction

The existing measures used to assess serviceability by Ofwat and the Inspectorate are included in Tables A1 and A2 in Appendix A. These measures are defined and described in the Ofwat Information Notes 35a and 35b dated March 1999 for infrastructure and non-infrastructure assets respectively. Current serviceability measures were examined and are discussed below.

4.2 Criteria

The current serviceability measures were examined in order to establish a firm base on which to develop the concepts for potential future approaches to the drinking water quality aspects of serviceability relating to infrastructure and non-infrastructure assets. The principal criteria applied in the assessment of current measures were:

Can the measure be dis-aggregated to treatment works or distribution zone?

Are the measures mainly representative of asset performance or operational performance or, if they are combined, can they be disaggregated in an appropriate way?

Are they ‘threshold measures’ or is a full range of data reported?

What is the extent of measures; are there sufficient data points?

Do the group of measures cover all areas of performance; are there any gaps in the measures?

Is the level of risk evident in terms of likelihood and consequence?

Are the measures forward looking?

4.3 Ofwat

Ofwat makes an assessment of stable, improving or deteriorating serviceability based on an analysis of trends in a number of individual measures. Data goes back to 1980 although there are some gaps.

Ofwat’s approach is based on a perception of serviceability, acting on from the viewpoint of the customer. The aim of the existing measures is to identify a clear overall trend in the serviceability of water company assets. They are a pragmatic means of providing a measure within the constraints of incomplete and variable quality information about asset condition, performance and customer perception.

4.4 DWI

The Inspectorate measures compliance against statutory water quality standards and assesses incidents against the requirements of Information Letter 13/99. Reporting to the Inspectorate is by total number of treatment works, although exception reports identify the location of any regulatory sample failures. A summary of the information is reported in the Chief Inspector’s Annual Report. The Annual Report also includes company-level indices for water quality, based on 17 water quality parameters and for water quality based on 6 operational parameters.

The existing measures are all high level and generally report performance by exception; that is, where values have exceeded threshold values, whether they be statutory or set by Regulators. By their nature, the measures reflect the tip of an iceberg and give little visibility on the underlying trend and extent of information.

4.5 Coliform Measures

Generally, coliform-based measures applied to treatment works and service reservoirs relate more to the operation of these assets than to the performance of the assets themselves. This is borne out from a review of Incident Reports prepared by the Inspectorate.

The coliform parameter is not a helpful indicator at asset level in that values are either zero or greater than unity, which is a regulatory failure. It is possible that companies will establish robust relationships between colony counts and the risk of coliform failure and that this relationship could in due course become a useful performance measure.

4.6 Distribution Zone Measures

Low Pressure

Low pressure is an indicator of the capacity of a distribution zone. The reported DG2 ‘headline’ figure has fallen significantly over the last ten years to, in most cases, a small number of properties usually at the extremities of the distribution systems. Companies are expected to maintain a similar ‘headroom’ figure into the future, indicative of maintaining a ‘steady state’ situation.

Solutions for low pressure problems are either operational, through rezoning, pump resetting and the like or minor capex schemes such as small distribution reinforcement.

DG2 is a sound, established measure of service to customers. It has been used since 1980. Data quality, while poor in earlier years, has improved with best practice companies using continuous pressure monitoring. It is a reflection of both asset performance and operational performance.

Interruptions

Information has been collected on the DG3 measure since 1980, although the > 3hour band has only been added recently. The duration of events in the >6hr band are driven mainly by operational practice. At the shorter duration band, the measure may become less influenced by operational practice and more by asset performance as has been illustrated at Sydney Water. The Corporation has a higher burst rate than many companies in England and Wales and yet it manages its system to minimise interruptions to customers, at a level lower than in the UK. This reflects the two components; the mains burst rate as an asset performance measure and the effectiveness of the management of the event as an operational measure.

Iron in Distribution

The number of zones failing against the iron pcv has been used as a high level measure. This reflects the water mains rehabilitation programme, but not sufficiently to link to capital maintenance.

The Inspectorate monitors the Section 19 Undertaking programme in detail; the pre- and post- renovation assessment (PPRA) provides a detailed assessment of progress and improvements. Companies have a statutory requirement to undertake this work.

Some companies have completed all their Section 19 Undertaking work and any further expenditure will be classified as capital maintenance. Other companies have varying programmes of work extending over the next ten years. Consideration is needed as to how reporting on Section 19 work can be more reflective of the capital maintenance needs to meet quality requirements.

The existing iron in distribution measure combines results at zone level, although zone information is limited to a ‘pass’ or ‘fail’ against the pcv. No use is made of the water quality data below the pcv; for example analysis of the distribution, in statistical terms, of regulatory water quality data.

 

5: SPATIAL DEFINITIONS

The assets and area of supply of the water industry in England and Wales have been classified and divided by custom and practice that has withstood the test of time. The spatial definitions relevant to serviceability measures are:

5.1 Company Boundaries

Company boundaries are defined in Licences and Inset Appointments. A company is an obvious grouping of assets and population to which serviceability measures or indices can be applied. All performance measures required by the Regulators can be related to company boundaries. The annual returns of water supply performance and other information made to Ofwat by the companies are at company level. Most of this information is produced by aggregating and collating separate items from organisational sub-divisions of the companies. In most cases it would be possible to assign data to the spatial definitions described below.

5.2 Raw Water Sources

The location and physical attributes of raw water sources are known and are listed in company Strategic Business Plans and EA abstraction licence records. They divide conveniently into underground and surface water sources. For the purpose of serviceability measurements they can be grouped under common aquifers or rivers. Private information about the chemical and bacteriological quality of raw water is held by companies.

5.3 Treatment Works

The location and physical attributes of treatment works are known and are available to the Regulators. The performance characteristics of treatment works are individual. They cannot be grouped. Some water quality information relevant to treatment works is in the public domain; additional information is collected for operational purposes and is held by companies. The Regulations require only six parameters to be sampled at treatment works - bacteriological, turbidity and nitrite plus others in certain circumstances.

Under the 1989 Regulations, monitoring at water treatment works is required for only total coliforms, faecal coliforms, colony counts and residual disinfectant. Under the 2000 Regulations, which come fully into force on 1st January 2004, monitoring at treatment works will include E-coli, coliform bacteria, colony counts, residual disinfectant, turbidity and nitrite. In addition, the 2000 Regulations require monitoring at supply point for further parameters, in practice many supply points may be treatment works.

5.4 Service Reservoirs

The definition of a service reservoir is understood throughout the industry. A service reservoir site can contain more than one service reservoir and any service reservoir can be divided into two or more compartments. The 1989 Regulations require sampling at service reservoirs for total coliforms, faecal coiforms, colony counts and residual disinfectant. In essence, the sampling under the 2000 Regulations at service reservoirs is the same.

5.5 Water Supply Zones

Companies define water supply zone boundaries and are permitted to change the delineation every year. The current limit on zone size is 50,000 population but this is due to change to 100,000 in 2004. The current average zone size is in the region of 20,000 population. At present zones are usually defined on practicable bases that include a requirement for hydraulic separation. Where Section 19 Undertakings are in force, zones are now divided into defined sub-zones. Sub-zones either correspond to or are an aggregation of DMAs. A typical DMA contains about 1000 properties.

The situation with respect to zones appears to be that companies have adjusted some but not all zone boundaries since 1991. The process has accelerated in recent years due to the creation of DMAs as part of the response to mandatory leakage targets.

 

6: STATISTICAL ANALYSIS

Companies hold water quality information in electronic format. In most companies the span of this record is at least ten years and in some cases is 15 years or more. The information is generally of high quality. Statutory sampling is, on the whole, random, the procedures are quality-assured and the relevant staff are independent of operational management.

Companies have developed monitoring and reporting systems that are based on this information. The reports are used to inform management decisions as well as to provide information to shareholders, the public and the Regulators. Some companies have also developed indices of performance that are derived from analyses of water quality data.

This situation offers an opportunity to produce robust water quality performance measures at company level and below, to derive estimates of relative serviceability, to identify trends in historic data and to extend those estimates and trends into the future.

6.1 Concepts

In general terms the concepts are:

Sample results demonstrating means and ranges at one point in time that are nowhere near the pcv would indicate satisfactory performance in the relevant assets;

Figures 2 and 3 are stylistic representations of how data analysis could provide insights into performance. They assume normal distribution for illustrative purposes.

6.2 Ways of Producing Performance Measures from Water Quality Data

Quantitative performance measures can be calculated from water quality parameter determinations for a set of assets and a set period of time by using conventional statistical techniques. The calculated figures can be compared with corresponding figures for other sets of assets or other time periods. Some - perhaps most - parameter determinations would yield estimates of the probability of occurrence of some event - pcv exceedance, for example. All would yield estimates of the proportion of samples that would exceed some measure - the pcv or a lower threshold value.

Estimates of probability could be used in risk calculations and would link directly with performance measures. Estimates of ‘failing’ proportions could be used qualitatively to inform the value of serviceability indicators.

Trend analysis can be applied to annual data sets to identify adverse or improving situations. Trends in both mean and upper bound parameter determinations can be identified. The length of record should be at least five years. Ten years would be better. Year-on-year differences would have to be tested statistically to demonstrate ‘significance’. There are a number of standard significance tests, all giving slightly different results. The choice of test and criteria would almost certainly be controversial.

Historic trends can be extrapolated to provide a forecast of future performance. Forecasts would depend on a ‘steady state’ assumption and would be uncertain. The ‘steady state’ assumption is unlikely to be valid in practice because of the effects of quality programmes, capital maintenance investment, zone boundary changes or changes to managerial procedures. The results can be adjusted for the effect of investment programmes but no simple remedy exists to counteract the effect of boundary changes. Boundary changes may have a significant effect after 2004. Historic trend analysis is likely to be valid at company level (provided that any company amalgamations within the analysis period are taken into account) but would require the examination of explanatory factors at zone or site level. Nevertheless, the identification of significant adverse or improving trends is useful information that could be used to inform indicators of overall serviceability.

6.3 Considerations Affecting the Statistical Analysis of Water Quality Data

  1. The number of samples taken for each parameter will affect estimates of probability. At low numbers the uncertainty is large but it diminishes rapidly in the range of 12 to 20 samples. The improvements in certainty as sample numbers increase above 20 are relatively small. We think that the problem of small sample numbers will not be significant. Parameters subject to small numbers of samples are usually not material to serviceability considerations - that is, the probability of pcv non-compliance is very low. If the small number of sample becomes an issue, the problem could be mitigated by grouping parameters or years.
  2. Not all sets of parameter determinations will be distributed in a statistically convenient way. Distributions that are markedly skewed will require special consideration. In these cases, the data is often ‘transformed’ to make its distribution more amenable to analysis. Transformation techniques can be controversial.

 

Figure 2: Data distributions (1)

"Time 1" indicates satisfactory serviceability at that point in time.
"Time 2" taken together with "Time 1" indicates worsening serviceability.
"Time 3" indicates effective management action keeping sufficient results within a ‘threshold’ value. Serviceability is satisfactory. The threshold value is set (statistically) at an acceptable level of risk.
"Time 4" indicates worsening serviceability as management action fails to keep sufficient results within the threshold value.

 

Figure 3: Data distributions (2)

"Time 1" indicates satisfactory serviceability at that point in time.
"Time 2" indicates worsening serviceability as the ‘shape of the distribution curve flattens. The mean value has not changed however.
"Time 3" indicates unsatisfactory serviceability as the shape of the distribution curve flattens to a point where there is a high risk of a PCV failure. Again, the mean value has not changed.
"Time 4" indicates very unsatisfactory serviceability as the shape of the distribution curve skews to a point where the risk of PCV failure is even higher.

 

  1. Parameter determinations should be filtered before analysis to remove outliers. The identification of outliers may prove to be controversial. For example, companies would probably wish to exclude the effects of wet weather similar to that experienced during the winter of 2000/01 but the Regulators could, with some justification, take a contrary view. On the other hand it would be misleading to include in a probability calculation a parameter value for iron that is more than (say) 500µg/l. Such results do occur from time to time and are sometimes the result of third party damage to a distribution system.
  2. Some parameters are of the ‘pass or fail’ type - for example, coliform bacteria. The distribution of these results will be binomial and failure proportions will have to be estimated accordingly. Coliform failures are comparatively rare so the proportion of ‘passes’ will be very large in relation to the ‘fails’. The WRc addressed this problem when constructing the methodology for the 17-parameter index currently used by the DWI. Its solution was to apply a statistical device that effectively reduced the proportion of ‘passes’. This device works well at company level but would not be universally effective at zone or site level. Most zones or sites ‘pass’ consistently over long periods of time and probability cannot be estimated. Statistics will not help in these circumstances therefore and an alternative measure or measures will have to be developed. One approach might be the use of plate colony counts or trends in counts in association with environmental factors to estimate the probability of coliform occurrence.
  3. Many parameters vary significantly with the time of year. Probability estimates would be affected by the wide spread of results and may lead to a pessimistic conclusion. Seasonal analyses may mitigate the problem but trials of real data would allow a better assessment to be made.

 

7: RISK

As with the definition of serviceability, there is a wide range of opinion on the definition of risk and its application. A suggested definition is set out.

Risk:a combination of the likelihood of a failure in asset or operational performance and the consequence of the failure on customers and the environment. In this context ‘consequence’ includes both the extent of the risk and its seriousness.

In annual reporting, Regulators collect historic information on actual performance of assets and serviceability to customers which are not reflective of risk but report actual outcomes in a defined period. However, this information, taken over a reasonable period, helps to calibrate any modelling of risk.

Risk can be used to justify asset or operational improvement from a current, perhaps unsatisfactory, position to one where companies or Regulators are content. Risk should be a matter for companies to manage. But Regulators have valid interests in that companies should not take undue risk which might impact on their ability to meet obligations. Regulators also wish to take a view of risk through the support or otherwise of asset improvements and relate expenditure to meet specific obligations.

One example is the Section 19 Undertaking work to renovate water mains, where companies have proposed a threshold measure, or measures, which where exceeded justify renovation work. Here the Regulators wish to take a view on the level of risk through the the approval of companies’ rehabilitation strategies. It is interesting to note that companies have taken different approaches to the renovation criteria; it was not for the Inspectorate to set criteria, but to assure itself that the risk of failure against the water quality regulations would be reduced to an acceptable level.

Another example is the improvements at treatment works to reduce the risk of cryptosporidium entering the water supply. Companies carried out risk assessments of vulnerable works and, on this basis, programmes of works have been agreed.

Ofwat used to collect risk based measures in annual reporting, such as a water resource measure (DG1) and risk of low pressure (DG2), although the latter has now been re-defined to report actual events. There is still an ‘at risk’ register for sewage flooding.

One high level method of assessing risk is to consider measures of likelihood and consequence, and combine these into a matrix. This is being applied by the Water Commissioner for Scotland as shown in Figure 4.

Figure 4: Example of Risk Matrix

 

The likelihood of a failure or occurrence could be classified as frequent or continuous, probable, rare and exceptional, applying appropriate definitions. Similarly the consequence of failure could be classified as low, medium or high. In terms of the water service, consequence could be linked to population.

The likelihood of failure or occurrence is an assessment of the various asset performance and operational performance measures on each asset or network of assets. The risk analysis could build on these measures rather than be considered as a separate measure.

The chart shows lowest risk in the top left (green) and highest (red) in the bottom right corner.

Companies also carry out detailed quantitative risk assessments of their networks of assets with a view to assessing relative risks and prioritising schemes. These assessments are a matter for companies to apply in making best use of their resources and are best left with companies. In informing Regulators of likelihoods, companies can use quantitative or hazops techniques which may differ across the industry.

 

8 THE IMPACT OF COMPANY ACTIVITY

8.1 Operational Activity and Maintenance Strategies

Differing levels of operations activity, maintenance strategies and capex can have a material influence on serviceability.

Two distinct types of operational activity can materially impact upon the performance of the assets:

The performance of infrastructure assets can be changed by normal operational activity. The condition of infrastructure assets reflects the level of capital maintenance. However, surrogate measures for the condition of infrastructure such as burst frequency are readily affected by operational activity, such as pressure reduction, rezoning or avoidance of pressure transients.

The effects of operational activity are similar to those obtained from the investment of relatively minor amounts of capital expenditure. ‘Minor schemes’ expenditure is usually well controlled and correctly allocated to assets and to purpose.

The maintenance strategy chosen by a water company should recognise the economic balance between operational and capital activity and hence expenditure to achieve target serviceability at an acceptable risk. Serviceability measures therefore should indicate the effectiveness of current strategy.

The level of infrastructure and non-infrastructure reactive and planned maintenance classified as operating costs are reported in Tables 21 and 22 of the June Return. Reported operating expenditure is not sufficiently dis-aggregated to allow a holistic overview of the expenditure as a factor affecting serviceability. A component approach to the reporting of costs might involve significant changes to the management accounting carried out by water companies, although there are other pressures on companies to introduce activity-based costing (ABC) at process and zone level.

Comparative opex efficiencies at company level are produced by Ofwat’s econometric models. The results from these models should be used to inform judgements made about the influence of current or future operational strategies on serviceability to customers.

8.2 The Impact of Non-Maintenance Capex on Serviceability Measures

Quality Enhancements

In distribution, the quality enhancements are clearly defined and agreed by the Inspectorate. Companies can chose to reline or renew mains, where a proportion of the renewal cost can, under current rules, be apportioned to capital maintenance. There are likely to be indirect benefits to serviceability due to the processes involved in scraping and relining mains. Surveys identify structurally weak mains identified for renewal. Mains renewals associated with renovation are therefore pre-emptive and have a positive effect on serviceability.

At treatment works, agreed programmes of work either improve the existing works or enhance the process to meet new standards. New quality standards represent a shift and imply that serviceability targets should be re-set. Improvements to works or service reservoirs to meet existing standards would not usually be allocated to the quality purpose but would probably count as base expenditure.

Growth in Demand

The current pricing structure assumes that, in the long term, the total cost of meeting increased demand is funded by capital contributions and income from new customers.

A large development can involve water main extensions and off-site mains to be funded by the developers. If the water company has confined off-site reinforcements to dealing with local problems, the take-up of redundant capacity in the rest of the supply system may bring forward the time at which pressure problems will become evident. A prudent company might have taken the opportunity to promote off-site reinforcements that enhance the overall system.

The overall impact of additional distribution assets on asset performance and serviceability should be neutral, provided companies are able to obtain realistic contributions to the additional assets they create. However, the impact of specific work to accommodate growth in demand on individual treatment works or distribution zones is likely to be a significant improvement in performance.

Levels of Service Enhancements

Few allowances were made in the Final Determinations for expenditure on enhancements to levels of service. Where companies increase expenditure to maintain the headline figure of DG2, Ofwat cost allocation rules require this expenditure to be allocated to capital maintenance. Enhancement expenditure occurs only where there is a marked and permanent improvement in performance. By definition, this would be an improvement of serviceability, albeit small.

Management and General

Expenditure under management and general on items such as vehicles, plant or computer systems is normally to maintain current operational capability. An improvement in operational capability through increased expenditure or restructuring might then improve operational performance and serviceability. The main problem here is that under Ofwat rules, these costs are grouped asset maintenance costs and reported as ‘non-infrastructure’ expenditure. For transparency, it would be better to allocate these costs to ‘support services’ and leave ‘non-infrastructure base maintenance’ to real improvements in assets.

8.3 The Impact of Customer Perceptions

Uncertainties remain about customer perceptions of the water industry and its component parts.

Positive influences on customer perceptions can come from high profile leakage detection, main laying, mains rehabilitation and the removal or reduction of negative influences. Public relations exercises and press releases should have a positive effect but sometimes mis-fire.

Negative influences on customer perceptions can come from visible leaks, bursts, inactivity at road openings, poor pressure, dirty water, poor tasting water, contaminated water and interruptions to supply. It can also result from high bills or increases in bills, impositions (eg. metering, payment method) and poor response to queries and complaints. Publicity by Regulators and subsequent use of the information by lobbying groups can have a negative influence.

 

9: CONCLUSIONS

(i) We consider that there should be sufficient information held by companies to derive asset and operational performance measures at treatment works and zone level. Information should be available for previous years, although the extent of disaggregation needs to be tested.

(ii)We conclude that the current measures for assessing serviceability are insufficient to provide adequate measures of asset performance and operational practice either at company or individual asset level. This is because these current measures are influenced mainly by operational performance. Two key measures - burst mains and water quality in distribution - are reflective of asset performance, although the measure of water quality is not sufficiently detailed.

(iii) We conclude that asset based spatial definitions are clear and are stable over time. Spatial definitions of zones are usually clear but have not been stable over time. We expect zone definitions to change significantly in 2004. We believe that water quality information underlying the returns to the DWI is or can be easily related to specific treatment works, service reservoirs and zones. We also expect that information sent to Ofwat in annual returns could be similarly assigned with some effort

(iv) We believe that water quality incidents are mainly due to failures of operational practice and not reflective of asset performance. Incidents are usually due to transient water quality and how well the assets are managed rather than how the assets are performing. It would be helpful to relate the incident to a particular zone or treatment works or service reservoir. Companies should be able to provide this information without difficulty.

(v) We conclude that water quality data is available within companies to apply to measures of asset performance, although there may be shortfalls in the number of samples and gaps in time series data. Access to operational data would help the development of a methodology for water treatment works. For distribution, we may have to rely on regulatory information where there should be adequate data points for broad statistical analysis but not sufficient for detailed work.

(vi) We conclude that a large volume of past and present water quality data exists in a useful format. The data is of a relatively high quality. Some companies already analyse the data to provide information about the performance of their assets and management systems. We believe that, after trials and consultation, standard methods of analysis should be applied to relevant parameter determinations.

(vii) We believe also that a general index of water quality would be useful to Regulators and to companies. It should be produced in accordance with a standard methodology, again after trials and consultation.

(viii) We conclude that risk is an important element in the assessment of serviceability, building on the various measures of asset performance and operational performance. It is a forward-looking measure that has been applied by Regulators in the agreement of programmes of work. A methodology can be developed to reflect the likelihood and consequence of occurrences.

(ix) We conclude that operational activity has a material impact on serviceability. But as available ‘headroom’ operational activity is used up, serviceability will become more dependent on capital expenditure. We therefore need to separate measures of operational and asset activity which contribute to serviceability. This cannot be precise, so a pragmatic way forward would be to identify measures that are ‘predominantly’ operational or ‘predominantly’ asset related.

(x) We conclude that maintenance costs classified as operating costs and operational strategies have an impact on serviceability. Where companies have applied activity based costing, it should be possible to derive opex for treatment works, service reservoirs and distribution zones.

(xi) We conclude that both quality and base expenditure to meet existing standards should be regarded as inputs to improve serviceability.

(xii) We conclude that the impact of growth in demand in distribution should not have a material impact on asset performance and serviceability overall, provided costs have been allocated to capex drivers with reasonable accuracy.

(xiii) We conclude that level of service enhancements will bring small local improvements in serviceability, either through asset performance improvement or operational performance improvement

(xiv) We conclude that any increased expenditure here on "management and general" might improve operational performance. However, expenditure classifications should be revised to split this current line into ‘support services’ and ‘non infrastructure asset expenditure’ for transparency and to assist forward projections.

(xv) We conclude that customers are likely to be most concerned about water quality, including transient events, and reliability of supply. Any development of serviceability measures should consider how best they could be explained and in terms that customers can relate to.

 

10: POSSIBLE ARCHETYPES FOR SERVICEABILITY MEASURES

A requirement of serviceability measures or indices is that they should reflect appropriate asset and operational performance measures. This requires a series of measures relating predominantly to asset performance or predominantly to operating performance. There are some common measures that provide a measure of overall performance to customers. The process followed in considering serviceability measures was as follows:

  1. identify the measures which may be predominantly related to either asset or operational performance;
  2. assess the possible methods for quantifying these measures;
  3. identifying possible methods for reporting at high level;
  4. considering scoring individual measures and methods of combining to derive an overall score.

The possible archetypes for serviceability measures are presented in the following sections, which differentiate between asset performance measures and operational performance measures. Measures are set out as P1-5 and P11-16 for asset performance measures and O1-7 for operational performance measures. The nomenclature is consistent with that used through the evolution of the project.

This basket of archetypes provides a basis for discussion on how to proceed towards AMP4. The final selection of measures to be assessed when considering the companies’ maintenance of overall serviceability to customers must consider a balance of what is appropriate and what can be achieved within the available time.

Brief consideration is given to the potential of combining of weighted measures and the construction of an asset/operational performance matrix. It is clearly not possible to move directly to such approaches, but they are worthy of consideration in the long term. Aspirations to a long-term integrated approach could influence the shape of the short and medium term approaches. The matrix approach is considered in more detail in Appendix D.

10.1 Asset Performance Measures

The Use of Existing Measures for Assessment of Serviceability

The potential for developing these measures against the criteria set out in Section 4.2 was assessed, with the following conclusions:

However, most measures are well established, definitions are clear and consistent and information is generally of good quality. Also there is a good time series of data from 1980, although data quality is less robust in early years.

Possible Development of Existing Asset Performance Measures - Water Treatment

The coliform measure at the works outlet is more reflective of the operation of the disinfection plant rather than asset performance. With the new Water Supply Water Quality Regulations 2000, there is a requirement to monitor turbidity at the works outlet. Further parameters will be sampled at supply points and many supply points may be water treatment works. These parameters can be combined with bacteriological measures and turbidity to make up a basket of parameters that is indicative of treatment work performance.

Figure 5: Asset Performance Measures - Water Treatment Works

 

* See under "The Possible Development of New Asset Performance Measures"

 

The asset performance measures for water treatment works that could be a development of existing measures are highlighted in Figure 5 and detailed as follows:

P1: Quality of Supply

Water quality information is collected by companies to meet regulatory reporting requirements following the sampling programme set out in the Water supply (Water Quality) Regulations 1989. New regulations started to come into force in 2001, and come fully into force on 1st January 2004.

The questions are then:

Parameters that reflect the performance of individual works are required. Applying a common range of parameters specified by the regulators would not be appropriate as some would not be relevant and others might not be used. On the other hand, leaving the selection entirely to companies is likely to lead to inconsistent approaches. A way forward would be to select a combination of the two as a ‘basket’ of parameters to reflect performance. For example, common parameters reflecting asset performance might include:

The measure might have to rely on companies’ routine operational samples. There should be sufficient data for analysis.

Where companies have routine samples at both inlet and outlet, it might be useful to assess the water quality at the inlet and outlet. It might then be possible to derive some relationship with extent of removal of certain parameters (as is done on sewage treatment works) but this would need to be tested.

The next issue is how to analyse this data. The issues surrounding methods of data analysis have been discussed in Section 7. The options depend on the extent and quality of the information available, but could include:

The effectiveness of this analysis needs to be tested with a range of real water quality data. The analysis would produce a performance measure for each parameter. The method for combining this analysis to provide an overall measure for a works needs further development and testing. One way forward would be to express the values in terms of risk, such as:

...the percentage of total distribution input supplied from treatment works failing against [any defined parameter], or at high risk of failing etc ... ... ....

This extension of existing measures does not require collection of additional information by companies but utilises and reports on information, which the companies use for their own purposes. There is concern by companies that using ‘non-regulatory’ data in this way might be extending the arm of regulation. This should be addressed by discussion and clarity in the purpose of the analysis.

P3: Treatment Works Headroom

The performance of a treatment works is in part a function of the hydraulic and process loading placed on it. For example, a works operating at half its capacity enjoys fairly light process loading rates and therefore should operate well. Where a works is operating at peak capacity over a long period, there is an increased likelihood that any shortcomings in the processes will be reflected in asset performance. Capacity is a key asset performance measure to be included within the basket.

Further consideration of the methodology for deriving this measure could be based on available headroom, for instance:

... ...(1 - the average day peak week throughput) x 100 %
works rated capacity

or reporting by headroom bands:

... ...the proportion of distribution input provided by works with headroom less than 2%, 5%, 10% or >10% etc....

Information on works throughput, deployable output and water available for use is available from companies as part of their Distribution Input returns to Ofwat and abstraction returns to the Environmental Agency. Works rated capacity would need to be assessed by the Company; unsatisfactory asset performance or condition could impair this capacity.

P4: Asset Condition Grade

Asset condition could be considered to be a subsidiary performance measure. The main reason for asset replacement or rehabilitation is because an asset fails or is at risk of failing against a performance parameter. There are other reasons such as technical obsolescence, savings in operating costs or combinations of both which need to be considered.

There is an established methodology and definitions for asset condition set out by Ofwat, referred to as Submission H of Periodic Review 1999 (PR99). A shortcoming of this approach is that asset condition is only assessed every five years, as part of the periodic review process, and in the intervening period methods and assumptions used for this assessment might vary. Hence it is difficult to compare changes between each periodic review. One method, developed by the Water Industry Commissioner in Scotland is to roll forward the asset condition base, reflecting the impact of new assets and, where new information becomes available, amending the condition database. This methodology is still being tested but would appear to be a feasible way of updating asset condition.

For individual treatment works, we would expect condition grades to be expressed at process level, and combined by Modern Equivalent Asset weighting to provide an overall condition profile. An appropriate measure for each treatment works might then be:

... the percentage of asset value in condition grade 5, and in condition grade 4...

Figure 6: Asset Performance Measures - Distribution


Possible Development of Existing Asset Performance Measures - Distribution

The asset performance measures for distribution that could be a development of existing measures are highlighted in Figure 6 and detailed as follows:

P11: Quality of Supply

Water quality information is collected by companies to meet regulatory reporting requirements following the sampling programme set out in the Water Supply (Water Quality) Regulations 1989. New regulations have started to come into force in 2001, and come fully into force on 1st January 2004.

The question is then what parameters should be used to assess inherent fitness for purpose? For distribution zones, the parameters are likely to be common across most zones. For example, common parameters reflecting asset performance might include:

Regulatory sampling can be at a standard, increased or reduced rate depending on whether there have been any contraventions of the pcv. A preliminary assessment of the one companies’ data, where increased sampling is applied to all zones, shows that there is sufficient data for some statistical analysis. For zones subject to standard or reduced monitoring, the number of samples is significantly reduced and there may be a need to combine years.

The next issue is how to present and analyse this data. The issues surrounding methods of data analysis are discussed in Section 7. The options depend on the extent and quality of the information available, but could include:

The analysis would produce a performance measure for each parameter. The method for combining this analysis to provide an overall measure for a zone needs further development and testing.

One option might be to reflect the likelihood of an event or failure of any zone in any year as a banding from black (any failure against pcv), red (high risk of failure), amber (medium risk) and green (low). The zones could be combined as a total measure and weight by population to reflect consequence. An appropriate measure might be:

...The percentage of population in zones which have failed (black), at high risk of failure (red) etc...

P13: Headroom Capacity in Distribution

As with treatment works, the performance of distribution zones is in part a function of the hydraulic loading placed on it. For example, a distribution system operating at half its capacity enjoys fairly low velocities and should operate well hydraulically. Where a zone is operating at peak capacity over a long period, there is an increased likelihood that any shortcomings in the network will increase flows and is likely to be reflected in asset performance. Headroom capacity is a key asset performance measure to be included within the basket.

Further consideration of the methodology for deriving this measure could be based on available headroom, where zone capacity can be derived from calibrated network models and peak flows from actual flowmeter readings. All this data should be available from within companies, for example:

... ...(1 - the average day peak week throughput) x 100 %
zone capacity

or reporting by headroom bands:

... ...the percentage of population in zones with headroom less than 2%, 5%, 10% or >10% etc....

There may be practical difficulties in providing continuously robust data across a large company.

P14: Burst Mains

Mains bursts is a powerful measure. Good time series information exists and efficient companies should be able to report robust data. It would be feasible to disaggregate reporting to zone level as most companies record this information on their job management systems by zone or geo-referencing.

The form of measure should be based on all available data but not overburden the regulator. One simple method would be to use a simple classification of high, medium and low burst rates. However, a measure to quantify burst clusters within zones may provide a better characterisation. Burst information need not be reported by zone to the regulator; a summary of the zone classification should suffice. Common definitions would allow consistent comparisons across zones to be made.

In terms of forecasting performance, a trend analysis showing whether a trend is steady, rising or falling is probably more important than absolute values. There is a need to look at data from sample companies and test the feasibility of alternate data analyses.

P15: Leakage Rate

Leakage is a measure of the integrity of a zone and, assuming that the larger leaks and other losses from networks have generally been addressed, is a measure of losses from pipe joints and through pipe walls. A zone at or near its economic level of leakage would have been subject to detailed detection and repair programmes and be relatively ‘tight’. Conversely, a zone with leakage levels higher than the Economic Level of Leakage (ELL) is indicative of needing more work to identify and repair leaks, and the likelihood of other low asset-related performance.

We assume that capital expenditure would be applied to reduce leakage to a defined level, possible the ELL. Any further repair work to maintain this leakage level would be classified as operating cost. This assumption is by no means universal across companies and a consistent approach would be beneficial.

Leakage is measured or assessed at zone or sub-zone level and information is normally collated for high level reporting in the June Return. This information should be available for analysis.

Leakage could be measured relative to the ELL in terms of being:

P16: Asset Condition Grade

Asset condition could be regarded as a subsidiary measure of performance. The main reason for asset replacement or rehabilitation is because an asset fails or is at risk of failing against a performance parameter.

There is an established methodology and definitions for asset condition set out by Ofwat, referred to as Submission H of PR99. A shortcoming of this approach is that asset condition is only assessed every five years and assessment procedures tend to change between reviews. However, as with water treatment works, there is a potential way forward.

For individual zones, we would expect condition grades to be used to provide an overall condition profile, based on surveys, pipe sampling, and opportunistic sampling. An appropriate measure for each zone might then be:

..... the percentage of asset value in condition grade 5, and in condition grade 4...

 

The Possible Development of New Asset Performance Measures

The study has identified gaps in the range of necessary performance measures. This has given rise to proposals for new measures, as follows:

P2: Appropriate Treatment Process

A treatment works asset should be fit for purpose in that the processes should be appropriate to treat the likely range of raw water quality to meet statutory water quality requirements.

A company should assess its treatment works processes for suitability. This exercise was carried out for PR99 when shortfalls in process requirements were agreed with the DWI and improvement works agreed to deal with a range of parameters, including nitrate and cryptosporidium.

This is an important element when considering ‘fitness for purpose’. For example, there may be treatment works where improvements were proposed by companies but not supported by the Inspectorate.

Guidelines should be developed on appropriate processes. This might be in the form of a checklist. This approach might generate ‘shopping lists’ from companies on the improvements they would like to make. This could be countered by introducing a ‘significance test’ that requires any improvements to demonstrate a statistically significant benefit to performance.

P5: Assessment of Process Performance

Some companies have carried out detailed process performance assessments to inform their view on serviceability. A review of examples of this approach is required.

P12: Water Quality Transient Performance

Customers sometimes receive poor water quality through transients within the distribution system due to, for example, iron pick-up mechanisms within the network. From our experience with PPRA customer surveys, we know this is a particular concern of customers. Regulatory sampling does not often pick these transients up. This is an important measure for zones subject to particulate iron deposits.

Information on iron pick-up is not normally collected through routine sampling. PPRA surveys may collect information on iron pick-up to justify rehabilitation work, but this is a one-off survey. Where there are known problems companies should carry out surveys as part of their distribution zone studies, although it is not clear whether this is done. One way would be to carry out downstream series sampling at say annual frequency in zones with a high propensity for iron transients.

Another method would be to measure the extent of transients through monitoring customer complaints, excluding those caused by operational problems, and classify zones as:

Other indicators could include the extent of mains flushing.

An added indicator could be based on a relationship between iron levels from treatment works and in distribution, although this might not be feasible with complex zone arrangements. This measure is more complex than others and information is not readily available. It needs further discussion, development and testing within companies to firm up on options for measurement.

One future method of dealing with this would be to use turbidity as a surrogate and use the continuous logging equipment currently under development. Such an approach appears to hold considerable promise.

10.2 Operational Performance Measures

There is an issue of trade-off between operating costs and capital expenditure, which should be recognised in this analysis. For example, operating a works in poor asset condition may provide a high level of serviceability but at high operating cost. The same service could be provided with a good assets and lower operating cost. This issue is addressed later in this section.

Operating costs were considered as a possible measure of operating effectiveness but were found to lack clear relationships. For example, a low level of operating cost might indicate an efficient company or that key activities were not being effectively carried out.

The possible measures of operational performance selected by the study are set out below.

Possible Development of Existing Operational Performance Measures

O1: Water Quality Incidents

The study indicated that the main cause of incidents was shortcomings in the management of assets. By definition, incidents have a material impact on customers so it is important to include them in a serviceability measure.

Events are reported to the Inspectorate in accordance with the Information Direction 1998. The Inspectorate decides whether an event should be classified as an incident. Incidents related to water treatment should be identifiable from the Inspectorate’s records, although there should be clearer links.

The number of incidents at individual works level is likely to be small and statistical analysis is unlikely to be relevant. Nevertheless this is an important measure reflecting on customers and should be applied to assess serviceability of a works.

These measures are generally qualitative but a method for scoring should be considered.

O2: Operational Strategy and Procedures

The Regulator needs assurance that a comprehensive and effective operational strategy is in place for each works. The existence of an operational strategy and its effective implementation provides assurance that any events can be effectively managed with little impact on customers. While not a driver of expenditure, it is a matter of good practice.

Distribution strategies are required as part of the Section 19 Undertakings and could be considered as a statutory requirement for those companies with agreed programmes of mains rehabilitation.

O5: Low Pressure

Rather than measuring the headline figure of properties experiencing low pressure, which is likely to be insensitive in future years, a better way forward might be to measure the number of properties solved by operational improvements. This information is currently collected in the June Return, but only at company level. Most companies collect this information at zone level to support their company total.

O6: Interruptions

Interruptions to Supply, using the duration bands of >12 hours and >24 hours is mainly reflective of operational performance. An efficient company should be able to repair a burst or provide alternative continuous supplies to customers within 12 hours, or maybe six hours for smaller diameter pipes. It is interesting to note that Yarra Valley Water in Victoria, Australia, as a general rule provides temporary overland pipelines to maintain supplies to customers in the event of a burst.

The introduction of the > 3hr duration band (or 3 to 6 hours) should be more reflective of asset performance than operational performance.

It is suggested that the current measure of interruptions to customers is included as an operational measure. This information is currently collected in the June Return, but only at company level. Most companies collect this information at zone level to support their company total.

O7: Distribution Zone Studies

There is a requirement in the June Return for companies to report on progress with distribution zone studies. These are required to cover water quality, hydraulic analysis and structural integrity. Companies with detailed and comprehensive distribution zone studies should have a greater understanding of any shortcomings in zones which could then inform improvement works or operational strategies. Our experience shows that the emphasis to date has been on hydraulic studies. Regulators can gain greater comfort if companies have completed detailed and comprehensive zone studies, which are then, reflected in operational strategies and investment plans.

The Possible Development of New OperationalPerformance Measures

O3: Incident Management Strategy

The Inspectorate needs assurance that an adequate management strategy is in place to deal with incidents in the most efficient way. A company without adequate procedures in this area is increasing its risk level significantly. There are several examples of water quality incidents where the main problem was not failure of assets or treatment processes but inadequate management of the incident, for example Sydney Water 1998. The Inspectorate is currently auditing the companies’ incident management procedures to seek this assurance or require improvements to be made.

O4: Maintenance Policy

Maintenance strategies may be reactive or proactive. In most cases proactive strategies will reduce risk and provide benefit to serviceability. The benefit is not without cost however.

 

11: COMBINING MEASURES

11.1 Overall Serviceability Index

Methods of combining specific measures to form an overall serviceability measure(s) or index have not yet been considered in detail. There are two issues to consider.

Firstly, the measures are not necessarily independent variables so there is a problem that any linear combination of measures might tend to lead to wider differentials between companies than might be the case. Statistical methods for combining data to derive a reasonable spread of data points that is reflective of performance should be chosen with care.

Secondly the issue of weightings needs to be considered. The issue of weighting components should reflect what regulators and companies believe to be important.

11.2 Combining Measures into an Asset/Operational Matrix

A methodology which builds on the basket of possible measures combines the performance and operational measures in a way that reflects the ‘likelihood’ component of risk. Operational performance is plotted along an x-axis and asset performance along a y-axis to form a matrix. This concept is developed in Appendix D.

 

Appendix A

Details of Existing Serviceability Measures

 

Table A1. Existing Serviceability Measures used by Ofwat

Asset Type

Serviceability Measure

Data Source and Mode of Collection

Type of Measure

WS Atkins Comments
Treatment Works Number of works where enforcement action was considered because of contravention of the coliform standards (information note 35b) Table DWI 1 Annual Report to Ofwat Operational Enforcement action is normally considered where there are clear and evident failures of the management of the incident. Examples in the Chief Inspector’s report Nearly all relate to failures of management rather than asset failure.
  The percentage of the total number of determinations taken at water treatment works containing coliform Table DWI 1 Annual Report to Ofwat Operational Any coliform failures are normally reported to the Inspectorate as an event, following the requirement of IL13/99. The Inspectorate will review the circumstances surrounding the reported event. This will be reported as an incident as defined in IL13/99.

Our analysis of a sample of incident reports prepared by the Inspectorate shows that failure against the coliform standard is due to management shortcomings or failure of chlorination plant due to minor operational reasons.

We concluded that while the coliform standard is a useful measure of serviceability to customers, it measures the effectiveness of a company’s operations rather than asset performance.
Service Reservoirs Service reservoirs having greater than 5% samples fail against coliform Table DWI 1 Annual Report to Ofwat Operational Comments as for treatment works above.
Distribution Extent of Low Pressure problems: the number (%) of properties at risk from receiving unacceptably low pressure. (IL35a) Companies. Collected at June Return and compiled from information at zone level. Asset/Operational An established measure of the capacity of zones to meet pressure standards. May be an indicator of asset performance or operational performance
  Interruption of Supply to Customers (IL35a). The number of customers experiencing unplanned interruptions greater than 12 hours Companies. Collected at June Return and compiled from information at zone level. Operational An established measure of availability. This measures a combination of the number of bursts and the management of the event in reducing the impact on customers.
  Burst rate: the number of bursts per 100 km of main Companies. Collected at June Return and compiled from information at zone level Asset A measure of the performance of the assets.
  Quality compliance: the number of zones failing against the iron pcv Companies. Table DWI 1 Annual Report to Ofwat Asset Quality investment has been targeted at those zones identified for S19 Undertakings and work funded by Quality capex, although some companies apportion part to capital maintenance.

 

Table A2. Existing Serviceability Measures used by the Drinking Water Inspectorate

Asset Type

Measure

Data Source and Mode of Collection

Type of Measure

WS Atkins Comments
Treatment Works The number of treatment works with coliform detected and the number and percentage of determinations containing coliform From Companies

DWI Annual report		 Operational Any coliform failures are normally reported to the Inspectorate as an event, following the requirement of IL13/99. The Inspectorate will review the circumstances surrounding the reported event. This will be reported as an incident as defined in IL13/99.

Our analysis of a sample of incident reports prepared by the Inspectorate shows that failure against the coliform standard is due to management shortcomings or failure of chlorination plant due to minor operational reasons.

We concluded that while the coliform standard is a useful measure of serviceability to customers, it measures the effectiveness of a company’s operations rather than asset performance.

Only summary water quality information is reported to DWI. Other regulatory compliance data is held with companies and is available for inspection.
  The number of treatment works with coliform detected and the number and percentage of determinations containing coliform From Companies

DWI annual Report		 Operational Same comments as for Coliform
Service Reservoirs The number of service reservoirs with coliform detected in more than 5% of samples and the number and percentage of determinations containing coliform From Companies
DWI Annual Report		 Operational Comments as for treatment works above. Only summary water quality information is reported to DWI. Other regulatory compliance data is held with companies and is available for inspection.
  The number of service reservoirs with coliform detected in more than 5% of samples and the number and percentage of determinations containing faecal coliform From Companies
DWI Annual Report		 Operational Comments as for treatment works above. Only summary water quality information is reported to DWI. Other regulatory compliance data is held with companies and is available for inspection.
Distribution The number of zones non-compliant against pcv for each parameter tested and the number and percentage of samples failing against pcv From Companies
DWI Annual Report		 Asset/Operational The number of zones covered by S19 Undertakings are also identified. Only summary water quality information is reported to DWI. Other regulatory compliance data is held with companies and is available for inspection.
  The number of events and incidents reported Companies report to DWI. The Inspectorate investigates and classifies as incident or event. Operational A measure, which has been in place since 1994 [check] but improvements in the quality of reporting, has resulted in an increase in reported incidents. Following investigation, the incident may be considered for enforcement action or prosecution.
Common Measures Customer surveys DWI Customer Perception Survey in 2000.   A quantitative and qualitative survey identifying the main areas of concern and dissatisfaction.

 

Appendix B

Sources of Quality and Non-Quality Information

Sources of Water Quality Data

  Data Item Source and Type Comments
1 Summary Compliance Data DWI Annual Reports and supporting files

Existing regulatory information - published		 Information reported by companies in summary format only following IL3/98. Data reported by prescribed parameter for each treatment works, service reservoir and water supply zone:
  1. the number of samples taken
  2. the number of samples exceeding the PCV or relaxed PCV
  3. the percentage exceeding the PCV or relaxed PCV
  4. the mean, minimum and maximum values

following the requirements of the Water Supply (Water Quality) Regulations 1989. Data for some ‘supply points’ may also be reported.
2 Full Compliance Data Public records within each water company

Existing regulatory information - held by companies		 Each company is required under Section 29 of the Water Supply (Water Quality) Regulations 1989 to maintain records of all water samples taken in complying with these Regulations.

The extent of data is laid down by the sample frequencies in Schedule 3 of the Regulations.

For treatment works, sampling is limited to microbiological parameters and residual disinfectant. Sampling at service reservoirs is limited to microbiological parameters and residual disinfectant.

Sampling in distribution is more extensive as set out in Tables 1 to 4 of Schedule 3. The number of samples is related to the parameter, zone size and whether a reduced, standard or increased frequency is required. In general, reduced frequencies relate to parameters that are unlikely to have much influence on serviceability; the same cannot be said for standard frequencies.

Data is kept in electronic form but the precise format varies from company to company. The ease of retrieval and analysis will also vary from company to company.

We have not assessed the format and ease of retrieval of compliance data from companies.
3. Operational samples Water Companies

Advanced operational information - held by companies		 Normal practice is to maintain all water quality results within companies’ Laboratory Information Systems.

Operational samples may be taken for several purposes.
  1. for regular monitoring and control of treatment processes,
  2. in response to specific operational problems, or
  3. in response to incidents or customer complaints

Operational samples should be used with care, in the knowledge of the original purpose of the samples.

We would need to confirm the extent of operational sampling in typical water companies, by purpose category.
4. PPRA assessments Water Companies with summary returns within DWI

Existing regulatory information - held by companies		 Planned sampling programme within defined sub-zones to justify the need for mains renovation work, and to demonstrate success of completed schemes.

Sample information would normally be archived on the laboratory information database (Data Items 2 and 3 above)
5 Incident Reporting DWI

Existing regulatory information - held by DWI		 Summary information held on spreadsheets for each report year. Inspector’s reports on each incident held on file in document format.

It is difficult to relate incidents to specific treatment works, service reservoirs and water quality zones. The background information in the Inspectors reports illuminates serviceability issues.
6 Relevant Documents DWI

Existing regulatory information - published
  1. New Water Quality Regulations - consultation document
  2. IL13/99 Incident Reporting
  3. IL17/2000 and IL9/99 on assessment of water quality index
  4. IL13/2000 and 4/4000 on cryptosporidium monitoring,
7 Iron pick-up in distribution Companies

Existing regulatory information - held by companies		 Some information available from PPRA studies related to the S19 work.

 

Sources of Non-Quality Data

  Data Item Source and Type Comments
1 Treatment Works by size band and process type. Ofwat Annual Returns
Existing regulatory information - published

Ofwat PR99 Submission H Existing regulatory information - held by companies and Ofwat

DWI

Existing regulatory information - held by DWI		 Annual Return Information available on CD.

Some companies chose not to place part or all of their Submission H in the public domain, limiting information available in the Ofwat Library.

Schedule 5 of the Annual Information Return includes output but not process.
2. AverageTreatment Works Outputs EA

Existing regulatory information - held by companies and EA		 EA report on companies Water Resource Plans and the Deployable Output by resources area
3. Peak Treatment Works Outputs EA

Existing regulatory information - held by companies and EA		 EA report on companies Water Resource Plans and the Deployable Output by resources area
4. Treatment Works, Service Reservoir and Distribution Condition and Performance Assessments Ofwat PR99 Submission H

Existing regulatory information - held by companies and Ofwat		 Relates to observations made in 1998

Some companies chose not to place part or all of their Submission H in the public domain, limiting information available in the Ofwat Library.
5. Number of zones by size band (population) DWI

Existing regulatory information - held by companies and DWI		 Schedule 5 of the Annual Information Return includes population but not mains length.
6. Length of main by zone Company

Basic operational information - held by companies		 DWI has mains length for S19 zones only.
Ofwat collects total mains length in table 11 of the June Return.
An efficient company should have mains length by zone from its GIS systems.
7. Serviceability Measures (pressure/ interruptions) at zone level Company

Basic operational information - held by companies		 Ofwat collects information on DG2 (pressure) and DG3 (interruptions) at company level in the June Return.
An efficient compa