Post-occupancy audits in P3 construction projects

How to ensure accountability for a property's long-term operating performance
Thursday, March 2, 2017
By Joe Pendlebury

Popular in the UK and Canada, the public-private partnership (P3) model for financing construction projects is making inroads into the U.S. market as an alternative to traditional development methods. A P3 can save time, money, and reduce risk to the government by having a single contractor design, build, finance, and operate and maintain a facility over an extended period of time — anywhere from 10 to 40 years. A reserve fund covers the maintenance expenses. When carefully and transparently planned, P3s offer some significant advantages to owners and communities.

Value for money: Taking the long view

A key part of that planning is the value for money (VfM) assessment. Done at the outset of a project, in the design and construction phases, the projections of a building’s performance can provide powerful, persuasive evidence of efficiency. However, these estimates frequently tell only part of the tale. They do not sufficiently examine the property’s operating performance over the long term, compromising their accuracy.

Unlike P3 pre-construction VfM assessments, the analysis of these “legacy” costs can provide crucial data but is often incomplete. This is due in part because P3s are still relatively new to the U.S., but also because North America has not yet implemented a standardized, comprehensive audit and monitoring system. Until one is in place, the P3 contract should clearly outline the processes that contribute to a complete, long-term VfM assessment.

This is especially important for projects that include an operations and maintenance role than lasts 25 years or more. The P3 contract should be worded so that the owner has access to a sufficient amount of operations and maintenance information to ensure the project’s legacy costs also provide value for money.

VfM essentially combines qualitative and quantitative factors. Benchmarking — a system of comparing the measured performance of a facility to itself, its peers, or established norms — is the principal tool used to yield data that inform VfM assessments. The benchmarking process typically accounts for fluctuations in the costs of labour, materials, equipment, and energy.

To capture the most complete picture of a project, the value for money test examines efficiencies on many levels. It evaluates the project as a whole (including equity, cost of debt, etc.), and compares it to a traditionally funded project of equivalent nature. Analyzing the following separate components will enhance the accuracy of the data:

  • Hard and/or soft facility management
  • Life-cycle cost/sinking fund/reserve fund
  • Sufficient equity and appropriate penalties to ensure the private-sector partner performs according to the specifications in the latter years of the project
  • Appropriate performance/availability mechanisms
  • The optimal level of risk transfer

Benchmarking: Assessing performance

It’s best to do the first round of benchmarking measurements after the building has been occupied and running for at least six months — depending on the complexity of the facility. As a rule of thumb, the benchmarking should take place after the initial commissioning, troubleshooting processes and fine-tuning of the building have been completed. This early evaluation will confirm that key performance indicators (KPI) can be documented and achieved. In many jurisdictions in Canada and the UK, P3 projects have a five-year benchmarking requirement.

It’s important for both public- and private-sector parties to ensure that critical aspects of the agreements are subject to periodic benchmarking reviews because P3 agreements are usually in place for significant periods of time. To assure an impartial analysis, these reviews should be performed by an independent audit team. If the benchmarking exercise does not yield an acceptable level of comfort for the public-sector party, it may order an enforced market testing of the product or system, if the contract contains provisions permitting such an action.

It’s critical to note that the post-occupancy audit should not be just a tally of the bills paid by the concessionaire; such a superficial accounting provides a one-dimensional view of the project. An audit prepared by a quantity surveyor with experience in building condition and reserve fund studies will establish a comprehensive, integrated understanding of the building’s performance and identify any operational deficiencies.

An analogy: Adding up household utility bills may show a homeowner that he’s spending $200 a month on electricity, but to put that information into context, a homeowner should seek advice from an authority. That authority may be able to tell him that his monthly expenditure on electricity should be only $50, and that there may be a problem with his house.

Quantitative feedback on a building’s performance should be augmented with qualitative information. Among the questions that can add depth to the VfM assessment:

  • Is the quality level appropriate? In other words, do the service level agreements and/or performance specifications accurately reflect initial and life-cycle client requirements?
  • Is the operation integrated and efficient?
  • Does the original purpose — and consequently, the specifications — for the project remain the same? Or has the project brief changed?
  • Are there current KPIs available for review, and, if so, are they appropriate to the project in its current form?

Once established, the benchmark will enable future audits to examine the efficiency of the systems and operations to see if further improvements can be made, to optimize the benefit to all parties. This may include replacing some of the suppliers, if the value or service levels can be improved by going back out to the competitive marketplace.

This can only be achieved if contract conditions can be formulated and carried out at the procurement stage to allow for the roll out of any proposed efficiencies. One scenario might be to confirm that the contracted KPIs have been achieved, with any optimizations to be completed as a separate exercise.

Another important provision in the paperwork: Both the owner and the concessionaire should be protected if the property is transferred during the period of the agreement. An independent third party should review the legal and economic provisions contained in the transfer agreement. It is quite common for P3 properties to change hands, whether the asset is sold or consumed by a merger.

Banking of performance data

A well-structured P3 contract with clear provisions for determining a building’s legacy costs is not only useful to the immediate partners on the project, but could advance the entire construction industry. Creating a database of audits from various projects would empower government partners to draw on this evidence to inform future contracts, KPIs, and operations and maintenance standards. The potential benefits of the information wouldn’t be limited to P3 jobs, either; such data could be applied to traditionally tendered projects, as well — making it a truly valuable proposition.

Joe Pendlebury is senior vice president of Rider Levett Bucknall’s North American practice and is responsible for leading the Toronto office. Joe is a Professional Quantity Surveyor with nearly 30 years of national and international project experience.

Leave a Reply

Your email address will not be published. Required fields are marked *