Ensuring that real-world commissioning tests are properly executed can benefit both equipment end users and suppliers
Every capital equipment project comes to a point when operations teams expect payback to begin on the investment. How soon those benefits are realized is driven by the effectiveness of the handoff from the equipment supplier. Those projects that rely only on acceptance tests to signal production readiness tend to experience delays, higher costs and lost output. The most effective handoffs involve comprehensive commissioning. The difference between acceptance and commissioning is key.
Acceptance tests are designed to show that a solution meets a set of objective contractual requirements. These are tangible deliverables, are not subject to interpretation, and are demonstrated according to tightly controlled conditions. Commissioning, however, is a real-world test using actual production resources (materials, staff and infrastructure) that are subject to uncertainties within an industrial ecosystem. Effective commissioning tests must incorporate elements of variability and interdependency, which are difficult to replicate in simpler acceptance tests (Figure 1).
Consider a project where a new piece of capital equipment works perfectly throughout both factory and site acceptance tests. The equipment is handed off to production operations and final payment is approved. Then, production starts and one of the first batches of material has properties that are slightly out of tolerance. The material-handling system fails to effectively convey the material, forcing a shutdown as the engineering team and supplier perform troubleshooting. Weeks of production output are lost. And both the equipment purchaser and supplier suffer unplanned labor and material costs.
This is a common example — where fully compliant equipment fails to deliver acceptable results because of interdependent activity or variability in other parts of the operation. In fact, there are many sources of troublesome interdependency, including the following:
- Inadequate training of maintenance technicians can result in faulty repairs that cause damage to sensitive components
- Utilities infrastructure may fail to provide sufficient stability in line voltage, generating controller faults that cause downtime
- Material shortages can force equipment into a turndown condition outside of its design range, causing quality defects and reduction in throughput
While these issues can prevent new equipment from delivering expected benefits, they are also outside the typical scope of the supplier. The inability for the equipment to meet business objectives turns the capital investment into a liability (even if only temporarily), damaging the supplier’s reputation and creating a lose-lose situation for buyer and seller.
For new capital equipment to reliably meet business objectives, there must be a commissioning period that accounts for both technical capabilities and operational interdependency. Effective commissioning involves five best practices that must be considered:
This article details each of these best practices and provides guidance in implementing them during capital project execution.
Define flowdown requirements
Planning for commissioning begins as specification requirements are being defined. Business objectives defined as part of the investment rationale are the basis for critical-to-quality (CTQ) attributes and, ultimately, detailed requirements (Figure 2). This flowdown of requirements provides a roadmap for commissioning. In commissioning tests, the goal must be to show that equipment requirements, CTQ attributes and business objectives are met.
For example, a business that seeks to increase its product throughput capacity might flowdown the business objective to equipment requirements as indicated in Figure 3. On the left side of the diagram, in the green shading, are typical technical requirements that would be in an equipment specification. To the right, in tan shading, are operational interdependencies that can have a significant impact on whether the throughput increase will ever be realized.
Note that CTQ attributes are measurable, even those related to operational interdependencies. It is vital that the commissioning plan identifies how CTQ attributes will be measured and the acceptable range of values. For instance, flowrate requirements may be measured at a particular point in the system and may apply only at a specific turndown ratio. “Reliable utilities” might be a measurement of voltage variation or cooling water flow and temperature.
Suppliers need to be made aware of operational interdependencies early in the design process. And commissioning tests must be set up in a way that assures interdependencies are addressed. This way, the buyer understands the standard to which adjacent factors must be managed to achieve the business objective. Even more important is the potential for substantive changes in equipment design that might mitigate some of the impact caused by interdependencies.
Create a comprehensive specification
Achieving a rapid and complete startup of new capital equipment demands clarity on what constitutes success. The success criteria are outlined in a specification that, too often, focuses heavily on technical details, and not enough on business performance. The only true measure of success is performance to the investment rationale established by business leadership.
A strong specification will incorporate both technical and performance criteria. Technical criteria include process flows, applicable standards, interfaces, system components, controls strategy and documentation requirements. To address performance requirements, specifications should also include several additional elements, which are detailed in the following sections.
CTQ attributes. These are measurable attributes that establish how the system is to meet business objectives. For instance, if the business objective is to achieve higher throughput, one of the CTQ attributes would be a certain percentage of uptime. Detailed technical requirements might address some aspects of this requirement, but by alerting the supplier that the entire system must achieve a defined uptime, better design decisions can be made.
Commissioning tests. The specification should be clear on which real-world tests will occur after acceptance tests. The request for quote can ask for vendor support of commissioning tests to be a separate time and materials quote, but it is important for the supplier to understand how success will be defined during commissioning. This section might address staffing of the equipment, range of materials to be tested, deliberate stress tests (for example, voltage fluctuation or pressurized air shutoff) or recipe variants.
Roles and responsibilities. The specification needs to define roles and responsibilities that demand a partnership approach between the supplier, technology team and operations staff. For instance, shop floor operators can provide invaluable information about the legacy processes. Maintenance technicians can explain their methods for executing preventive maintenance and any issues associated with plant infrastructure, and production schedulers can provide information on the reliability of material flows and expected shift patterns. This is a strategy whereby suppliers understand potential sources of variation and account for them throughout the design, build, install and commissioning tasks.
High-risk proof-of-concept tests. Many new equipment designs incorporate a technical “leap of faith.” This could be reliance on a brand-new sensor or processing a larger batch than ever before. To de-risk the project, proof-of-concept tests should be required as part of the specification. Besides providing confidence that a high-risk attribute is largely reduced, these tests make supplier design teams smarter about the process their equipment will serve.
Documentation. Most specifications will have standard requirements for documentation, which may include operating manuals, maintenance instructions and drawings. However, operating manuals do not typically go far enough in explaining how the process will run. They tend to have an equipment-centric view. There should be a documentation requirement that speaks to the operator and the maintenance technician. If a particular parameter setting is critical to achieve consistent quality, it should be documented. If there is a safer sequence for disassembling a part of the system, the manuals should provide that guidance. Very few manuals have this level of detail and shop-floor personnel usually express frustration at the lack of completeness. Of course, suppliers are not necessarily process experts, so operations teams and suppliers must work together to achieve a higher standard of documentation.
Design for commissioning
The handoffs within the equipment purchasing process — from business leadership to engineering and then to a sourcing team — can obscure the business objective that drove the initial investment decision. As a result, suppliers typically do not fully understand the reason why many requirements exist. This disconnect can result in lower-performing equipment.
A more effective way for equipment purchasers to engage with suppliers is to incorporate commissioning requirements into the design phase. This does not relieve the purchaser from being clear on technical capabilities. Instead, it expands the interactions between the buyer and seller to include performance-related needs and establishes a design team that will typically work together throughout the project.
As shown in Figure 4, operational integration is a set of activities undertaken by the design team to embed real-world effects into design concepts. It contemplates the environment into which equipment will be installed and ensures that interdependencies are considered by designers. This is where CTQ attributes are clarified and made part of the commissioning performance tests.
Commissioning should be made part of the contractual framework, with obligations of the supplier and purchaser clearly established. By capturing these obligations during the design phase of the project, the likelihood of commissioning success increases. Many of these contractual obligations cannot be outlined prior to the start of design, so they may be incorporated into a separate contract. Also, since many of the commissioning tasks have a higher level of uncertainty, they may lend themselves to a time and materials contract structure — as opposed to the fixed-price contract typically used for equipment purchases.
Finally, the commissioning strategy and plan must be detailed during the design phase. Factory and site acceptance can be viewed as the baseline set of commissioning activities, where equipment capabilities are demonstrated. But additional commissioning specifications must be outlined, capturing the operational interdependencies discussed above. Startup test details would be part of the commissioning plan, to include resources and timing. Materials needed in commissioning tests may take some time to prepare or source. By making these plans during the design phase, schedule impact can be minimized.
De-risk the project
When new capital equipment does not meet business objectives, it is usually due to component failures or process disruptions. When designing robust equipment, suppliers should evaluate these sources of variance and identify methods for mitigating their impact. When commissioning equipment, these same variations should be part of testing. Two design tools, borrowed from product engineering, that can help equipment project teams, are the parameter diagram (p-diagram) and the failure modes and effects analysis (FMEA).
The p-diagram was developed by Joseph Juran to take inputs from a system and relate those inputs to desired outputs of a design. Used in many signal-processing applications, the p-diagram establishes a set of ideal inputs and ideal outputs, then guides the engineer to consider factors that can disrupt the ideal process (Figure 5). Ideal inputs, process description and ideal outputs are defined in the best case, where everything works without disruption. To keep the process running as close to this ideal as possible, control factors are applied. An example would be valve position in a flow application.
Noise factors are sources of variance likely to impact a process. Typically, these factors will involve the environment, infrastructure stability, human factors, equipment condition, materials variations, degradation effects or interactions with other equipment. Developing a p-diagram forces design teams to consider how noise is to be addressed. One option is to modify the design to convert a noise factor into a control factor.
By identifying error states, the p-diagram allows designers to consider what happens to the system when things go wrong. Commissioning plans will typically test each type of error listed on the p-diagram to show that the system achieves a desired error state.
Several specific noise factors identified in the p-diagram that should be evaluated during commissioning are described in the following paragraphs.
Human impact. Operators that run commissioning tests should be those who will be responsible for production after equipment handoff is complete. They should run equipment without guidance from engineering or the supplier. After full handoff to the operations team, operators will make mistakes. Commissioning should confirm that equipment is robust enough to handle these mistakes. And, as commissioning tests are executed, gaps in training will become clear, allowing the operations team to engage the supplier on a more effective training process.
Material variabilities. Identical materials sourced from different suppliers can behave differently when processed. Individual suppliers can have process disruptions that inject new material properties not captured in procurement specifications. These can impact equipment performance. Upstream processes inside the factory may have their own variations that are passed downstream. During commissioning, testing should include a range of material characteristics and properties. Tests should be run on known out-of-spec materials to determine how the new equipment will respond.
Infrastructure disruption. Any infrastructure that plays a critical role in equipment operation (such as power, cooling water, vacuum, air pressure, heating, ventilation, air conditioning and so on) should be considered in commissioning tests. For instance, disruptions can occur in some processes if the room temperature rises on an abnormally hot day. Systems must be able to survive power failures and return to a known state upon power restoration. These types of issues must be resolved if they prevent equipment from meeting business objectives.
Volume swings. Fluctuations in factory material flow can impose a turndown condition on a piece of equipment that is at or beyond the design turndown ratio. Forcing an out-of-spec turndown condition during commissioning allows assessment of impact on quality and reliability.
Degradation effects. Most maintenance documentation will identify consumable components within systems. As these components degrade, the equipment must continue operating. Well-designed commissioning tests can evaluate how the system may operate with clogged filters or leaky seals. These are examples of conditions that the system will eventually encounter. Controls must be able to handle these situations without damage to equipment or hazard to personnel.
The failure modes and effects analysis (FMEA) is a decision-support tool developed by the U.S. military in the late 1940s to improve the reliability of its defense systems. In the FMEA, project teams identify likely ways in which the system might fail, the impact of those failures, why they might occur, how likely it is to happen and whether there is warning beforehand (Figure 6). Failure modes are given a score for each characteristic and then prioritized. Failures that have the biggest impact, happen most frequently and occur without warning will have the highest priority, expressed as the product of three numerical scores: severity, likelihood and detectability.
The FMEA is not just an assessment tool. It is a means for identifying design issues and driving toward solutions that will reduce the risk to an acceptable level. By incorporating the FMEA into capital equipment design, suppliers can deliver more reliable solutions. However, it is critical that the supplier and operations team be involved in developing the FMEA to create a common vision of how the system is likely to perform and identify acceptable methods for dealing with failures.
Having performed an FMEA, the supplier should demonstrate that design mitigations were effective within the commissioning period. Specific failure modes should be stress-tested with passing results. When an installed solution is shown to survive the highest-priority failure modes, the risk of having outages that negatively impact business objectives is greatly reduced.
Extend supplier engagement
On fixed-price contracts, suppliers have a natural incentive to complete their work, close out the project and receive final payment — that payment usually represents their profit. Furthermore, equipment buyers have their own incentive to begin showing financial gain from significant funds invested. For these reasons, equipment is usually handed off to operations before it is fully ready to interact with all of the surrounding and interdependent systems.
If a deliberate commissioning effort is not planned and executed, a form of “casual commissioning” still occurs. It just happens in a slower and more costly manner. When operations teams encounter issues in the course of daily activity, instead of deliberate commissioning tests, skilled resources must be reallocated from other priorities. In extreme cases, suppliers are pulled back into the plant for support — possibly triggering a warranty dispute. Many suppliers will assume this cost in the name of customer-centricity, reducing their profit, even when it is not an issue within their control. This casual commissioning process can be costlier than the initial investment in equipment and usually is a lose-lose scenario.
A better way to achieve full performance of new capital equipment is to add a separate extended service engagement to the end of the purchase contract. This extension should include services by the supplier to support real-world commissioning. The contract would be customized to the type and complexity of equipment, and it would typically have multiple phases, described as follows:
- The first phase would be intensive and onsite engagement of supplier technical resources to resolve any equipment issues encountered under deliberate testing of noise factors
- The second phase would be reduced onsite presence of a supplier project manager to monitor performance and work with operational teams to resolve issues associated with interdependencies (for example, maintenance procedures, accurate work instructions or scheduling anomalies). If new technical issues arise, the project manager can call in resources from the technical team at the supplier
- The final phase might be a longer-term service contract that allows for “pay-as-you-go” supplier engineering assistance if less common performance issues occur
The definition of extended support scope would occur in the design phase, as described above. Performance targets would be agreed upon by both parties. As there are many factors involved in meeting performance targets, the supplier cannot be expected to provide a guarantee. However, a sliding payment scale can be incorporated to provide incentive for the supplier to help drive the system to target performance.
Improved commissioning processes demand a renewed alignment between purchaser and supplier objectives. The frustration purchasers have when their new capital equipment does not work as expected is usually mirrored within the supplier’s business. Most suppliers care that their solutions deliver investment returns for their customers. They also feel frustration when circumstances beyond their control degrade performance. By following commissioning best practices, capital equipment investments can more reliably achieve their investment objectives, creating a win-win scenario for the purchaser and supplier. ■
Edited by Mary Page Bailey
Peter Kalish is director of business development at O’Brien & Gere Engineers, Inc. (OBG; 94 New Karner Road, Suite 106, Albany, NY 12203; Phone: 518-724-7272; Email: firstname.lastname@example.org; Website: www.obg.com). He has spent 25 years working in industrial operations and has been responsible for purchasing and commissioning hundreds of millions of dollars in enterprise applications, production equipment and facilities infrastructure. Prior to OBG, Kalish held several technical and leadership positions within GE, including work in the company’s fuel cell, energy storage and nuclear divisions. He holds B.S. and M.S. degrees in nuclear engineering from Massachusetts Institute of Technology (MIT) and is a certified Six Sigma Green Belt.
Taking commissioning and startup into consideration during the early design phase can result in more predictable project schedules and more…
Prudent planning and scheduling during a project’s front end can lead to more expedient commissioning and startup activities The cost…
Following a five-step process enables better outcomes for fluid system control Most often, operators and technicians will interact with components…
Research and development in the brewing industry is peaking, driven by efforts to improve quality, efficiency and sustainability Within the…
Taking commissioning and startup into consideration during the early design phase can result in more predictable project schedules and more…
Five Reasons Why Chemical Companies Are Switching to Tunable Diode Laser Analyzer Technology
Simplify sensor handling and maintenance with ISM
Three reasons to measure pH in-line
ABB Ability™ technology to transform BASF rotating equipment into intelligent machinery and improve uptime and reliability