Development of control quality factor for HVAC control loop performance assessment—II: Field testing and results (ASHRAE RP-1587)

Introduction

This article is the third paper from the research project RP-1587, focusing on presenting a comprehensive field test of the proposed control quality factors (CQFs; i.e., CQF-Harris and CQF-exponentially weighted moving average [EWMA]) and testing results. Firstly, the simulated control loops and real control loops are evaluated for offline assessment. Then, the field experiment implemented for different HVAC control loops is assessed online using the proposed CQFs. Test results show that the proposed CQFs are capable of adequately and effectively assessing the HVAC control loop performance. The methodology of obtaining those CQFs is provided in the companion paper: Development of Control Quality Factor for HVAC Control Loop Performance Assessment I—Methodology (ASHRAE RP-1587) (Li et al. 2019).

Buildings account for 40% of the total energy consumption in the United States (U.S. Department of Energy; Dechert [4]). According to the U.S. Energy Information Administration ([30]), commercial buildings in the United States annually contribute to 18 quadrillion British thermal units (quads) of primary energy. The increasing population and urbanization demand more buildings and lead to more energy consumption. How to reduce energy consumption effectively is still challenging. Various building technology studies have been published, such as occupant behavior (Hong et al. [13]; O'Neill and Niu [25]), energy conservation measures (Qian et al. [26]), sensors and controls (Li and Wen [18]), etc. Among them, controls were proven to be effective in reducing energy consumption (Dalamagkidis and Kolokotsa [2]; Lau et al. [17]). Control loops are widely used in the process industry and HVAC systems. There are multiple types of control systems for the building and HVAC industry, such as supervisory control (Wang and Ma [32]), model predictive control (Afram and Janabi-Sharifi [1]), fault-tolerant control (Wang and Chen [31]), and CO2-based demand control ventilation (O'Neill, Li, Zhou, et al. [22]). A typical HVAC system might contain multiple control loops, and most are a proportional–integral (PI) type. Poor control loop performance can lead to increased building energy consumption or shorter equipment life. Fernandez et al. ([9]) showed that applying advanced controls and sensors could result in energy savings of as much as 30% for commercial buildings. Similar conclusions can be drawn for energy savings benefits through utilization of advanced controls in self-tuning for commercial buildings (Fernandez et al. [6], [7], [8]).

There has been no mature real-time control loop performance assessment (CPA or CPLA) metric used in HVAC control loops. A typical HVAC system has tens of control loops working concurrently to maintain the desired built environment (e.g., temperature, pressure, etc.). A comprehensive and critical literature review was conducted that covers the CPA metrics in the process industry and possible applications for the HVAC control loops with implementations using low-memory direct digital control (DDC) controllers (O'Neill, Li, and Williams [22]). There are many CPA algorithms available for process control, such as statistical methods (Horch [14]; Horch and Heiber [15]), dynamic response methods (Hägglund [11], [12]; Miao and Seborg [20]), closed-loop identification (Salsbury [27], [28], [29]), etc. However, limited CPA methods are applied in HVAC control systems. This is mainly because the HVAC industry is cost sensitive to deploying advanced CPA algorithms directly from the process control industry.

Two control loop performance assessment indices were proposed in ASHRAE RP-1587 (see O'Neill, Li, Williams, et al. [23]): control quality factor (CQF)-Harris, and CQF-exponentially weighted moving average (EWMA). The proposed CQF indices are capable of detecting abnormal behaviors (e.g., set-point off track) of HVAC control loops. The reversal behavior is also incorporated by using the reversal index (O'Neill, Li, Williams, et al. [23]). The reversal index will always overwrite the calculated CQF-Harris and CQF-EWMA as long as reversal behaviors are being detected. The CQF-Harris is based on the process control theory considering the unmeasured disturbances and is relatively complex for the online implementation in a typical HVAC controller. The CQF-EWMA is from the signal filtering and is relatively simple. For detailed calculations of CQF-Harris and CQF-EWMA, please refer to the companion paper with the title: Development of Control Quality Factor for HVAC Control Loop Performance Assessment I—Methodology (ASHRAE RP-1587) (Li et al. [19]). The formula for calculating the CQFs and the reversal index are summarized as follows. For details on obtaining each variable, please refer to the companion methodology paper.

(1)

Graph

(2)

Graph

(3)

Graph

where is the CQF-Harris index, is the square of minimum variance, is the variance of control outputs, is the CQF-EWMA index, is the scale factor for CQF-EWMA, is the error ratio for CQF-EWMA, is the reversal index, is the variance of the unmeasured disturbance (white noise), and is the EWMA value of control loop outputs.

In this third paper from ASHRAE RP-1587, comprehensive testing and evaluations of the proposed CQFs are presented with the results from both simulated control loops and real control loops through offline assessments and online assessments. The types of controls loops include room air temperature control under cooling/heating modes; air handling unit (AHU) supply air static pressure control; and heat exchanger supply water temperature control.

• The rest of this article is organized as follows:
• Evaluation with model-based control loops
• Offline evaluations with real control loops
• Online evaluations with real control loops
• Discussions and conclusions.

Evaluation with model-based control loops

The performance of the proposed CQFs was first tested and evaluated using data from a simulated building. A total of 16 simulated HVAC control loops were evaluated in ASHRAE RP-1587 (O'Neill, Li, Williams, et al. [23]). Only the evaluations of the room air temperature control loop using the proposed CQFs will be presented in this article.

Model-based evaluation

A dynamic building model with a variable air volume (VAV) system was developed using Modelica (Fritzson [10]) due to its flexibility in dynamically modeling control components. This model is used to generate simulated data for testing the proposed CQFs. The simulation engine is Dymola 2015 (Dassault Systèmes [3]). This case study utilizes the free open-source LBNL Modelica building library (Wetter et al. [33]). The details of this building model are given in the first paper from the research project (Li et al. [19]). The outputs and set-points of the room air temperature control in the cooling mode are given in Figure 1 for two continuous days. Only data from the occupied hours (e.g., 7 a.m. to 7 p.m.) are presented in this figure. Figure 2 shows the room supply airflow fraction (i.e., the ratio of supply airflow rate and the design maximum airflow rate) with the minimum airflow fraction set-point for the same period.

PHOTO (COLOR): Fig. 1 Room air temperature control in cooling mode (VAV).

PHOTO (COLOR): Fig. 2 Room supply airflow fraction in cooling mode (VAV).

We can see that the room air temperature on the first day is not very good until 1:00 p.m. The temperature maintains the set-point very well from 9:00 a.m. on the second day, with only small fluctuations. The calculated CQF index values are shown in Figure 3. The associated assessment scales are listed in the table next to plots in Figure 3.

PHOTO (COLOR): Fig. 3 Room air temperature control loop assessment (VAV cooling mode).

From Figure 3a we can see that on the first day, the CQF-Harris decreases from 1 at 7:00 a.m. to 0 at 1:30 p.m. This decrease indicates that the control loop is approaching the set-point slowly. At about 12:00 p.m., the CQF-Harris jumps from the excellent scale back to the fail scale. After 1:30 p.m., the control outputs are well maintained to the set-point, which is indicated by a CQF-Harris index value close to 0. This indicates excellent control performance. After 6:30 p.m., the CQF-Harris rapidly increases to 1. On the second day, the CQF-Harris index decreases rapidly from 1 at 7:00 a.m. to about 0 at 10:00 a.m. After 10:00 a.m., the control loop has CQF-Harris index values close to 0, which indicates excellent control.

Similarly, the CQF-EWMA assessment (Figure 3b) of this control loop on the first day decreases from 1 at 7:00 a.m. to close to 0 at 1:00 p.m. At 12:00 p.m., the CQF-EWMA downgrades from the excellent scale to the good scale. Beginning at 6:30 p.m., the CQF-EWMA index values increase rapidly to 1. On the second day, the CQF-EWMA decreases from 0.9 to about 0 and then remains almost constantly near 0, with small variations.

We see that both indices can recognize the control loop performance with corresponding scales regardless of the differences in scale values between the CQF-Harris and CQF-EWMA.

Figures 4 and 5 show the hourly weighted control loop performance for the room air temperature control by the same VAV box. Table 1 shows the CQF scores for the control performance scales correspondingly. For the first day between 12:00 p.m. and 4:00 p.m. the hourly averaged control performance is in an excellent scale (CQF-Harris). For the second day between 10:00 a.m. and 7:00 p.m., the hourly averaged control performance is also in an excellent scale (CQF-Harris). The control loop performance is similar using the CQF-EWMA.

PHOTO (COLOR): Fig. 4 Weighted control performance for room air temperature control (CQF-Harris).

PHOTO (COLOR): Fig. 5 Weighted control performance for room air temperature control (CQF-EWMA).

Table 1. Weighted CQF scores.

Scales	Scores
Excellent	(4, 5]
Good	(3, 4]
Fair	(2, 3]
Bad	(1, 2]
Fail	[0, 1]

Offline evaluations with real control loops

The proposed two CQF indices (i.e., CQF-Harris and CQF-EWMA) were first tested and evaluated using offline data from real HVAC control loops in an office building near Chicago, Illinois (O'Neill and Eisenhower [21]). A total of 168 HVAC control loops were evaluated with the proposed indices (i.e., CQF-Harris and QF-EWMA; O'Neill, Li, Williams, and Zhou [23]). Only the evaluations for two control loops are presented in this article: heat exchanger hot water supply temperature control and AHU supply air static pressure control.

Heat exchanger hot water supply temperature control loop

Figure 6 shows the hot water supply temperatures (blue line) and associated set-points (red line) from a heat exchanger for 54 days with a timestep of 5 min. Figure 7 shows the heat exchanger supply water valve opening in the same evaluation period. We can see that the controlled device of the valve was opening normally. Figure 8 shows the assessment results using CQF-Harris (left) and CQF-EWMA (right) for the heat exchanger supply water temperature control loop. There are four stages of control performance. Firstly, between t = 0 and t = 500 mins, the CQF-Harris indicates that the control performance is excellent, with variations in good performance. The CQF-EWMA is similar to CQF-Harris. Secondly, both metrics identify the failed control performance for t = 500 to t = 520 min while the valve is closed (as indicated by 0% opening). This indicates that the failure of maintaining the set-point during this period is likely due to the valve. Thirdly, for t = 530 to t = 2200 min, the control loop performance is excellent with few variations from CQF-Harris. However, the CQF-EWMA shows the assessment scales from excellent to good for the control performance. Lastly, for t = 2200 to t = 2750 min, the CQF-Harris indicates the assessment scales varying from excellent to failed. CQF-EWMA is similar to CQF-Harris.

PHOTO (COLOR): Fig. 6 Heat exchanger hot water supply temperature control.

PHOTO (COLOR): Fig. 7 Heat exchanger supply water valve opening.

PHOTO (COLOR): Fig. 8 CQF assessment for heat exchanger water supply temperature control loop: (left) CQF-Harris; (right) CQF-EWMA.

AHU air static pressure control loop

Figure 9 depicts the AHU supply air static pressure control loop with the output of pressure (blue line) and associated set-points (red line). Figure 10 shows the supply fan speed in the same period. We can see that the fan speed was never at the maximum of 100%. Figure 11 shows the assessment results using the proposed CQFs. From time t = 0 to t = 2250 timesteps, the assessment scales varies drastically. From time t = 2,250 to t = 2600 timesteps, both CQFs shows the control performance to be failed.

PHOTO (COLOR): Fig. 9 AHU supply air static pressure control.

PHOTO (COLOR): Fig. 10 AHU supply air fan speed.

PHOTO (COLOR): Fig. 11 Assessment for AHU supply air static pressure control: (left) CQF-Harris; (right) CQF-EWMA.

Online evaluations with real control loops

The second group of evaluations of the proposed CQFs was conducted online for the selected control loops in a real test facility. A total of 17 HVAC control loops were evaluated during the field test at the Iowa Energy Center (O'Neill, Li, Williams, and Zhou et al. [23]). Only evaluations of two control loops are presented in this article: room cooling air temperature control and room heating air temperature control.

The test facility with the detailed experimental setup is described below.

Field test platform

Testing facility

The field test of CQF-Harris and CQF-EWMA was conducted at Iowa Energy Center's Energy Resource Station (ERS) located in Ankeny, Iowa (Figure 12). The Iowa Energy Center established the ERS to examine various energy-efficiency measures and demonstrate innovative HVAC concepts. The ERS is unique in that it allows for dynamic testing of an entire system within a controlled environment. The facility has laboratory testing capabilities combined with real building characteristics and is capable of simultaneously testing two full-scale commercial building systems side by side with identical thermal loadings and control schemes.

PHOTO (COLOR): Fig. 12 Energy Resource Station—southeast view.

The floor plan of the Energy Resource Station, as shown in Figure 13, includes three distinct and separate areas; the A test rooms, the B test rooms, and the general area. The A rooms are serviced by AHU A and the B rooms by AHU B. The exterior test rooms are mirrored side-by-side pairs of A and B with different cardinal exposures. The test rooms are identical in construction specifications. The general area consists mostly of office spaces and classrooms that are served by air handling unit AHU-1.

PHOTO (COLOR): Fig. 13 Energy Resource Station floor plan.

There are four test rooms in each set of A and B rooms: East A/B, West A/B, South A/B, and Interior A/B test rooms.

HVAC system

The CQF test was conducted in the B test system. The HVAC system for the B test rooms includes a central AHU and an overhead ducted air distribution that terminates with four room-level VAV terminal air unit boxes. There are two supply air diffusers downstream of the VAV box and one return air grille in each room. Each B test room also has a fan coil unit, which was not used in this test.

Figure 14 provides an overview of the test system HVAC plan. Each testing room is equipped with a pressure-independent, single-duct VAV box. Each VAV box has both a hydronic and electric reheat coil. The advanced control sequences suggested by the ASHRAE-RP-1455 (Kiriu and Taylor 2017) project were implemented on the VAV controllers.

PHOTO (COLOR): Fig. 14 Schematic of test room HVAC system.

The key specifications for the VAV boxes at ERS are listed in Table 2.

Table 2. ERS VAV box design specification.

Design item	Exterior test rooms	Interior test rooms
Unit type	Single duct, pressure independent	Single duct, pressure independent
Inlet size	9 in. (0.2286 m)	7 in. (0.1778 m)
Airflow—cooling design	1000 ft3/min (0.471947 m3/s)	460 ft3/min (0.217096 m3/s)
Hydronic coil flow rate	3.0 gallons/min (0.000189 m3/s)	2.0 gallons/min (0.000126 m3/s)
Electric coil capacity	5.0 kW (17,060.71 Btu/h)	2.0 kW (6824.284 Btu/h)
Electric coil, number of stages/kW per stage	Three/1.67 (5698.27714 Btu/h)	Two/1.0 kW (3412.142 Btu/h)

Internal load

Each exterior test room has six 2 ft × 2 ft recessed grid troffers, and the interior test rooms have four 2 ft × 4 ft recessed troffers for lighting. Each 2 ft × 2 ft fixture contains three U-shaped T8 fluorescent tube lamps sized at 31 W. The 2 ft × 4 ft troffers are three lamp fixtures and currently have lamps with varying Kelvin values. All of the test room fixtures have dimmable ballasts and are set up for two-stage lighting.

All eight test rooms are equipped with two-stage electric baseboard heaters that can be utilized to introduce false internal thermal loads simulating various usage patterns. They provide 100% sensible heat loading and can be operated in three modes as shown in Table 3.

Table 3. Stages of baseboard heat.

	Stage
Control mode	1	2	Total nominal power (W)
1	Off	Off	0.0
2	On	Off	900
3	On	On	1800

Additional false loads can be introduced into the test rooms by activating the fan coil units in either a heating or cooling mode of operation simultaneous with the overhead air distribution system.

Each testing room has an "android" to simulate people. These sheet metal cylinders have an incandescent lightbulb inside and can also provide controlled and regulated CO2 production through a solenoid valve and airflow regulator connecting to a central CO2 cylinder. A computer workstation is also available to simulate typical office equipment loads. The android systems are activated and deactivated as control points on the building automation system to follow any prescribed schedule. Table 4 indicates the control modes available.

Table 4. Occupancy simulator control modes.

	Control mode	Capacity per person	Activity level	Operation description
CO2	On	0.75 standard ft3/h (0.35 L/min) (0.01236 ft3/min)	Office work	CO2is produced at a controlled and regulated level
Off	0 standard ft3/h (0 L/min) (0 ft3/min)	Absent	The solenoid valve controlling CO2 flow to the android is closed
People	On	250 Btu/h sensible (72.3 W) 200 Btu/h latent (58.6 W)	Office work	Heat is generated with a lightbulb heat source, 75 W/person (256 Btu/h)
Off	0 Btu/h (0 W)	Absent	Relay controlling the heat source is turned off
Equipment	On	42 W—Computer (143.3 Btu/h) 46 W—Monitor (156.96 Btu/h)	Office work	Personal computer and monitor activated
Off	4 W—Computer (13.6 Btu/h) <1 W—Monitor (3.4 Btu/h)	Absent	Relay controlling office equipment deactivated, equipment in standby mode

Test DDC system

The ERS Test DDC System is based on the modern, commercial-grade BACnet-compatible Building Automation and Energy Management Platform. The system is a comprehensive Web-based, multiprotocol platform powered by the NiagaraAX Framework®. The open structure creates a common development and management environment for the integration of LONWORKS®, Modbus™, and other control standards. It also provides seamless and intelligent integration of the HVAC, lighting, access control, CCTV, and energy management systems, along with additional building systems.

The ERS Test DDC System has over 800 monitoring and control points trended at 1-min sampling intervals on a server. The system has a flexible graphical user interface that provides traditional building management functions such as scheduling, trending, alarming, historical data collection, and advanced energy management applications.

The direct digital controllers utilized by the ERS Test DDC system are specifically designed to control various building automation applications such as air handling units, chillers, boilers, pumps, cooling towers, central plant, fan coil units, unit ventilator, heat pumps, etc. The controllers use the BACnet® MS/TP LAN communication protocol and are BTL®-Listed as BACnet advanced application controllers or application-specific controllers. Additional features include large nonvolatile flash memory for applications and data storage, a built-in real-time clock with a rechargeable battery, a 16-bit analog/digital converter resolution for analog input channels, and a 12-bit digital/analog converter resolution for analog output channels.

These controllers can be custom programmed using proprietary graphical programming software. This software facilitates the configuration of these controllers with a user-friendly, customizable interface and features a wide array of built-in basic and advanced programming blocs such as proportional–integral–derivative loops, time delay, schedules, real-time clock, optimum start, stage sequencing, logical gates, mathematical and comparator functions, psychometric functions, etc.

All of the sensors in the ERS Test System are systematically calibrated on a routine basis, either in the field or by the manufacturers. Recalibration schedules are rigorous but flexible to address project needs.

Test design and implementation

Test design

A total of four PI control loops were examined using the two proposed CQFs during the field test at ERS. These PI control loops included VAV airflow control, zone cooling control, zone heating control, and VAV discharge air temperature control. These control loops were tested in East B, South B, West B, and Interior B VAV controllers, respectively.

Table 5 lists the test design matrix. Essentially, two cycles of tests were conducted, with each cycle containing six test days. Each cycle included three normal operation tests days (two dual-max heating test days and one single-max heating test day). During normal operation tests, the VAV controllers run automatically to meet room heating or cooling demand without interruptions. AHU supply air temperature and room temperature set-points remained the same. The lighting, occupancy, and computer systems operated on a fixed schedule (on from 8 a.m. to 12 p.m. and from 1 p.m. to 5 p.m.; off the rest of the day) to mimic a typical office occupancy schedule. No additional internal load was applied during these normal testing days.

Table 5 Test design parameters.a

Test room	East B	South B	West B	Interior B	AHU
Control loop under test	Airflow percentage	Room cooling	Room heating	VAV discharge air temperature
Test settings	Heating set-point, °F (°C)	Cooling set-point, °F (°C)	Extra internal load	Heating set-point, °F (°C)	Cooling set-point, °F (°C)	Extra internal load	Heating set-point, °F (°C)	Cooling set-point, °F (°C)	Extra internal load	Heating set-point, °F (°C)	Cooling set-point, °F (°C)	Extra internal load	Supply air temperature
Day 1 normal operation	12:00 a.m.	70 (21.1)	74 (23.3)	BB heat two stages	70 (21.1)	74 (23.3)	BB heat two stages	70 (21.1)	74 (23.3)	None	70 (21.1)	74 (23.3)	None	55 (12.8)
3:00 a.m.
6:00 a.m.
9:00 a.m.
12:00 p.m.
3:00 p.m.
6:00 p.m.
9:00 p.m.
Day 2 room temperature set-point variation	12:00 a.m.	75 (23.9)	79 (26.1)	BB heat two stages	75 (23.9)	79 (26.1)	BB heat two stages	65 (18.3)	69 (20.6)	None	65 (18.3)	69 (20.6)	None	55 (12.8)
3:00 a.m.	65 (18.3)	69 (20.6)	65 (18.3)	69 (20.6)	66 (18.9)	70 (21.1)	66 (18.9)	70 (21.1)
6:00 a.m.	75 (23.9)	79 (26.1)	75 (23.9)	79 (26.1)	68 (20)	72 (22.2)	68 (20)	72 (22.2)
9:00 a.m.	68 (20)	72 (22.2)	68 (20)	72 (22.2)	73 (22.8)	77 (25)	73 (22.8)	77 (25)
12:00 p.m.	66 (18.9)	70 (21.1)	66 (18.9)	70 (21.1)	65 (18.3)	69 (20.6)	65 (18.3)	69 (20.6)
3:00 p.m.	71 (21.7)	75 (23.9)	71 (21.7)	75 (23.9)	75 (23.9)	79 (26.1)	75 (23.9)	79 (26.1)
6:00 p.m.	66 (18.9)	70 (21.1)	66 (18.9)	70 (21.1)	65 (18.3)	69 (20.6)	65 (18.3)	69 (20.6)
9:00 p.m.	65 (18.3)	69 (20.6)	65 (18.3)	69 (20.6)	72 (22.2)	76 (24.4)	72 (22.2)	76 (24.4)
Day 3 internal load variation	12:00 a.m.	70 (21.1)	74 (23.3)	BB two stages	70 (21.1)	74 (23.3)	BB two stages	70 (21.1)	74 (23.3)	FCU 20%	70 (21.1)	74 (23.3)	FCU 20%	55 (12.8)
3:00 a.m.	None	BB one stage	FCU 40%	FCU 40%
6:00 a.m.	FCU 80%	None	None	None
9:00 a.m.	BB one stage	BB one stage	FCU 80%	FCU 80%
12:00 p.m.	FCU 40%	None	None	None
3:00 p.m.	BB two stages	BB two stages	FCU 60%	FCU 60%
6:00 p.m.	FCU 100%	None	FCU 100%	FCU 100%
9:00 p.m.	BB one stage	BB one stage	None	None
Day 4 AHU supply air temperature variation	12:00 a.m.	70 (21.1)	74 (23.3)	None	70 (21.1)	74 (23.3)	BB heat two stages	70 (21.1)	74 (23.3)	None	70 (21.1)	74 (23.3)	None	50 (10)
3:00 a.m.	55 (12.8)
6:00 a.m.	60 (15.6)
9:00 a.m.	65 (18.3)
12:00 p.m.	55 (12.8)
3:00 p.m.	50 (10)
6:00 p.m.	60 (15.6)
9:00 p.m.	65 (18.3)
Day 5 normal operation	12:00 a.m.	70 (21.1)	74 (23.3)	BB heat two stages	70 (21.1)	74 (23.3)	BB heat two stages	70 (21.1)	74 (23.3)	None	70 (21.1)	74 (23.3)	None	55 (12.8)
3:00 a.m.
6:00 a.m.
9:00 a.m.
12:00 p.m.
3:00 p.m.
6:00 p.m.
9:00 p.m.
Day 6 normal operation single-max heating for East B and West B	12:00 a.m.	70 (21.1)	74 (23.3)	None	70 (21.1)	74 (23.3)	BB heat two stages	70 (21.1)	74 (23.3)	None	70 (21.1)	74 (23.3)	None	55 (12.8)
3:00 a.m.
6:00 a.m.
9:00 a.m.
12:00 p.m.
3:00 p.m.
6:00 p.m.
9:00 p.m.

1 Note: aBy default, dual-max heating logic applies to all test rooms and cases. Single-max heating only applies to day 6 for East B and West B. For all cases, the lighting, occupancy, and computer followed a fixed schedule: on from 8 a.m. to 12 p.m. and from 1 p.m. to 5 p.m.; off the rest of the day. For all cases, the AHU supply air pressure was fixed at 1.4 in.wc (348.376 Pa). BB: Baseboard; FCU: fan coil unit cooling valve open position.

For each test cycle, three test days with different load disturbances were also conducted. The disturbances included room temperature set-point changes, extra internal loads to test rooms, and AHU supply air temperature variations. Every disturbance was changed every 3 h to ensure that the system recovered to stable control and the test duration covered the CQF moving window, which is 80 min. The room air temperature set-point step changes were 1 °F (0.56 °C), 2 °F (1.12 °C), 5 °F (2.8 °C), 7 °F (3.92 °C), 10 °F (5.6 °C). The AHU supply air temperature set-point step change was 5 °F (2.8 °C). Extra internal heat gains changes were realized through the electric baseboard heat and the fan coil unit cooling.

For all cases, CQF1 (referred as CQF_Harris) and CQF2 (referred as CQF_EWMA) were run at the same time for a given PI loop on each controller.

DDC programming

Before conducting the experiments, the two CQF logics were programmed using the DDC control language according to the MATLAB codes that were used to develop and offline-test the CQFs. The reversal behaviors are incorporated in the programming. The control loop would be a failed loop if reversal behaviors were detected regardless of how well the set-point was maintained. The DDC codes were then integrated into the existing VAV control programs. These new DDC programs were then downloaded to each of the local VAV controllers in four B test rooms. All of the DDC programming was done at the local controller level, and no resources from the network controllers or the Web supervisor were used. It is worth mentioning that the CQF-Harris implementation is complicated. To implement just one proportional–integral–derivative control loop's CQF-Harris, the final DDC program consumes 80% of code space usage and 93% of RAM usage on each local controller. Before implementing CQF-Harris, 51% of code space usage and 62% of RAM usage were used by existing VAV control programs, which include advanced sequences of operation such as dual-max heating and minimum ventilation calculate based on a 5-min moving window. The implementation of CQF-EWMA was straightforward and easy. The following items were the major issues experienced during the programming and debugging of the CQF-Harris index:

• The CQF-Harris index calculation requires a moving window that contains 20 samples of historical data. The sampling frequency was 4 min, so the moving window contained the readings of controlled variables and set-points for the past 80 min. The moving window needed to continuously rolled in real time. The programming tool for the DDC system at ERS, unlike some other DDC systems, does not have a way to write directly and then read historical data to/from a data file on the controller. Thus, a complicated algorithm was developed to hold each reading for 80 min and arrange 20 samples in sequence for Harris index calculation. Specifically, two logical modules, "Equal" and "Switch," were grouped together to hold one sample temporarily and 20 of these groups were built and connected to achieve 20-sample moving window. In addition, a "Timer" group was programmed to count the time elapsed and reset every 4 min, keeping the moving window rolling. All of these modules consume resources of the DDC system. Similar algorithms may be needed for DDC control systems that do not have the capability to directly read/write historical data from the DDC controllers.
• In each moving window, for-loop calculations were involved for the autoregressive moving average model. Specifically, for each step of the for loop, three of 20 samples in the moving window and the fixed initial values of intermediate variables were involved to finish one round of calculation. The results of this round were the initial values of the intermediate variables for the next round, which involved the next three samples of the moving window. This calculation kept rolling until it reached the last sample of the 20 samples. Then the program reset the initial values of the intermediate variables and the for loop started over from the first three samples in the moving window. In addition, each step of the for loop included several sets of matrix calculation (1 × 3, 3 × 1, or 3 × 3) and many DDC modules were used to achieve that. All logic described above requires resources in the control system.
• The CQF-Harris calculation could result in large intermediate variables (such as 1050) when the actual values of the control variables are used as the inputs to the CQF-Harris logic (e.g., using a 74 °F [23.3 °C] cooling set-point and a 73 °F [22.8 °C] temperature reading as the inputs). Such a large number could not be handled by the building controller processor. At a certain point during the test, the intermediate numbers became too large and the program started to show the infinity sign (∞) instead of actual numbers and CQF-Harris results showed "null" values. To solve this problem, the initial inputs to CQF-Harris were subtracted by a given constant number. For example, 68 °F (20 °C) was subtracted for the cooling control loop test to make intermediate variables manageable by the controller processor.

CQF evaluations

Room cooling air temperature control

The room cooling control loop was assessed in the South B room. The cooling set-point was fixed at 74 °F (23.3 °C) for most test days except for day 2, in which the cooling set-point varied between 69 °F (20.6 °C) and 79 °F (26.1 °C). In all cases, the heating set-point was always 4 °F (2.2 °C) lower than the cooling set-point, which left a 4 °F (2.2 °C) deadband between the cooling and heating modes.

Figures 15–18 show the test results at South B for day 2 (room temperature variation) and day 6 (normal operation). Figure 15 shows the temperature outputs (blue dashed line) with varying set-points (red line), in addition to the online assessment using the CQF-Harris (black line). Figure 16 shows the temperature outputs (blue dash line) with varying set-points (red line) and the online evaluation by the CQF-EWMA (black line). Figure 17 shows the comparisons between the online assessment (blue dash line) and the offline assessment by scripts (red line) for the CQF-Harris with the assessment scales. Figure 18 shows the comparison between online assessment (blue dash line) and offline assessment by scripts (red line) for the CQF-EWMA with the assessment scales.

PHOTO (COLOR): Fig. 15 CQF inputs and CQF-Harris (room cooling South B).

PHOTO (COLOR): Fig. 16 CQF inputs and CQF-EWMA (room cooling South B).

PHOTO (COLOR): Fig. 17 CQF-Harris comparison (room cooling South B).

PHOTO (COLOR): Fig. 18 CQF-EWMA comparison (room cooling South B).

In Figures 15–18, three interesting assessment behaviors in three periods are identified. Between timesteps t = 0 and t = 350, the set-point is dynamically reset on purpose. It is shown that both the CQF-Harris and the CQF-EWMA also vary dynamically. Between timesteps t = 350 and t = 600, the set-point is kept as a constant. Both the CQF-Harris and the CQF-EWMA have assessments in the excellent (indicated by "A") scale. For the timestep after t = 600, the set-point is not well maintained. It can be seen the CQF-Harris increases from an excellent scale to a failed scale. The CQF-EWMA increases from an excellent scale to a fair scale.

In addition, the assessments between the real-time online and the MATLAB offline implementations are matched well with negligible differences (Figures 17 and 18). This demonstrates that the proposed assessment algorithms can be applied for an online assessment.

During the assessment, the CQF-Harris shows high values of 0.8–1, thus indicating a "failed" control when there is more than a 4 °F (2.2 °C) difference between the cooling set-point and the actual temperature reading. The CQF-EWMA shows high values of 1, thus also indicating "failed" control for the same data points. CQF-Harris normally stays at a high value longer, dropping toward 0 about an hour after the temperature becomes stable and closed control occurs around the set-point. This is because CQF-Harris requires an 80-min moving window to conduct the assessment. In contrast, CQF-EWMA reaches 1 every time the set-point changes but stays at 1 for only one or two sampling frequencies (4–8 min). It starts to drop to 0.5–0.7, indicating "fair" control, even when the difference between the set-point and the temperature reading remains the same. The CQF-EWMA drops to 0 right away when the temperature nears the set-point.

Interestingly, this control loop does not have the reversal behaviors detected because the small fluctuations did not reach the threshold of the reversal assessment.

Room heating air temperature control

The room heating control loop was tested in the West B room. The heating set-point was fixed at 70 °F (21.1 °C) for most test days except day 2, in which the set-point varied between 65 °F (18.3 °C) and 72 °F (22.2 °C). For all cases, the heating set-point was always 4 °F (2.2 °C) lower than the cooling set-point, which left a 4 °F (2.2 °C) deadband between the cooling and heating modes.

Figures 19–22 show the test results in West B for the same cases with cooling control as in South B. Figure 19 shows the CQF-Harris online assessment (black line) for temperature outputs (blue dash line) with varying set-points (red line). Figure 20 shows similar plots but with the online evaluation with the CQF-EWMA (black line). Figure 21 shows comparisons between the online assessment (blue dash line) and the offline assessment by scripts (red line) for the CQF-Harris scales. Figure 22 shows similar comparisons but with assessments from the CQF-EWMA scales.

PHOTO (COLOR): Fig. 19 CQF inputs and CQF-Harris (room heating West B).

PHOTO (COLOR): Fig. 20 CQF inputs and CQF-EWMA (room heating West B).

PHOTO (COLOR): Fig. 21 CQF-Harris comparison (room heating West B).

PHOTO (COLOR): Fig. 22 CQF-EWMA comparison (room heating West B).

In Figures 19–22, two interesting assessment behaviors with two periods are demonstrated. Between timesteps t = 0 and t = 320, the set-point is dynamically reset on purpose. It is shown that both the CQF-Harris and CQF-EWMA also vary dynamically. The CQF-Harris shows that the control loops are bad to failed when there is a set-point change. Between timesteps t = 320 and t = 700, the set-point is kept as a constant. Both the CQF-Harris and the CQF-EWMA have assessments in the excellent (indicated by "A") scale. Furthermore, there is a small period (t = 580–600) when the control loop cannot maintain the set-point. It can be seen the CQF-Harris is degrading from an excellent scale to a fair scale. The CQF-EWMA is varied from an excellent scale to a good scale. Both CQFs are sensitive to the control output changes.

Figures 21 and 22 show that the online assessment and the MATLAB offline implementations are matched. In addition, no reversal behaviors were detected from this room temperature control loop in the heating mode.

Discussion and conclusions

This article presented assessment results from offline model-based control loops, offline real control loops, and online real control loops using the CQFs proposed by ASHRAE RP-1587. It is concluded that both the CQF-Harris and the CQF-EWMA can consistently assess the control loop performance with their own assessment scales. The overall performance from the two CQFs is consistent with slight differences. This is due to the different underlying calculation methods for CQF-Harris and CQF-EWMA. The assessment results using these two indices are aligned with each other for the majority of the test cases.

Through the field testing of the proposed CQFs, the comparisons of the proposed CQFs are summarized:

• The strength of CQF-Harris is that the unmeasured disturbances of the control loop are considered and no plant tests are needed. The shortcoming of CQF-Harris is that it is relatively complicated in terms of computational resources required from the controllers. To correctly program this index on DDC controllers may be challenging and will require careful debugging.
• The strength of CQF-EWMA is that it is simple and takes up less memory. Based on the field test at the Iowa Energy Center using BACnet-compatible controllers with 1 MB flash memory and 2 MB storage memory, the CQF-Harris demands a higher memory for the regression (i.e., autoregressive moving average fitting) implementation, whereas the CQF-EWMA takes up much less memory and DDC programming is easier.

To properly use the proposed CQFs in real buildings, we propose the following recommendations:

• We recommend the adoption of the CQF-Harris index for DDC controllers with a higher CPU power and larger memory; for example, higher than 2 MB. The CQF-EWMA index can be implemented in most modern DDC controllers.
• The purpose of the proposed CQFs is assessing a normal loop operation after recovering from a set-point change and/or a disturbance (e.g., load change). To assess any loop after a set-point change, it this dynamic period should be filtered out, especially for the average weighted CQFs over a longer period. A dynamic response analysis is outside the scope of the proposed HVAC control loop performance assessment in this article.
• The proposed CQFs can only be used to assess the control loop performance when the controlled device is not saturated/maxed out. To further understand whether the poor CQF index is caused by the local controller itself or the controlled device is saturated/maxed out, further investigations are necessary.

Nomenclature

•

• error ratio for CQF-EWMA

•

• CQF-EWMA index

•

• CQF-Harris index

•

• scale factor for CQF-EWMA

•

• reversal index

•

• EWMA value of control loop outputs

•

• square of minimum variance

•

• variance of the unmeasured disturbance (white noise)

•

• variance of control outputs

1 Ran Liu, Member ASHRAE, is a Technical Director. Yanfei Li, Associate Member ASHRAE, is a System Engineer. Zheng O'Neill, Member ASHRAE, is an Associate Professor. Xiaohui Zhou, Member ASHRAE, is a Senior Researcher. 2 Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/uhvc. Afram, A., and F. Janabi-Sharifi. 2014. Theory and applications of HVAC control systems—A review of model predictive control (MPC). Building and Environment 72 : 343 – 55. Dalamagkidis, K., and D. Kolokotsa. 2008. Reinforcement Learning for Building Environmental Control. Vienna, Austria: InTech. 3 Dassault Systèmes. 2015. Dymola. https://www.3ds.com/products-services/catia/products/dymola/. 4 Dechert, S. 2015. Better buildings saves over $1 billion in energy. https://cleantechnica.com/2015/06/23/better-buildings-saves-1-billion-energy-opens-online-solution-center/. 5 EERE. 2011. Buildings Energy Data Book. Washington, DC: Office of Energy Efficiency and Renewable Energy. http://buildingsdatabook.eere.energy.gov/. 6 Fernandez, N., M.R. Brambley, S. Katipamula, H. Cho, J.K. Goddard, and L.H. Dinh. 2010. Self-Correcting HVAC Controls Project Final Report. Richland, WA : Pacific Northwest National Lab. 7 Fernandez, N., S. Katipamula, W. Wang, Y. Huang, and G. Liu. 2012. Energy savings modeling of standard commercial building re-tuning measures: Large office buildings. Washington, DC: US Department of Energy. 8 Fernandez, N., S. Katipamula, W. Wang, Y. Huang, and G. Liu. 2015. Energy savings modelling of re-tuning energy conservation measures in large office buildings. 8 (6): 391 – 407. 9 Fernandez, N., S. Katipamula, W. Wang, Y. Xie, M. Zhao, and C.D. Corbin. 2017. Impacts of Commercial Building Controls on Energy Savings and Peak Load Reduction. Richland, WA : Pacific Northwest National Lab. Fritzson, P. 2010. Principles of Object-Oriented Modeling and Simulation with Modelica 2.1. 1st ed. Hoboken, NJ : John Wiley & Sons. Hägglund, T. 1995. A control-loop performance monitor. Control Engineering Practice 3 (11): 1543 – 51. Hägglund, T. 1999. Automatic detection of sluggish control loops. Control Engineering Practice 7 (12): 1505 – 11. Hong, T., S.C. Taylor-Lange, S. D'Oca, D. Yan, and S.P. Corgnati. 2016. Advances in research and applications of energy-related occupant behavior in buildings. Energy and Buildings 116 : 694 – 702. Horch, A. 1999. A simple method for detection of stiction in control valves. Control Engineering Practice 7 (10): 1221 – 31. Horch, A., and F. Heiber. 2004. On evaluating control performance on large data sets. IFAC Proceedings Volumes 37(9):535–40. Kiriu, R., and S.T. Taylor. 2017. Automatic fault detection and diagnostics and hierarchical fault suppression in ASHRAE RP-1455. ASHRAE Transactions 123(2):101–10. Lau, J., X. Lin, and G. Yuill. 2014. CO2-based demand controlled ventilation for multiple zone HVAC systems. ASHRAE Research Project RP-1547, Final Report. Atlanta, GA: ASHRAE. Li, X., and J. Wen. 2014. Review of building energy modeling for control and operation. Renewable and Sustainable Energy Reviews 37 (Suppl. C): 517 – 37. Li, Y., Z. D. O'Neill, and X. Zhou. 2019. Development of control quality factor for HVAC control loop performance assessment I—Methodology (ASHRAE RP-1587). Science and Technology for the Built Environment. https://www.tandfonline.com/doi/abs/10.1080/23744731.2018.1556055. Miao, T., and D.E. Seborg. 1999. Automatic detection of excessively oscillatory feedback control loops. Proceedings of the 1999 IEEE International Conference on Control Applications (Cat. No. 99CH36328), Vol. 1, IEEE, Kohala Coast, HI, pp. 359–364. O'Neill, Z., and B. Eisenhower. 2013. Leveraging the analysis of parametric uncertainty for building energy model calibration. Building simulation: An International Journal, Vol. 6, pp. 365–377, Springer. O'Neill, Z., Y. Li, and K. Williams. 2017. HVAC control loop performance assessment: A critical review (1587-RP). Science and Technology for the Built Environment 23 (4): 619 – 36. O'Neill, Z., Y. Li, K. Williams, R. Liu, and X. Zhou. 2016. ASHRAE RP-1587: Control loop performance assessment. ASHRAE Research Project Final Report. Atlanta, GA; ASHRAE. O'Neill, Z., Y. Li, X. Zhou, S. Taylor, and H. Cheng. 2017. RP-1747—Implementation of RP-1547 CO 2 -Based Demand Controlled Ventilation for Multiple Zone HVAC Systems in Direct Digital Control Systems (ASHRAE Technical Report). Atlanta, GA: ASHRAE. O'Neill, Z., and F. Niu. 2017. Uncertainty and sensitivity analysis of spatio-temporal occupant behaviors on residential building energy usage utilizing Karhunen-Loève expansion. Building and Environment 115 : 157 – 72. Qian, D., Y. Li, F. Niu, and Z. O'Neill. 2018. Nationwide savings analysis of a variety of energy conservation measures. ASHRAE Winter Conference, 1/20–1/24, 2018, Chicago, IL. Salsbury, T.I. 1999. A practical algorithm for diagnosing control loop problems. Energy and Buildings 29 (3): 217 – 27. Salsbury, T.I. 2006. Control performance assessment for building automation systems. IFAC Proceedings Volumes 39(19):7–18. Salsbury, T.I. 2007. Continuous-time model identification for closed loop control performance assessment. Control Engineering Practice 15 (1): 109 – 21. U.S. Energy Information Administration. 2016. Annual energy outlook 2016. https://www.eia.gov/forecasts/aeo/tables%5fref.cfm. Wang, S., and Y. Chen. 2002. Fault-tolerant control for outdoor ventilation air flow rate in buildings based on neural network. Building and Environment 37 (7): 691 – 704. Wang, S., and Z. Ma. 2008. Supervisory and optimal control of building HVAC systems: A review. HVAC&R Research 14 (1): 3 – 32. Wetter, M., W. Zuo, T.S. Nouidui, and X. Pang. 2014. Modelica buildings library. Journal of Building Performance Simulation 7 (4): 253 – 70.

By Ran Liu; Yanfei Li; Zheng D. O'Neill and Xiaohui Zhou

Reported by Author; Author; Author; Author