KPI Example¶
In this example we look at the options for KPI tracking and their evaluation.
To illustrate the default settings, we first create a simulation environment, using the Offices model.
[1]:
import energym
env = energym.make("OfficesThermostat-v0", simulation_days=300)
[OK] fmi2Instantiate: The Resource location of FMU with instance name %s is %s.
[WARNING] fmi2Instantiate: Argument loggingOn is set to %d
. This is not supported. loggingOn will default to '0'.
[OK] The current working directory is %s
[OK] fmi2Instantiate: Path to fmuUnzipLocation is not %s.
[OK] fmi2Instantiate: Path to fmuUnzipLocation is not %s.
[OK] fmi2Instantiate: Path to fmuUnzipLocation is not %s.
[OK] fmi2Instantiate: Path to fmuUnzipLocation %s
[OK] fmi2Instantiate: Path to fmuResourceLocation %s
[OK] Command executes to copy content of resources folder: %s
[OK] fmi2Instantiate: Path to model description file is %s.
[OK] fmi2Instantiate: The FMU modelIdentifier is %s.
[OK] fmi2Instantiate: The FMU modelGUID is %s.
[OK] fmi2Instantiate: Slave %s is instantiated.
[OK] fmi2Instantiate: Instantiation of %s succeded.
[OK] fmi2EnterInitializationMode: The sockfd is %d.
[OK] fmi2EnterInitializationMode: The port number is %d.
[OK] fmi2EnterInitializationMode: This hostname is %s.
[OK] fmi2EnterInitializationMode: TCPServer Server waiting for clients on port: %d.
[OK] fmi2EnterInitializationMode: The number of input variables is %d.
[OK] fmi2EnterInitializationMode: The number of output variables is %d.
[OK] Get input file from resource folder %s.
[OK] Searching for following pattern %s
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Found matching file %s.
[OK] done searching pattern %s
[OK] Get input file from resource folder %s.
[OK] Searching for following pattern %s
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Found matching file %s.
[OK] done searching pattern %s
[OK] Get input file from resource folder %s.
[OK] Searching for following pattern %s
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Found matching file %s.
[OK] done searching pattern %s
[OK] This version uses the **energyplus** command line interface to call the EnergyPlus executable. **RunEPlus.bat** and **runenergyplus** , which were used in earlier versions, were deprecated as of August 2015.
[OK] fmi2EnterInitializationMode: The connection has been accepted.
[OK] fmi2EnterInitializationMode: Slave %s is initialized.
During the simulation, the important values for the KPIs are stored in a KPI object and can be retrieved with the get_kpi() method.
[2]:
for _ in range(500):
action = env.sample_random_action()
output = env.step(action)
print(env.get_kpi())
{'kpi1': {'name': 'Fa_Pw_All', 'type': 'avg', 'kpi': 1933.097892619751}, 'kpi2': {'name': 'Z01_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.1818520220347843}, 'kpi3': {'name': 'Z02_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.3843820872934631}, 'kpi4': {'name': 'Z03_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.4492356886047587}, 'kpi5': {'name': 'Z04_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.4769999114379626}, 'kpi6': {'name': 'Z05_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.28759098919405063}, 'kpi7': {'name': 'Z06_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.7346050381770025}, 'kpi8': {'name': 'Z07_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.7412273635737812}, 'kpi16': {'name': 'Z15_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 2.2118444553631686}, 'kpi17': {'name': 'Z16_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 1.208163739536243}, 'kpi18': {'name': 'Z17_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.03245551101207099}, 'kpi19': {'name': 'Z18_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.3290193419351164}, 'kpi20': {'name': 'Z19_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.046590732155760685}, 'kpi21': {'name': 'Z20_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.7139002375762351}, 'kpi26': {'name': 'Z25_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.022292636987625934}, 'kpi27': {'name': 'Z01_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 351}, 'kpi28': {'name': 'Z02_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 225}, 'kpi29': {'name': 'Z03_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 222}, 'kpi30': {'name': 'Z04_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 251}, 'kpi31': {'name': 'Z05_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 327}, 'kpi32': {'name': 'Z06_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 307}, 'kpi33': {'name': 'Z07_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 307}, 'kpi41': {'name': 'Z15_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 501}, 'kpi42': {'name': 'Z16_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 475}, 'kpi43': {'name': 'Z17_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 42}, 'kpi44': {'name': 'Z18_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 432}, 'kpi45': {'name': 'Z19_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 62}, 'kpi46': {'name': 'Z20_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 463}, 'kpi51': {'name': 'Z25_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 40}}
Through this, we can directly see the default KPIs that are tracked for the Offices model. Those are * The average power demand of the whole facility * The average temperature deviation from a fixed interval for all zones * The total temperature interval violations for all zones
In the example above, we did not specify any arguments for get_kpi(), so the KPIs were computed over all performed simulation steps. To get the KPIs from a specific time interval, we can specify the simulation steps in the method call:
[3]:
print(env.get_kpi(start_ind=100, end_ind=300))
{'kpi1': {'name': 'Fa_Pw_All', 'type': 'avg', 'kpi': 1921.4823123126669}, 'kpi2': {'name': 'Z01_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.1573853083208828}, 'kpi3': {'name': 'Z02_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.37227112901825216}, 'kpi4': {'name': 'Z03_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.4261302830511886}, 'kpi5': {'name': 'Z04_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.4957972201715515}, 'kpi6': {'name': 'Z05_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.26159124363016045}, 'kpi7': {'name': 'Z06_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.660215604516257}, 'kpi8': {'name': 'Z07_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.7200224910693038}, 'kpi16': {'name': 'Z15_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 2.297474883555688}, 'kpi17': {'name': 'Z16_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 1.40532108250742}, 'kpi18': {'name': 'Z17_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.0675676410001173}, 'kpi19': {'name': 'Z18_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.4480624623851421}, 'kpi20': {'name': 'Z19_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.030272357503866}, 'kpi21': {'name': 'Z20_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.6901404546664289}, 'kpi26': {'name': 'Z25_T', 'type': 'avg_dev', 'target': [19, 24], 'kpi': 0.006139037759710159}, 'kpi27': {'name': 'Z01_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 141}, 'kpi28': {'name': 'Z02_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 89}, 'kpi29': {'name': 'Z03_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 84}, 'kpi30': {'name': 'Z04_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 97}, 'kpi31': {'name': 'Z05_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 113}, 'kpi32': {'name': 'Z06_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 121}, 'kpi33': {'name': 'Z07_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 122}, 'kpi41': {'name': 'Z15_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 200}, 'kpi42': {'name': 'Z16_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 200}, 'kpi43': {'name': 'Z17_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 29}, 'kpi44': {'name': 'Z18_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 193}, 'kpi45': {'name': 'Z19_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 16}, 'kpi46': {'name': 'Z20_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 196}, 'kpi51': {'name': 'Z25_T', 'type': 'tot_viol', 'target': [19, 24], 'kpi': 8}}
The get_kpi() method comes in handy if we want to use the KPI values somewhere in the code. To simply print the KPIs, we can directly use the print_kpis() method:
[4]:
env.print_kpis()
####################################################################
Variable name: Fa_Pw_All, kpi type: avg, kpi value: 1933.097892619751
####################################################################
Variable name: Z01_T, kpi type: avg_dev, kpi value: 0.1818520220347843
####################################################################
Variable name: Z02_T, kpi type: avg_dev, kpi value: 0.3843820872934631
####################################################################
Variable name: Z03_T, kpi type: avg_dev, kpi value: 0.4492356886047587
####################################################################
Variable name: Z04_T, kpi type: avg_dev, kpi value: 0.4769999114379626
####################################################################
Variable name: Z05_T, kpi type: avg_dev, kpi value: 0.28759098919405063
####################################################################
Variable name: Z06_T, kpi type: avg_dev, kpi value: 0.7346050381770025
####################################################################
Variable name: Z07_T, kpi type: avg_dev, kpi value: 0.7412273635737812
####################################################################
Variable name: Z15_T, kpi type: avg_dev, kpi value: 2.2118444553631686
####################################################################
Variable name: Z16_T, kpi type: avg_dev, kpi value: 1.208163739536243
####################################################################
Variable name: Z17_T, kpi type: avg_dev, kpi value: 0.03245551101207099
####################################################################
Variable name: Z18_T, kpi type: avg_dev, kpi value: 0.3290193419351164
####################################################################
Variable name: Z19_T, kpi type: avg_dev, kpi value: 0.046590732155760685
####################################################################
Variable name: Z20_T, kpi type: avg_dev, kpi value: 0.7139002375762351
####################################################################
Variable name: Z25_T, kpi type: avg_dev, kpi value: 0.022292636987625934
####################################################################
Variable name: Z01_T, kpi type: tot_viol, kpi value: 351
####################################################################
Variable name: Z02_T, kpi type: tot_viol, kpi value: 225
####################################################################
Variable name: Z03_T, kpi type: tot_viol, kpi value: 222
####################################################################
Variable name: Z04_T, kpi type: tot_viol, kpi value: 251
####################################################################
Variable name: Z05_T, kpi type: tot_viol, kpi value: 327
####################################################################
Variable name: Z06_T, kpi type: tot_viol, kpi value: 307
####################################################################
Variable name: Z07_T, kpi type: tot_viol, kpi value: 307
####################################################################
Variable name: Z15_T, kpi type: tot_viol, kpi value: 501
####################################################################
Variable name: Z16_T, kpi type: tot_viol, kpi value: 475
####################################################################
Variable name: Z17_T, kpi type: tot_viol, kpi value: 42
####################################################################
Variable name: Z18_T, kpi type: tot_viol, kpi value: 432
####################################################################
Variable name: Z19_T, kpi type: tot_viol, kpi value: 62
####################################################################
Variable name: Z20_T, kpi type: tot_viol, kpi value: 463
####################################################################
Variable name: Z25_T, kpi type: tot_viol, kpi value: 40
Sometimes we want to evaluate KPI for more than one zone, for example the average temperature deviation or the number of temperature violations over all zones. For this, the get_cumulative_kpi() method can be used. We need to specify a part of the name that all relevant KPIs have in common (here e.g. “_T“) the KPI type that should be aggregated (e.g. ”avg_dev“) and the type of aggregation (e.g. ”avg”). In code this looks as follows:
[5]:
print(env.get_cumulative_kpi(names="_T", kpi_type="avg_dev", out_type="avg"))
print(env.get_cumulative_kpi(names="_T", kpi_type="tot_viol", out_type="sum"))
0.5585828396344302
4005
Another way to aggregate KPIs over different zones is to pass a list of names that should be aggregated:
[6]:
print(env.get_cumulative_kpi(names=["Z01_T", "Z02_T", "Z03_T"], kpi_type="avg_dev", out_type="avg"))
env.close()
0.3384899326443354
[OK] fmi2Terminate: fmiFreeInstanceSlave must be called to free the FMU instance.
[OK] fmi2FreeInstance: The function fmi2FreeInstance of instance %s is executed.
[OK] freeInstanceResources: %s will be freed.
Specifying custom KPIs¶
To track custom KPIs instead of default ones, it is possible to specify the kpi_options parameter in the construction of the simulation environment. We will illustrate this through tracking the average power demand at the HVAC, building and facility level, and the average deviation of two zone temperatures from a fixed target:
[7]:
kpi_options = {"kpi1": {"name":"Fa_Pw_All", "type":"avg"},
"kpi2": {"name":"Bd_Pw_All", "type":"avg"},
"kpi3": {"name":"Fa_Pw_HVAC", "type":"avg"},
"kpi5": {"name":"Z01_T", "type":"avg_dev", "target":23},
"kpi6": {"name":"Z02_T", "type":"avg_dev", "target":23}
}
env = energym.make("OfficesThermostat-v0", simulation_days=300, kpi_options=kpi_options)
[OK] fmi2Instantiate: The Resource location of FMU with instance name %s is %s.
[WARNING] fmi2Instantiate: Argument loggingOn is set to %d
. This is not supported. loggingOn will default to '0'.
[OK] The current working directory is %s
[OK] fmi2Instantiate: Path to fmuUnzipLocation is not %s.
[OK] fmi2Instantiate: Path to fmuUnzipLocation is not %s.
[OK] fmi2Instantiate: Path to fmuUnzipLocation is not %s.
[OK] fmi2Instantiate: Path to fmuUnzipLocation %s
[OK] fmi2Instantiate: Path to fmuResourceLocation %s
[OK] Command executes to copy content of resources folder: %s
[OK] fmi2Instantiate: Path to model description file is %s.
[OK] fmi2Instantiate: The FMU modelIdentifier is %s.
[OK] fmi2Instantiate: The FMU modelGUID is %s.
[OK] fmi2Instantiate: Slave %s is instantiated.
[OK] fmi2Instantiate: Instantiation of %s succeded.
[OK] fmi2EnterInitializationMode: The sockfd is %d.
[OK] fmi2EnterInitializationMode: The port number is %d.
[OK] fmi2EnterInitializationMode: This hostname is %s.
[OK] fmi2EnterInitializationMode: TCPServer Server waiting for clients on port: %d.
[OK] fmi2EnterInitializationMode: The number of input variables is %d.
[OK] fmi2EnterInitializationMode: The number of output variables is %d.
[OK] Get input file from resource folder %s.
[OK] Searching for following pattern %s
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Found matching file %s.
[OK] done searching pattern %s
[OK] Get input file from resource folder %s.
[OK] Searching for following pattern %s
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Found matching file %s.
[OK] done searching pattern %s
[OK] Get input file from resource folder %s.
[OK] Searching for following pattern %s
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Read directory and search for *.idf, *.epw, or *.idd file.
[OK] Found matching file %s.
[OK] done searching pattern %s
[OK] This version uses the **energyplus** command line interface to call the EnergyPlus executable. **RunEPlus.bat** and **runenergyplus** , which were used in earlier versions, were deprecated as of August 2015.
[OK] fmi2EnterInitializationMode: The connection has been accepted.
[OK] fmi2EnterInitializationMode: Slave %s is initialized.
We run this simulation again for a few timesteps with random actions and observe the KPIs:
[8]:
for _ in range(500):
action = env.sample_random_action()
output = env.step(action)
env.print_kpis()
env.close()
####################################################################
Variable name: Fa_Pw_All, kpi type: avg, kpi value: 2194.336931927295
####################################################################
Variable name: Bd_Pw_All, kpi type: avg, kpi value: 0.0
####################################################################
Variable name: Fa_Pw_HVAC, kpi type: avg, kpi value: 2194.336931927295
####################################################################
Variable name: Z01_T, kpi type: avg_dev, kpi value: 3.8400736461086904
####################################################################
Variable name: Z02_T, kpi type: avg_dev, kpi value: 3.640314683871775
[OK] fmi2Terminate: fmiFreeInstanceSlave must be called to free the FMU instance.
[OK] fmi2FreeInstance: The function fmi2FreeInstance of instance %s is executed.
[OK] freeInstanceResources: %s will be freed.
[ ]: