Vault

Description

The Kaizen Vault is where all of a Building’s Trend Logs, Objects, and Device Backups are found.

Trend Logs

Trend logs are time-value pairs recording the value (or status) of object properties within controllers. They are collected from the controllers for storage elsewhere to overcome the limited storage space of the controllers.

Trend Logs can be used as inputs in order to perform analysis in Kaizen. They can be viewed within the Vault feature. The Trend Log object is used to provide units, polling frequency, and polling type. The object is also part of the information used to drive Golden Standards.

The types of Trend Logs collected into Kaizen and displayed in the Vault are Trend Logs (TL) which show a single trend, Multi-Trends (MT) which show multiple trend logs, Calculated Trend Logs (CTL) which are created by Working Rules, and Imported Trend Logs (Imported TL) which are logs imported from CSV files. Weather Trend Logs (WT) use the location information of the building to collect weather information using the nearest weather stations.

Components

The Coppercube collects trend logs from controllers and then both store the data locally and send the data to Coppertree’s servers. Typically every trend log is visited every 4 hours, resulting in a steady trickle of data flowing to Coppertree. If the connection between the Coppercube and Coppertree is interrupted – the pending data is held until the link is restored. Several Coppercube capacities are available with the largest one able to collect 5000 (1 min Polled) trend logs for 5 years.

Every minute the Coppercube sends newly collected data to the Intake Server using a transmission queue in the way given below:

1. Data sits in the transmit queue until it is successfully received by the Coppertree servers.
2. Data that has been successfully sent then remains in the transmit queue (marked for deletion) for another 24 hours.

The Coppertree Intake Server receives the incoming trend log data and stores it in the Coppertree trend log database. Trend logs are stored by day according to their originating building, device, and TL#.

Trend logs are the raw data upon which most of Kaizen’s analysis is performed. Raw trend log data can be viewed via the Trend Log (TL) Viewer and the Multi-trend (MT) Viewer.

Types of Trend Logs

There are 5 kinds of Trend Logs used in Kaizen: BACnet Trend Logs (TL), Multi-Trends (MT), Calculated Trend Logs (CTL), Weather Trends (WT), and Imported Trend Logs (Imported TL).

Trend Logs (TL)

These are the most typical Trend Log type in Kaizen and are the Trend Logs that the CopperCube has collected from your BACnet network and sent to the Kaizen Vault.

These Trend Logs will have a Device Number associated with them that matches the BACnet Device Number that the Trend Log comes from. By default, they are updated every 4 hours when the CopperCube sends a new set of samples it has collected from the BACnet network.

Multi-Trend Objects (MT)

Multi-Trends, or MTs, are BACnet Objects that are used to reference other Trend Logs to display them in a single Trend Log view. They do not store any Trend Log samples in them and are just used to point to the different Trend Logs that are to be combined in a single graph.

Calculated Trend Log (CTL)

Calculated Trend Logs are generated by working rules, and are also assigned a Device ID of 0.

Imported Trend Logs (Imported TL)

Imported Trend Logs are imported CSV files that will be assigned a Device ID of 0. The CSV file must use the following format: YYYY-MM-DD hh:mm:ss, VALUE
This allows you to create your own Trend Log data (say from extensive baseline modeling in Excel) that can be used in the same way as Trend Logs from other sources such as the CopperCube or Calculated Trend Logs.

Other than this difference in how they originate, there is no distinction between Trend Logs and Imported Trend Logs (either in structure or how they may be used within Kaizen).

This feature can be used to create new Trend Log data in Kaizen. Additionally, existing Imported Trend Logs can be updated and modified at any time.

Note: Excel will change the CSV file’s date format to M/D/YYYY h:mm, VALUE and make it incompatible to be imported into Kaizen.

Weather Trend Logs (WT)

Assigning a Weather Location in a Kaizen Building’s Settings page allows Kaizen to find and pull data from the nearest Weather Station.

That data will then be accessible from the Building’s Vault as any other Trend Log, labeled as Weather Trends.

Other than this difference in how they originate, there is no distinction between Trend Logs and Weather Trend Logs (either in structure or how they may be used within Kaizen).

Trend Log Guidelines

The table below outlines the requirements for creating a Trend Log.

The chart below outlines the typical sizes of Trend Logs based on their sampling frequency.

Trend Log Navigation

The Kaizen Trend Log Search and Viewer are useful ways to see the Trend Logs (TLs) within your building and to see their value against the time stamps over a span of time.

The Trend Log Viewer is accessed by clicking  Vault and then Trend Logs.

Trend Log Search and Filter

The Trend Log Search page shows the TLs in your building and allows multiple ways to navigate them.

The table on the left side of the screen displays the Trend Log Name, Reference ID (in the BACnet format Device.Object+TL Instance), and the Last Sample Date. Each of the headings can be clicked to sort the Trend Logs accordingly.

Above the table is a search bar and above the search bar are filters, both to help find a specific Trend Log or a series of Logs.

Clicking on one of the Trend Logs will display it on the Right side of the screen.

The icon opens a wizard that guides you through the steps of importing Trend Logs via CSV. Please see TrendLogImport. for more information.

The Filters above the search bar are designed to help find specific Trend Logs, they are as follows:

• All: display all objects
• TL: displays only BACnet objects (i.e. no Kaizen-generated TLs)
• MT: short for Multitrend, displays only BACnet MT objects
• CTL: displays only calculated TLs that are generated from Working Rules
• WT: displays the weather information, including Temperature, Cloud Coverage, and Pressure using the building information
• Imported TL: displays only TLs generated via Kaizen’s import tool

The option to select between the “Active” and “Not Active” TLs is immediate to the Right of the Filters. Trend Logs that are “Not Active” usually mean that the object data is being collected by the CopperCube but the samples are not being archived.

The search bar can be used to search both the Trend Log Name and Reference. Both fields can be filtered by adding a space between terms. E.g. searching “SAT 104.” will narrow the table to TLs from device 104 which contains SAT in their name.

Trend Log Viewer

There is a Sliding View button near the top left of the screen, just to the Right of the Monitored Objects Information button. If you slide the viewer to the Right, you can access the trendlog and the viewer side-by-side.

If you slide the viewer to the Left, it collapses the trend log list and shows the graph view.

Pushing the Sliding View Button to the Left allows the Graph to be Full Screen. Hovering the mouse over the Trend Log name will show the monitored object of that Trend Log. To the right is a down arrow, which gives the user a number of options depending on the type of Trend Log:

• Hide others – hides all other TLs.
• Move to other axis – Analog TLs only. Useful for improving scale granularity.
• Display as – Allows the display to change between Analog and Digital and a few other options. Default is Analog Polled.
• Open TL in a new tab – Will open the TL in a new tab.
• View TL Object – Will open the Object page for that TL

Hovering the mouse over the graph will show the sample’s timestamp and value on the top-right of the graph. Clicking and dragging will zoom in on the selection and double-clicking will zoom out to display the Trend Log’s entire range of data. While holding shift, clicking and dragging will pan the graph left and right accordingly.

Below the graph is an overview of the Trend Log throughout its entire time range. The date range shown in the graph can be adjusted by moving the sliders at the bottom.

In addition to the sliders at the bottom, the date range can be adjusted by selecting the drop-down or calendar icon.

Above the Graph, on the Right side, there are double triangles for “Fast Forward” and “Rewind”. The buttons will cause the date to jump forward or back by the date range. E.g. If the TL Viewer is looking at 30 days, the “Rewind” button will navigate to the previous 30 days.

Important: If the date range is quite large, the data will be aggregated on either 4-hour intervals or daily, to improve visibility. Shaded areas indicate the range of values that exist in the date range, with the solid line representing the average value.

The “CSV” button exports the TL(s) into a CSV file.

Standard Trend Log vs Binary Trend Log

There are two general types of Trend Log – Standard and Binary. Standard Trend Logs tend to monitor analog values (typically corresponding to an AI, AO or AV BACnet objects), while Binary Trend Logs monitor objects with two states (typically BI, BO or BV BACnet objects).

Binary Trend Logs are displayed on the very bottom of the graph, underneath the Standard Trend Logs. The high values correspond to 1s (generally “On” status) and the low values correspond to 0s (generally “Off” status).

Trend Log API

The Application Programming Interface (API) is used to get CopperTree data from the cloud, for use outside of Kaizen. This API can be used to power dashboards, provide data to program BACnet controls, and run third-party reports.

Every Trend Log that you archive into Vault and use in Kaizen is available for other uses. In order to obtain this data, you use the Kaizen API. The API can return data in JSON format. The data is drawn directly from Trend Logs over a date range. There is also another API to directly access the Insight data The TL API returns the time/value data pairs from the specified trend logs for the requested time period.

API Parameters

The API consists of a request URL and a list of parameters. In order to obtain the desired data, you need to provide the appropriate parameters. The parameters are of the form “&parameter=field”, where the “parameter” is what the URL is referencing (&Start means start time, for example), and the “field” is the entered value (such as ‘2014-03-24T16:43:57’ for the start time).

API Parameters Presentation

The parameters are explained below:

Parameter	Description	Example
api_key	This is the “Coppertree Cloud Key” key found within a user’s profile in Kaizen. It is security to protect you data.	1234567890abcde
tl	This specifies the building id, device id, and Trend Log id number.	1001.202.TL8
start	the start date of your time range using this format “YYYY-MM-DDTHH:MM:SS”	2014-03-14T16:43:57
end	the end date of your time range using this format “YYYY-MM-DDTHH:MM:SS”	2014-03-24T16:43:57
format	the API will return value with a specific format (json).	json
data	the API can create by default synthetic data at the beginning and the end of the range for display purposes. You can remove the synthetic data by adding this parameter.	raw

API Sample

The API is always used as a 1-line URL : In this example – use the values provided as examples above

https://kaizen.coppertreeanalytics.com/public_api/api/get_tl_data_start_end?
&api_key=1234567890abcde
&tl=2131.103.TL8
&start=2014-03-14T16:43:57&end=2014-03-24T16:43:57
&format=json&data=raw
 

API Parameters Definition

api_key

This is the Coppertree Cloud Key found within a user’s profile in Kaizen. To navigate to the user’s profile:

• from Kaizen home page
• Click on your name (located n the top right-hand side of the page)
• Then click Profile

Cloud key for each user

tl

This is the Trend Log ID which could be found on the Vault page. To find the Trend Log’s ID:

• From your Building Dashboard page
• Click on  and then on 
• Select your needed Trend Log
• Copy the Trend Log ID from the Trend Log viewer page

start/end

The start and end parameters define the time range the API is referring to. The API runs the instance over the time range specified here, where “start” is the beginning of the range, and “end” is the end of it. The date range is formatted as: year-month-day T hour : minute : second (spaces added only for clarity)

Example: 2014-04-01T00:00:00 The above example refers to April 1st, 2014 at midnight.

format

The Kaizen API provides one output format:

• json

example:

json
[ [{“ts”: “2014-10-01T00:00:00”, “v”: “702000.00”}, {“ts”: “2014-11-01T00:00:00”, “v”: “2106000.00”}, {“ts”: “2014-11-16T00:00:00”, “v”: “2106000.00”}]

data

The API by default return two synthetic data points, one at the beginning of the range, a second one at the end of the range. If you want to return the raw samples only, you can add this parameter equal:

• raw

Kaizen API – Intake Trend Data

There is a desire for 3rd-party devices (controllers) to send sensor data to Kaizen for use in analytics. Doing so is actually very straightforward. The key is for the 3rd-party device to mimic enough of a Coppercube’s behaviour so that Kaizen’s Intake system recognizes the device’s data as sensor data and can determine the building it belongs to so that it can be written into the Kaizen Trend database.

There are several items involved in getting this mimicry to succeed. The device needs:

• to connect to the Internet – to connect to Kaizen.
• an Ethernet MAC address – to identify itself to Kaizen.
• to speak AMQP (advanced message queuing protocol) – to send its data
• to format its data in JSON (as detailed below)
• to organize its data along with BACnet concepts and guidelines

Then Kaizen needs:

• to be informed of the device’s existence
• a client and building to receive the device’s data. This can be an existing building or a new one can be created.

Once Kaizen is configured to receive the data, and the device has data to send, then the device:

• may send data as often as needed. Data should be sent in batches. Typically these would be every 4 hours with a maximum of once per minute, and a minimum of once per week.
• data may be sent out of order. New data will overwrite previously sent data.
• data may be repeated.
• data may be sent for past dates.
• data describing the sensor data (aka meta-data) (see below) should be sent daily.
• [recommended] Status (Heartbeat) information about the device’s operating status should be sent every 15 minutes (via http)

Concepts

JSON – Kaizen uses JSON as its native data format. See: https://en.wikipedia.org/wiki/JSON

JSON (JavaScript Object Notation), is an open standard format that uses human-readable text to transmit data objects consisting of attribute-value pairs.

BACnet – Kaizen expects to receive BACnet Object data and BACnet Trend log data. See: (see https://en.wikipedia.org/wiki/BACnet).

BACnet is a standard that was developed from within the HVAC industry to be able to provide communication between devices. It views sensor data as a series of time-value pairs – called Trend Logs. It also views the meta-data that describes the trend logs, as objects (sets of key-value pairs).

BACnet defines many types of objects and many services for accessing those objects. For our purposes, only the Device (DEV) object, the Trend Log (TL) object, and the trend log contents are of interest. Objects are grouped (and identified by) the controller that contains them. An HVAC controller contains only ONE Device object. It may contain an unlimited number of trend logs, each one defined (and described by) a TL object. The trend logs contain a history of the sensor values; sampled at frequency and times of the controller’s choosing.

NOTE: there is no requirement that your device uses BACnet, only that the data sent to Kaizen looks like BACnet data.

AMQP – Kaizen expects to receive data via AMQPS (wrongly abbreviated as AMPQS). See: (https://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol).

AMQP is a protocol for sending messages between queues on different systems across the Internet. It views communications as sessions, occurring on channels, targeting specific queues. For us, your device is the sending client (producer), and Kaizen is the receiving server (consumer). Kaizen uses AMQP extensively. It exposes several public queues for the receipt of incoming data.

Trend Log Data

Kaizen expects to see Trend Log messages in the following format on its AMQP input queue (cta.trendlogs) at amqps://intake1.coppertreeanalytics.com:5671Note any number of samples, for any time period within the day, can be sent at any time, any number of times, and in any order; without restriction. Kaizen keeps merging the messages to build complete day_buckets. Typically we send 6 or more messages per TL per day. If you want to correct/replace data, just send another message containing the replacement data.

{
    "mac": "000bab39f41e",           # Ethernet MAC address of sending !CopperCube
    "d" : 100000,                    # BACnet device#  [1- 4194303;  0x1- 0x3fffff]
    "dt" : "2014-10-20",             # Building local date of the samples
    "i" : 2,                         # BACnet Trendlog Instance# [1- 4194303;  0x1- 0x3fffff]
    "l" : "building name",           # User assigned name
    "rev" : 4,                       # internal use. ignore
    "t" : "TL",                      # BACnet type mnemonic.  TL-trendlog
    "ts" : ISODate("2014-10-20T15:46:18.000-08:00"),   # local epoch timestamp when data sent
    "tt" : "Polled",                 # Type of trend.  Polled || COV (change-of-value)
    "uf" : 14400,                    # TL collection frequency (in secs).  i.e. TL is retrieved every 4 hrs.   NOT the TL Polling freq (i.e. 15 min.)
    "00:00:00" : 22.94293403625488,  # sample data.  This trend is - recording an Analog Input every 15 min.
    "00:15:00" : 22.92698097229004,  # Note: time of sample.  (Date is redundant & stored only once in 'dt' to save space).
    "00:30:00" : 22.90200424194336,
    "00:45:00" : 22.87795639038086,
    "01:00:00" : 22.86405563354492,
                …
    "23:00:00" : 22.47274589538574,  # Note: all times are originating controller's local time in 24 hour format.  UTC is not used. 
    "23:15:00" : 22.45730209350586,  #          Too many issues with timezones, time conversion libraries, etc. 
    "23:30:00" : 22.44637489318848,  #          Building local time is all that matters.
    "23:45:00" : 22.43587493896484,
}

Trend Log (TL) Object

{
    "mac" : "003018c3184a",    # Ethernet MAC of device sending this report
    "l" : "BRWHQ",             # Device assigned building identifying string
    "d" : 260047,              # Device assigned controller identity number owning the trend log
    "t" : "TL"                 # BACnet memonic for trend log
    "i" : 1,                   # TL identifier within the controller
 
    "ts" : ISODate("2014-12-05T05:06:51.045-08:00"),
 
    "LogInterval" : 3000,      # controller's sampling frequency (in seconds)
    "InputRef" : "260001.AI1.Present_Value",    # I/O point in controller whose values are being sampled
    "Out_Of_Service" : false,
    "Object_Name" : "ANALOG INPUT 1 (Percent) (260001.AI1.Present_Value)",
    "Log_DeviceObjectProperty" : "260001.AI1.Present_Value",
    "Manual_Override" : false,
}

Device (DEV) Object

{
    "mac" : "003018c3184a",    # Ethernet MAC of device sending this report
    "l" : "BRWHQ",             # Device assigned building identifying string
    "d" : 260047,              # Device assigned controller identity number owning the trend log
    "t" : "DEV"                # BACnet memonic for trend log
    "i" : 1,                   # TL identifier within the controller

    "ts" : ISODate("2014-12-05T05:06:50.977-08:00"),
 
    "System_Status" : "Operational",
    "Vendor_Identifier" : 0,   # Vendor's assigned BACnet vendor ID
    "v" : 0,                   # Vendor's assigned BACnet vendor ID
    "Object_Name" : "BRWHQ TR-1",
    "Application_Software_Version" : "1.0",
    "Vendor_Name" : "your company's name",
    "Model_Name" : "your device's model name",
}

Heartbeats

Heartbeats are optional status reports a device may send to inform Kaizen (and more importantly, CopperTree’s Support Staff) of its operational status. At a minimum, a heartbeat shall contain the device’s make, model, some hardware statistics, and some date transmission statistics. Additional fields may be included.

Note: A CopperCube sends more detailed internal information to aid Customer Support with client configuration issues.

Heartbeats are JSON documents sent (via https) to https://kaizen.coppertreeanalytics.com/device_monitoring/heartbeat/post_heartbeat/heartbeat.json

{
    "mac": "00500807d892",          # Device identifier (Ethernet MAC)

    "model": "CopperCube-XL-SQL",   # Basic device Make, Model, and last reboot time
    "version": "1.21.1403",
    "uptime": "2015-07-01 01:39:58",

    # Data transmission statistics
    "samples": {"success": 9650,  "failure": 26 },  # #of trend log reports sent (today).  #of retries encountered
    "objects": {"success": 17091, "failure": 0 },   # #of meta-objects sent (today).  #of retries encountered

    # basic hardware statistics. 
    "cpu": 11,      # CPU load (in %)
    "memory": {"used": 1015, "tot": 1989},  # RAM (in MB)
    "disk": {"used": 13, "tot": 54}  # Disk space (in GB)
}

Device Constraints

• Data collection algorithm – no constraints.
• Internal database – no constraints.
• Data volume (transmitted): no limit.
  • Rule-of-thumb guidelines:
  • Trends: Devices sending data for 5000 Trends is normal.
  • Objects: 10 Trends per device. 500 devices per building are very common.
• Data frequency (transmitted): Trend updated every 4 hours and BACnet Objects once daily is normal.
• Data frequency (sampling rate): max: 1 minute. min: none. typical rates: 15 min, 1 hour, and daily.
• TL types: Polled or COV (change-of-value)

Device Backups

The Device Backup feature is a powerful tool that automatically takes database backup files from building controllers and stores them in secure servers on the cloud. The device backups can then be restored to the controllers (using enteliWEB) in the case of controller failures or to speed up the configuration of a number of similar types of controllers.

The CopperCube performs both incremental and full backups of your controller network. Every day the CopperCube scans your network looking for controllers which have had databases changed. When a change has been found, a backup of that controller is taken and sent to the Kaizen cloud servers. Once a week the CopperCube takes a backup of every controller on your network; this is a full backup of the network.

Backup Feature Procedure

1. Controller Backup is temporarily stored in CopperCube
2. Backup is sent to the Kaizen servers and removed from the CopperCube
3. Backups are then available in Kaizen under the Vault menu 

Device Backup Status

The status of a Device Backup is easily recognized by the 3 colors that represent how current the last backup for that Device is:

•  Green represents a Recent Backup that was made in the last 7 days.

•  Yellow represents a Stale Backup that has had more than 7 days since the last backup of this controller.

•  Red represents that Not Available, that is a backup has never been made. This could be due to the controller not supporting the BACnet service required for this feature, or alternatively, the CopperCube could be having an issue communicating with the controller.

You can quickly show Device Backups of a particular status(es) by clicking one or more of the quick filter buttons located on the top left of the Vault/Device Backups page shown below. These represent the 3 status colors listed above.

The options to the right of the Backup are as follows:

•  Shows you the list of all device backups that you can download individually.

•  Starts downloading the most recent device backup

•  Will delete all backups for this device. Requires password. Warning: Cannot be undone!

Availability of Device Backups

The Device Backup feature allows for the retention of up to 21 backups for each controller going as far back as one year. The one-year date range implies that the Device Backups are self-maintained and that “expired” Device Backups are removed from the server storage. If you wish to retain controller backups for a longer period of time, you can select some or all backups for download or email, and save them on a file server of your own.

Available Device Backups are:

• The 7 most recent daily backups
• The 3 most recent weekly backups
• The 11 most recent monthly backups

Users may select the day of the week for which the weekly backups are taken on the CopperCube; the default is Saturday.