KbA0019: Kaizen Data Retention Policy

Description

Whereas data is needed to perform analytics, there is such a thing as too much data. Data consumes storage space and processing time. And while current data is valuable, it loses its value as it ages. Eventually, the data’s costs exceed its value. At that point, it should be deleted. A data retention policy establishes this cross-over point for the types of data it covers.

In general,

  • data shall be retained only for as long as it is needed.
  • CopperTree collects raw data from client buildings only for the purpose of providing Kaizen services.
  • It analyses the data (in the process of transforming it) to derive information from it.
  • It does not collect data for data’s sake, for pure research purposes, nor for strict archival reasons.
  • Instead, CopperTree collects only subsets of the available data.
  • Transformation of the data generates derivative works, which belong to CopperTree. The raw data remains the property of its owners (the building/users).

CopperTree retains building/user data in accordance with the Service Use Agreement and retains derivative works and intermediate results in accordance with this policy. The goal of the Data Retention Policy is to:

  • honor the Service Use Agreement(s) while
  • ensuring the continued good-working order of the Kaizen System(s) and
  • the ongoing ability of CopperTree to provide its services.

Specific Data Retention Periods

Data category Retention while
Active[1]
Retention once
Abandoned[2]
Retention once
Deleted[3]
(Grace period)
Return to Client
upon request[7]
Building Data:        
Regular Trend Logs 10 yrs 180 days 0 days
Calculated Trend Logs 10 yrs 180 days 0 days
Imported Trend Logs 10 yrs N/A [4] 0 days
Controller Objects indefinite 180 days 0 days
Golden Standard Objects indefinite 180 days 0 days
Controller Backups[5] 1 yr 1 yr 0 days
Controller Events[6] indefinite 180 days 0 days
User Data:        
User Logic Rules indefinite N/A 0 days
User Report Formats indefinite N/A 0 days
System (Equipment) Definitions indefinite N/A 0 days
System (Equipment) Tags indefinite N/A 0 days
User Insight templates indefinite N/A 0 days
Working Rules indefinite N/A 0 days  No
Chart Templates indefinite N/A 0 days  No
Working Charts indefinite N/A 0 days  No
Golden Standard Templates indefinite N/A 0 days
Meter definitions indefinite N/A 0 days
         
Generated Outputs:        
Insights (FDD, Energy, KPI) 2 yrs 180 days 0 days  No
Insights (Golden Std, Infrastructure) N/A N/A 0 days  No
Daily Reports (pdf) up to 60 historical Reports N/A 0 days
Weekly Reports (pdf) up to 60 historical Reports N/A 0 days
Monthly Reports (pdf) up to 60 historical Reports N/A 0 days
Annual Reports (pdf) up to 60 historical Reports N/A 0 days
Email Notifications 3 retries N/A 0 days  No
         
!CopperTree / System Data :        
Community Logic Rules indefinite[8] 0 days 0 days  No
Report Formats indefinite[8] 0 days 0 days  No
Insight templates indefinite[8] 0 days 0 days  No
System Calculated Trend Logs indefinite 0 days 0 days  No
Chart Templates indefinite[8] N/A 0 days  No
Working Charts indefinite N/A 0 days  No
Golden Standard Templates[8] indefinite N/A 0 days  No
Weather Trend Logs[8] 5 yrs 0 days 0 days  No
System logs 7 days N/A 0 days  No
  • [1] Active means the data is updated regularly (typically daily, weekly, or monthly).
  • [2] Abandoned means the data has not been updated for an extended time period; significantly longer than its active update frequency.
  • [3] Deleted means requested to be deleted (user-initiated), or marked for deletion (system initiated).
  • [4] Historical Baseline trends are indistinguishable from abandoned since they are never updated.
  • [5] See Controller Backup automated pruning algorithm.
  • [6] Reserved for future use.
  • [7] Returnable but no implied ‘warranty of fitness’ for a particular purpose, nor that the data format will be suitable or compatible with competitive products.
  • [8] When discontinuing an item of this type, CopperTree will attempt to give notice (typically 30 days) before deleting it.

Enforcement Process

The enforcement of these data retention limits may be manual or automated. Where manual, the databases are regularly reviewed looking for data eligible for deletion and then the data is deleted when convenient, thus some data exceeding the described limits may be present at times without implying a change to the policy has been made. Where automated, the data is deleted according to the policy on a daily or weekly basis.

Much of this policy is currently enforced manually.

The following are cases where it is enforced automatically.

Controller Backups automated pruning algorithm

  • a backup is taken from every BAS controller once a week. [typically Saturday].
    • exceptions: those not supporting BACnet FileRead, or password protected.
  • a backup is taken every day the controller database revision changes.
  • daily backups are deleted weekly.
  • 1 backup per week for the last 4 weeks is retained.
  • 1 backup per month for the last 11 months is retained.
  • no backups > 1 yr are retained.
  • A maximum of 21 backups per device is retained.

The policy is automatically enforced every Sunday.