KbA0019: Kaizen Data Retention Policy

Description

Whereas data is needed to perform analytics, there is such a thing as too much data. Data consumes storage space and processing time. And while current data is valuable, it loses its value as it ages. Eventually, the data’s costs exceed its value. At that point, it should be deleted. A data retention policy establishes this cross-over point for the types of data it covers.

In general,

  • data shall be retained only for as long as it is needed.
  • CopperTree collects raw data from client buildings only for the purpose of providing Kaizen services.
  • It analyses the data (in the process of transforming it) to derive information from it.
  • It does not collect data for data’s sake, for pure research purposes, nor for strict archival reasons.
  • Instead, CopperTree collects only subsets of the available data.
  • Transformation of the data generates derivative works, which belong to CopperTree. The raw data remains the property of its owners (the building/users).

CopperTree retains building/user data in accordance with the Service Use Agreement and retains derivative works and intermediate results in accordance with this policy. The goal of the Data Retention Policy is to:

  • honor the Service Use Agreement(s) while
  • ensuring the continued good-working order of the Kaizen System(s) and
  • the ongoing ability of CopperTree to provide its services.

Specific Data Retention Periods

Data categoryRetention while
Active[1]
Retention once
Abandoned[2]
Retention once
Deleted[3]
(Grace period)
Return to Client
upon request[7]
Building Data:    
Regular Trend Logs10 yrs180 days0 days
Calculated Trend Logs10 yrs180 days0 days
Imported Trend Logs10 yrsN/A [4]0 days
Controller Objectsindefinite180 days0 days
Golden Standard Objectsindefinite180 days0 days
Controller Backups[5]1 yr1 yr0 days
Controller Events[6]indefinite180 days0 days
User Data:    
User Logic RulesindefiniteN/A0 days
User Report FormatsindefiniteN/A0 days
System (Equipment) DefinitionsindefiniteN/A0 days
System (Equipment) TagsindefiniteN/A0 days
User Insight templatesindefiniteN/A0 days
Working RulesindefiniteN/A0 days No
Chart TemplatesindefiniteN/A0 days No
Working ChartsindefiniteN/A0 days No
Golden Standard TemplatesindefiniteN/A0 days
Meter definitionsindefiniteN/A0 days
     
Generated Outputs:    
Insights (FDD, Energy, KPI)2 yrs180 days0 days No
Insights (Golden Std, Infrastructure)N/AN/A0 days No
Daily Reports (pdf)up to 60 historical ReportsN/A0 days
Weekly Reports (pdf)up to 60 historical ReportsN/A0 days
Monthly Reports (pdf)up to 60 historical ReportsN/A0 days
Annual Reports (pdf)up to 60 historical ReportsN/A0 days
Email Notifications3 retriesN/A0 days No
     
!CopperTree / System Data :    
Community Logic Rulesindefinite[8]0 days0 days No
Report Formatsindefinite[8]0 days0 days No
Insight templatesindefinite[8]0 days0 days No
System Calculated Trend Logsindefinite0 days0 days No
Chart Templatesindefinite[8]N/A0 days No
Working ChartsindefiniteN/A0 days No
Golden Standard Templates[8]indefiniteN/A0 days No
Weather Trend Logs[8]5 yrs0 days0 days No
System logs7 daysN/A0 days No
  • [1] Active means the data is updated regularly (typically daily, weekly, or monthly).
  • [2] Abandoned means the data has not been updated for an extended time period; significantly longer than its active update frequency.
  • [3] Deleted means requested to be deleted (user-initiated), or marked for deletion (system initiated).
  • [4] Historical Baseline trends are indistinguishable from abandoned since they are never updated.
  • [5] See Controller Backup automated pruning algorithm.
  • [6] Reserved for future use.
  • [7] Returnable but no implied ‘warranty of fitness’ for a particular purpose, nor that the data format will be suitable or compatible with competitive products.
  • [8] When discontinuing an item of this type, CopperTree will attempt to give notice (typically 30 days) before deleting it.

Enforcement Process

The enforcement of these data retention limits may be manual or automated. Where manual, the databases are regularly reviewed looking for data eligible for deletion and then the data is deleted when convenient, thus some data exceeding the described limits may be present at times without implying a change to the policy has been made. Where automated, the data is deleted according to the policy on a daily or weekly basis.

Much of this policy is currently enforced manually.

The following are cases where it is enforced automatically.

Controller Backups automated pruning algorithm

  • a backup is taken from every BAS controller once a week. [typically Saturday].
    • exceptions: those not supporting BACnet FileRead, or password protected.
  • a backup is taken every day the controller database revision changes.
  • daily backups are deleted weekly.
  • 1 backup per week for the last 4 weeks is retained.
  • 1 backup per month for the last 11 months is retained.
  • no backups > 1 yr are retained.
  • A maximum of 21 backups per device is retained.

The policy is automatically enforced every Sunday.