...
Please see this C3.ai Developer Documentation for a full list of the C3 AI Suite's ExpressionEngineFunctions: https://developer.c3.ai/docs/7.12.0/type/ExpressionEngineFunction
...
Developing Metrics on
...
Timeseries data
C3 can store timeseries data using many different types, however knowing the exact type of timeseries data isn't super important when it comes to evaluating so-called 'Metrics' on that data.
Normalization Process
Usually, timeseries data goes through a 'normalization' process, the purpose of which is to take non-uniform, and possibly multiple datasets and produce a single uniform timeseries which can be analyzed a little more easily in most cases. We copy here the list of normalization steps that are currently performed within the C3 platform, these are available from C3's official documentation here: https://developer.c3.ai/docs/7.12.0/guide/guide-c3aisuite-basic/ts-normalization-engine
- Drop data points with irregular dates. For example, dates where start date is after end date, dates are > 50 years apart, etc.
- Remove duplicate data points that might have been sent due to data loading issues or issues with IoT sensor hardware.
- Correctly apportion the values in the correct time interval in case of overlapping data points.
- Convert data points in various units into a homogenous unit utilizing C3's unit conversion capabilities.
- Automatic detection of the natural frequency of the data.
- Aggregate or disaggregate data into coarse or finer intervals to optimize for storage and accuracy.
Once the normalization process is complete, a single time series sampled at a uniform interval is given.
SimpleMetrics
Simple metrics form the 'base' of the Metrics system. They are defined on a specific Type and reference timeseries data stored within. Essentially, the Simple metric defines:
- The Type on which the metric is defined
- How to find the timeseries data on the Type
- Configuration of the Normalization engine
- The name of the metric
The C3 AI Suite also offers several features to handle timseries data. To interact with timeseries C3.ai developers typically use simple and compound metrics. These metrics are used in several places in the C3 AI Suite such as:
- Alerts and Application Logic
- Machine Learning Features
- User Interface (to Visualize Data)
Simple Metrics
Simple metrics allow C3.ai developers to produce timeseries from raw data, and are often used to construct more advanced metrics (i.e., Compound Metrics), in practice. Simple metrics are linked to a specific C3.ai Type and reference the timeseries data stored within that C3.ai Type. To declare a simple metric, users should specify the following fields:
- id: simple metric's unique id, which should follow the convention "name_srcType" (e.g., Apple_DrivingMobility_OutbreakLocation)
- name: simple metric's name (e.g., Apple_DrivingMobility)
- description: simple metric's description (optional field)
- srcType: the C3.ai Type the simple metric is analyzed on (e.g., OutbreakLocation)
- path: path from the srcType to the C3.ai Type, that stores the raw data referenced by the simple metric (e.g., pointMeasurements) Note: if the srcType itself stores the raw data referenced by the simple metric, path field is optional.
- expression: the expression (or ExpressionEngineFunction) applied to the raw data, referenced by the simple metric (e.g., avg(avg(normalized.data.quantity)). Note: the "normalized" key term, instructs the simple metric to use normalized (instead of raw) data on the C3 AI Suite (to learn more about Normalization, see this C3.ai Developer Documentation: https://developer.c3.ai/docs/7.12.17/topic/normalization )
Here is an example of a Simple MetricAn example SimpleMetric is:
| Code Block | ||
|---|---|---|
| ||
sample_met = c3.SimpleMetric({
'id': 'SampleMetric_SampleType',
'name': 'SampleMetric',
'srcType': 'SampleType',
'namepath': 'SampleMetrictimeseriesValues',
'srcTypeexpression': 'SampleType',
'path': 'timeseriesValues',
'expression': 'avg(avg(normalized.data.x))'
})avg(avg(normalized.data.x))'
}) |
To learn more about Simple Metrics, please see the C3.ai Developer Documentation here:
https://developer.c3.ai/docs/7.12.17/topic/tutorial-simple-metrics
- https://developer.c3.ai/docs/7.12.17/topic/metrics-simple-metrics
Another type of SimpleMetric is a tsDecl (Timeseries Declaration) metric. tsDecl metrics are often used to turn non-timeseries raw data (e.g., event data, status data, or data with irregular intervals) into timeseries. tsDecl metrics have the same fields as standard SimpleMetric, except for the 'tsDecl' field, which replaces the 'expression' field. tsDecl metrics may allow users the added Another variety of SimpleMetric is a tsDecl Metric. These can be used to turn traditionally non-timeseries data such as event data or status data into timeseries. A tsDecl metric is the same as a SimpleMetric, but instead of an 'expression' field, a 'tsDecl' field is used. tsDecl metrics can sometimes provide some additional flexibility to define new metrics which the expression field may not support. The same example Using a tsDecl metric, the above metric can be re-written as:
| Code Block |
|---|
sample_met = c3.SimpleMetric({
'id': 'SampleMetric_SampleType',
'name': 'SampleMetric',
'srcType': 'SampleType',
'path': 'timeseriesValues',
'tsDecl': {
'data': 'data',
'treatment': 'AVERAGE',
'start': 'start',
'value': 'value'
}
}) |
Please note that the above examples do not have an example context in which they work. This will be updated soon with a version backed up by a working exercise.
For more detail, To learn more about tsDecl metrics, please see the C3 documentation on SimpleMetrics .ai Developer Documentation here:
...
...
...
...
...
CompoundMetricsCompound Metrics
Compound metrics are generally easier to define and use as they operate on already defined metrics either Simple or Compound. They essentially just consist of and id/name, and an expression defining the metric in terms of constants and already defined metrics. If you try and execute a CompoundMetric on a type for which some necessary SimpleMetric is not defined, you'll get an error.
...