Versions Compared

Old Version 78

changes.mady.by.user Sohaib Shaikh

Saved on

compared with

New Version 79

changes.mady.by.user Sohaib Shaikh

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Some example FetchSpec parameters include:

  • filter: Filter expression to return a subset of the data (e.g., age <= 20). Filter expressions must evaluate to a Boolean type (i.e., true or false)
  • limit: the maximum number of rows that should be returned. Be default, if no limit is specified, the C3 AI Suite returns 2,000 rows from the C3.ai Type. Specifying a limit is often helpful to debug a fetch 'method', without returning too many records.
  • include: Specifies the particular fields from a C3.ai Type to return to the FetchResult. By default, if no include spec is defined, all fields from the C3.ai Type will be returned.
  • order: Specifies the order to return the query's results (either "ascending" or "descending")

...

Another useful command is 'fetchCount'. Like 'fetch', users can also provide a FetchSpec (or parameters) to 'fetchCount'. The 'fetchCount' method then returns the number of records that match the FetchSpec. This is useful when trying to determine whether a given search is refined enough.

Code Block
languagejs
OutbreakLocation.fetchCount({'filter': 'exists(latestTotalPopulation)'})

The same command You can run the same 'fetchCount' in python is: 

Code Block
languagepy
c3.OutbreakLocation.fetchCount(spec={'filter': 'exists(latestTotalPopulation)'})

...

When using a Jupyter Notebook, C3.ai developers typically modify FetchResults for data analysis. This section shows a couple of ways to convert FetchResults into easy-to-analyze forms.

...

The C3 AI Suite also provides a pre-built library of "ExpressionEngineFunctions". Expression EngineFunctions take a variety of arguments and perform various data processing tasks. For example, the function 'contains' takes two strings as arguments, and checks whether the first argument contains the second argument. The function 'lowerCase' takes as input a string, and returns that same string with all lowercase letters. In addition to these string processing functions, the C3 AI Suite's ExpressionEngine includes many math functions such as 'log', 'avg', and 'abs', which operate on a various input data types (e.g. int, double, float).

...

...

Computations on C3.ai Types using Evaluate

The C3 AI Suite provides Using the 'evaluate' method to compute simple expressions on the data stored within a C3 , C3.ai developers can run aggregations or other computations on data fetched from a C3.ai Type. (e.g., compute the average area of across all OutbreakLocations which are countries and for which we have area information)countries with area data available, in the OutbreakLocation Type).

The 'evaluate' method takes several The evaluate function takes the parameters:

  • 'projection': [Required] A comma-separated list of valid expressions or ExpressionEngineFunctions to evaluate on the aggregated Type data. Behind the scenes, the C3 AI Suite translates these expressions to necessary SQL queries, but not all ExpressionEngineFunctions can be evaluated in SQL. In these cases, evaluate will try to do this itself, but without other SQL abilities like grouping or ordering.
  • 'group': A comma separated list of valid expressions or ExpressionEngineFunctions to evaluate as a group parameter of the SQL query
  • expressions (from the ExpressionEngineFunction library) to apply to data from a C3.ai Type (e.g., avg, unique, min, max). You can simply think about a projection as the columns/fields, calculated or otherwise, which the "evaluate" method should return.
  • group: A comma-separated list of columns/fields, to group the aggregated/transformed data by (e.g, compute the average area by the 'locationType' field in OutbreakLocation). Please note, in any 'evaluate' command, all columns in the 'group' field MUST ALSO BE in the 'projection' field, as the example below shows. 
  • having: A 'having': An SQL style having clause.
  • 'order': A comma-separated list of valid expressions or ExpressionEngineFunctions to perform an ordering of the results by
  • 'filter': A fetch filter expression which restricts the rows evaluate is run against.
  • columns/fields, to order aggregated/transformed data by. Users can return data in 'ascending' or 'descending' order. Please note, in any 'evaluate' command, all columns in the 'order' field MUST ALSO BE in the 'projection' field.
  • filter: A filter expression that restricts the rows in a C3.ai Type, on which the evaluate method is run. 

In static console, 'c3Grid' displays the 'evaluate' methodOn the static console, using the 'c3Grid' displays evaluate's result nicely:

Code Block
languagejs
var eval_result = OutbreakLocation.evaluate({
    'projection': 'avg(countryArea), locationType',
    'group': 'locationType',
    'filter': 'exists(countryArea) && exists(locationType)'
})
c3Grid(eval_result)


We Users can also use run the 'evaluate' method in Python, but we have to use a helper function. We've defined this for you with the python. In this case, users often modify the 'evaluate' method's results for data analysis. To view and analyze the 'evaluate' method's results in Python, please use the helper function available in DTI's c3python module available here: https://github.com/c3aidti/c3python

Code Block
languagepy
eval_spec = {
    'projection': 'avg(countryArea), locationType',
    'group': 'locationType',
    'filter': 'exists(countryArea) && exists(locationType)'
}
eval_res = c3.OutbreakLocation.evaluate(eval_spec)
df = c3python.EvaluateResultToPandas(result=eval_res, eval_spec=eval_spec)


Here's another example of running the 'evaluate' method in Python:

Code Block
languagepy
spec = c3.EvaluateSpec(
    projection="ethnicity, count(ethnicity)",
    group="ethnicity"
)
c3python.EvaluateResultToPandas(result=c3.SurveyData.evaluate(spec), eval_spec=spec)


To learn more about the 'evaluate' method, please see the C3.ai resources here:

  • Developer Documentation

...

Developing Metrics on Timeseries data

The C3 AI Suite also offers several features to handle timseries time series data. To interact with timeseries time series C3.ai developers typically use simple and compound metrics. These metrics are used in several places in the C3 AI Suite such as:

...

Simple metrics allow C3.ai developers to produce timeseries time-series from raw data , and are often used to construct more advanced metrics (i.e., Compound Metrics), in practice. Simple metrics are linked to a specific C3.ai Type and reference the timeseries data stored within that C3.ai Type. To declare a simple metric, users should specify the following fields:

  1. id: simple metric's unique id, which should follow the convention "name_srcType" (e.g., Apple_DrivingMobility_OutbreakLocation)
  2. name: simple metric's name (e.g., Apple_DrivingMobility)
  3. description: simple metric's description (optional field)
  4. srcType: the C3.ai Type the simple metric is analyzed on (e.g., OutbreakLocation)
  5. path: path from the srcType to the C3.ai Type, that stores the raw data referenced by the simple metric (e.g., pointMeasurements) Note: if the srcType itself stores the raw data referenced by the simple metric, path field is optional.
  6. expression: the expression (or ExpressionEngineFunction) applied to the raw data, referenced by the simple metric (e.g., avg(avg(normalized.data.quantity)). Note: the "normalized" key term, instructs the simple metric to use normalized (instead of raw) data on the C3 AI Suite (to learn more about Normalization, see this C3.ai Developer Documentation: https://developer.c3.ai/docs/7.12.17/topic/normalization )

...

Another type of SimpleMetric is a tsDecl (Timeseries Declaration) metric. tsDecl metrics are often used to turn non-timeseries time series raw data (e.g., event data, status data, or data with irregular intervals) into timeseriestime series. tsDecl metrics have the same fields as standard SimpleMetric, except for the 'tsDecl' field, which replaces the 'expression' field. tsDecl metrics may allow users the added flexibility to define new metrics which that the expression field may not support. Using a tsDecl metric, the above metric can be re-written as:

...

Compound metrics allow C3.ai developers to manipulate or combine existing metrics into more complex timeseriestime series. Compound metrics are built on top of one or many existing Simple or Compound metrics. Please note, to evaluate a Compound metric on a C3.ai Type, all Simple metrics, used in that Compound metric, must be defined on that C3.ai Type, as well. Otherwise, an error is returned.

To declare a compound metric, users should specify the following fields:

  1. 'id': compound metric's unique id, typically the same as 'name' (e.g., BLS_UnemploymentRate)
  2. 'name': compound metric's name (e.g., BLS_UnemploymentRate)
  3. description: compound metric's description (optional field)
  4. expression: the expression (or ExpressionEngineFunction) applied to the metrics underlying the Compound metric (e.g., "BLS_LaborForcePopulation ? 100 * BLS_UnemployedPopulation / BLS_LaborForcePopulation: null")

...

To evaluate a metric, users must provide the following parameters (called an EvalMetricSpec) to the 'evalMetrics' or 'evalMetricsWithMetadata' methods.

  1. ids ([string]): A list of ids in the C3.ai Type, on which you want to evaluate the metrics (e.g., "Germany", "California_UnitedStates")
  2. expressions ([string]): A list of metrics to evaluate (e.g., "JHU_ConfirmedCases", "Apple_DrivingMobility")
  3. start (datetime): Start datetime of the time range to be evaluated (in ISO 8601 format) (e.g., "2020-01-01")
  4. end (datetime): End datetime of the time range to be evaluated (in ISO 8601 format) (e.g., "2020-08-01")
  5. interval (string): Desired interval for the resulting timeseries data (e.g., MINUTE, HOUR, DAY, MONTH, YEAR)

...

Note: Metrics can only be evaluated on C3.ai Types that mix in the 'MetricEvaluatable' Type.

...

Additional Resources

Official C3.ai Developer Documentation:

Review and Next Steps


In For most data analysis, C3.ai developers run the 'fetch' and 'evalMetrics' methods. This C3.ai DTI Quickstart guide provides an introduction to these methods, in which the C3 AI Suite is used as a read-only database, accessed via APIs. In the following guides, you will learn how to run 'write' operations on the C3 AI Suite such as:

...