...

• Data Integration (Not yet available)
• Metrics (Not yet available)
• Machine Learning (Not yet available)

Terminology

To best understand the C3 AI Suite and this guide, let's introduce key terminology used by C3 AI Suite developers:

• Type: Nearly all aspects of the C3 AI Suite (e.g., data, machine learning models, cloud-provider microservices) are stored and accessed through Types. C3 AI Types are logical objects akin to a Java class, and contain 'fields' and 'methods'. Some Types are persisted to internal databases (like Postgres or Cassandra), while others are not. 
• Field: A field of a Type. Fields contain attributes or data associated with the Type.
• Method: A method declared on a Type. Methods define business logic associated with the Type. 
• Vanity Url: The URL at which a specific tenant/tag of a C3 Cluster can be accessed. 
• Cluster: A deployment of the C3 AI Suite. A C3 Cluster is a collection of hardware or virtualized cloud instances (e.g., servers, databases, load balancers) used to run the C3 AI Suite and C3 AI Applications. The C3 AI Suite can run on any public or private cloud infrastructure or on a local machine (in a docker container).
• Tenant: A logical partition of a C3 Cluster. All tenants in a cluster generally share the same compute and storage resources. Data within tenants, while stored in a single database, are logically separated. C3 AI Suite users on one tenant can't see data stored on another tenant. In other words, users are only able to view data for tenants to which they are explicitly granted access.
• Tag: A logical partition of a tenant. A single tag hosts one C3 AI application (deployed package). 
• Package: All the code a developer writes for an application. The C3 AI Suite runs a package on a tag. 
• Provisioning: Deploying a package onto a tenant/tag in a C3 Cluster. 
• Static Console: A browser-based tool that C3 AI developers use to interact with the C3 AI Suite. Static console is available on all modern browsers, including Google Chrome, Mozilla Firefox, and Apple Safari. Developers use the console to query data, evaluate expressions, view documentation, monitor environments, develop and test code, and manage environment configurations. You can access the static console at the url 'https://<vanity_url>/static/console' (replacing <vanity_url> with your Vanity Url).
• Metric: Expressions that transform raw data into a time-series. 

C3 Cluster Overview

The C3 AI Suite is a Platform as a Service (PaaS), which enables organizations to build, deploy, and operate enterprise-scale Big Data, AI, and IoT applications. The C3 AI Suite can be deployed on any private or public cloud infrastructure such as AWS, Azure, and Google Cloud Platform. When developing and operating applications, a C3 cluster is responsible for managing and supporting all the features of the C3 AI Suite. A C3 Cluster has at least one Master node and many Worker nodes. Master nodes prioritize and distribute jobs to Worker nodes and handle user requests. Worker nodes carry out jobs, allocated by the Master node. Other components of a C3 Cluster include databases (e.g., Postgres, Cassandra, Azure Blob), logging services (i.e., Splunk), and Jupyter. Atop these hardware or virtualized cloud resources is a logical software structure, with the highest level being a Cluster. A C3 Cluster is broken out into numerous tenants. Tenants are logically separated from each other (i.e., a particular tenant's data and packages are not accessible or visible to any other tenants), and contain many tags. Tags host C3 AI Packages (i.e., the code that C3 AI developers write and provision to the C3 AI Suite). A typical multi-tag, multi-tenant C3 Cluster is shown in a logical diagram below:

...

• Developer Documentation
  • https://developer.c3.ai/docs/7.12.25/guide/guide-c3aisuite-basic/arch-home
  • https://developer.c3.ai/docs/7.12.25/guide/guide-c3aisuite-basic/arch-clusters-tenants-and-tags
• C3.ai Academy Videos:
  • Packages
  • Clusters, Tenants, and Tags

Provision a C3 AI Package

To provision a package to your tag, follow the instructions available at the DTI Guide: Provisioning.

...

• Developer Documentation
  • https://developer.c3.ai/docs/7.12.17/topic/arch-provisioning
• C3.ai Academy Videos:
  
  • Provisioning

Connecting to the C3 AI Suite 

The static console is the main tool that developers use to interact with the C3 AI Suite. However, we anticipate that most DTI members will use Python (via Jupyter notebook) for data analysis. That being said, the static console is an essential part of working with the C3 AI Suite and you will use it frequently. For example, the static console is the best place to find documentation tailored directly to your package. It's also a great place to quickly test queries as no specialized environments need to be set up to use it. Static console is ready-to-go in all modern browsers, including Google Chrome, Mozilla Firefox, and Apple Safari.

Accessing the Static Console

Once you have provisioned a package to your tag, navigate to the static console page at this url: 'https://<vanity_url>/static/console' (replacing <vanity_url> with your Vanity Url provided in your C3.ai DTI Training Cluster Onboarding Email). The static console page looks like this:

...

Most tools are also accessible through a series of icons in the upper right-hand corner:

Using the Static Console

Developers interact with the static console through the JavaScript console tab in the browser. When the static console page loads (or when you run the c3ImportAll() command), JavaScript methods associated with all of your Package's defined Types are populated. You can write and run JavaScript code directly in the console tab to interact with your package.

...

Finally, let's write some JavaScript commands to see the console in action!

Tutorial Video

The DTI Team have recorded a short video introducing and describing the static console functionality:

Console Commands

Here are common JavaScript console commands used on the static console page.

• c3ImportAll: A console command which loads all the C3 Types from your package into the static console. Always be sure refresh the static console page or run c3ImportAll() after provisioning a new package.
• c3Grid: A console command to display a table of data stored within a C3 Type. (e.g., data returned from a fetch operation, or an evaluate operation among many others).
• c3Viz: A console command to produce quick visualizations or plots for some C3 Types. (e.g., time-series data like EvalMetricsResult)
• c3ShowType: A console command to access documentation for a given C3 Type. (e.g., c3ShowType(OutbreakLocation))

Official C3 AI Training Materials on Static Console

• Developer Documentation
  • https://developer.c3.ai/docs/7.12.25/topic/console-home
  • https://developer.c3.ai/docs/7.12.25/topic/console-documentation-via-console
• C3.ai Academy Videos:
  
  • Console Overview
  • Console Orientation

Using Python with the C3 AI Suite

We anticipate most DTI researchers will want to use Python for data analysis. There are two options to connect to a C3 Cluster via Python. Please follow the links below for detailed information about each.

...

• Developer Documentation
  • https://developer.c3.ai/docs/7.12.25/guide/guide-c3aisuite-basic/ds-home
  • https://developer.c3.ai/docs/7.12.25/guide/guide-c3aisuite-basic/ds-python-apis
  • https://developer.c3.ai/docs/7.12.25/guide/guide-c3aisuite-basic/ds-home
  • https://developer.c3.ai/docs/7.12.25/guide/guide-c3aisuite-basic/ds-jupyter-notebooks
  • https://developer.c3.ai/docs/7.12.25/guide/guide-c3aisuite-basic/actionruntime
• C3.ai Academy Videos:
  
  • Python APIs
  • Python Runtimes

Fetching Instances of C3 Types

All data in the C3 AI Suite are stored in C3 Types. Users can access data from a Type with the 'fetch' method. Behind the scenes, the 'fetch' method submits a query directly to the database underlying a Type, and retrieves and presents the query results.

...

Note: Please see the official FetchSpec documentation for a full list of parameters: https://developer.c3.ai/docs/7.12.17/type/FetchSpec

Examples of Fetch Calls

The OutbreakLocation Type contains information from various locations for which the Covid-19 Data Lake has virus-related information. We can fetch OutbreakLocation records for which the 'latestTotalPopulation' field exists (i.e., is not null). We can also retrieve these records in descending order by their 'countryArea':

...

• https://github.com/c3aidti/casesExample
• https://github.com/c3aidti/genomicsExample
• https://github.com/c3aidti/HouseCoverageExample
• https://github.com/c3aidti/VaccineListing
• https://github.com/c3aidti/EpidemiologyExample

Tutorial Video

This tutorial video goes over fetching and filtering:

The fetchCount Method

Another useful command is 'fetchCount'. Like 'fetch', users can also provide a FetchSpec (or parameters) to 'fetchCount'. The 'fetchCount' method then returns the number of records that match the FetchSpec. This is useful when trying to determine whether a given search is refined enough.

...

To learn more about the 'fetchCount' method, please see the fetchCount method definition in the Persistable Type documentation: https://developer.c3.ai/docs/7.12.25/type/Persistable

Converting Fetch results to usable forms in Jupyter Notebook

When using a Jupyter Notebook, C3 AI developers typically modify FetchResults for data analysis. This section shows a couple of ways to convert FetchResults into easy-to-analyze forms.

Python

In python, first retrieve the 'objs' field from the FetchResults object, and then call the toJson() function. The toJson() function returns an array of dictionaries each with keys equal to the requested fields of the fetched C3 Type. Using the Pandas library, this array can be turned into an analysis-ready DataFrame, as the below example shows:

...

Users can then manipulate the resulting DataFrame, using common programming libraries and frameworks.

ExpressionEngineFunctions

The C3 AI Suite also provides a pre-built library of "ExpressionEngineFunctions". Expression Engine Functions take a variety of arguments and perform various data processing tasks. For example, the function 'contains' takes two strings as arguments and checks whether the first argument contains the second argument. The function 'lowerCase' takes as input a string and returns that same string with all lowercase letters. In addition to these string processing functions, the C3 AI Suite's ExpressionEngine includes many math functions (such as 'log', 'avg', and 'abs') which operate on various input data types (e.g. int, double, float).

...

• Developer Documentation

  • https://developer.c3.ai/docs/7.12.25/type/ExpressionEngineFunction

• C3.ai Academy Videos:
  
  • ExpressionEngineFunctions

Computations on C3 Types using Evaluate

Using the 'evaluate' method, developers can run aggregations or other computations on data fetched from a C3 Type. (e.g., compute the average area across all countries with area data available, in the OutbreakLocation Type).

...

• Developer Documentation
  • https://developer.c3.ai/docs/7.12.25/topic/tutorial-the-evaluate-method
  • https://developer.c3.ai/docs/7.12.25/topic/console-evaluate-method
• C3.ai Academy Videos:
  
  • The Evaluate Method

Developing Metrics on Timeseries data

The C3 AI Suite also offers several features to handle time series data. To interact with time series, C3 AI developers typically use simple and compound metrics. These metrics are used in several places in the C3 AI Suite such as:

• Alerts and Application Logic
• Machine Learning Features
• User Interface (to Visualize Data)

Timeseries Video Tutorial

To supplement the documentation below, we also have recorded a video lecture about Time Series data on the C3 AI Platform.

Simple Metrics

Simple metrics allow developers to produce time-series from raw data and are often used to construct more advanced metrics (i.e., Compound Metrics), in practice. Simple metrics are linked to a specific C3 Type and reference the timeseries data stored within that Type. To declare a simple metric, users should specify the following fields:

...

• Developer Documentation
  • https://developer.c3.ai/docs/7.12.25/topic/metrics-tsdecl-metrics
  • https://developer.c3.ai/docs/7.12.25/topic/tutorial-tsdecl-metrics
• C3.ai Academy Videos:
  
  • Time Series Declaration Metrics

Compound Metrics

Compound metrics allow C3 AI developers to manipulate or combine existing metrics into more complex time series. Compound metrics are built on top of one or many existing Simple or Compound metrics. Please note, to evaluate a Compound metric on a C3 Type, all Simple metrics used in that Compound metric must be defined on that Type as well. If not, an error is returned.

...

• Developer Documentation
  • https://developer.c3.ai/docs/7.12.25/topic/metrics-compound-metrics
  • https://developer.c3.ai/docs/7.12.25/topic/tutorial-compound-metrics
• C3.ai Academy Videos:
  
  • Compound Metrics
    

Finding, Evaluating, and Visualizing Metrics

Users can find, evaluate, and visualize metrics built in the C3 AI Suite via the JavaScript console or a hosted Jupyter notebook.

Finding Metrics

All metrics that users build and deploy in the C3 AI Suite are also stored in C3 Types. To view a list of all the simple and compound metrics applicable to a C3 Type, run the 'listMetrics' method as shown below:

...

After finding a metric, the next step is to evaluate on data in a C3 Type.

Evaluating Metrics

Metrics are evaluated with either the 'evalMetrics' or 'evalMetricsWithMetadata' methods. Behind the scenes, 'evalMetrics' and 'evalMetricsWithMetadata', fetch and transform raw data from a C3 Type into easy-to-analyze timeseries data. 'evalMetrics' is used to evaluate metrics provisioned (deployed) to a tenant/tag. 'evalMetricsWithMetadata' allows users to evaluate metrics either provisioned to a tenant/tag, or defined on-the-fly in JavaScript console, or a hosted Jupyter notebook (typically for debugging).

...

Note: Metrics can only be evaluated on C3 Types that mix in the 'MetricEvaluatable' Type.

Additional Resources

Official C3 AI Developer Documentation:

• Timeseries Normalization
• Timeseries Treatments
• Timeseries Metrics
• Simple Metrics
• Compound Metrics

Review and Next Steps

For most data analysis, C3 AI developers run the 'fetch' and 'evalMetrics' methods. This C3.ai DTI Quickstart guide provides an introduction to these methods in which the C3 AI Suite is used as a read-only database accessed via APIs. In the following guides, you will learn how to run 'write' operations on the C3 AI Suite such as:

...