The C3 AI Suite has Types specifically designed to facilitate certain machine learning pipelines. Taking their inspiration from the Scikit learn machine learning pipeline, at the most general, we have the `MLPipe` type defining the most basic behavior of the pipeline with the `train`, `process`, and `score` methods, and various specializations which define a whole pipeline of operations. With this structure, you can call 'train' on the top level pieline Type, and the whole pipeline is trained. Same with 'process', and 'score' to both process inputs and score results. We'll start by discussing the basic types which are available starting with the abstract types forming the basis of this Machine Learning system, then some concrete implementations of those types, and then some examples. We'll then discuss how a new machine learning model may be ported onto the C3 platform.
Abstract Types
MLPipe - An abstract type which, defines general behaviors like train, process, and score. This type is mixed in by nearly all types in the C3 Machine Learning Space.
MLPipeline - An abstract type which mixes MLPipe, and connects multiple steps into a full pipeline., The provided concrete implementation of this type is `MLSerialPipeline`. The MLPipeline contains a list of MLSteps to perform.
MLStep - A helper type to allow MLSerialPipeline and other implementations of MLPipeline store their step pipelines and metadata.
MLLeafPipe - This is an abstract type mixing MLPipe which is meant for steps in a machine learning pipeline which have no child steps, i.e. this step performs a specific action of the pipeline. In the terminology of the C3 AI Suite, this is a 'leaf' of the pipeline. There are several concrete versions of this type, and they are usually different implementations within various machine learning systems.
CustomPythonMLPipe - A helper type to act as a 'base' for defining new python based machine learning Pipes.
Concrete Types
MLSerialPipeline - This is the concrete implementation of the MLPipeline Type. Since MLSerialPipeline is so general, you won't have to subclass it.
SklearnPipe - This concrete implementation of MLLeafPipe provides a straightforward way to use sklearn machine learning pre-processing and modelling functions.
TensorflowClassifier - (Concrete MLLeafPipe) This Type allows the user to store a tensorflow estimator-based model, specifically a classifier. This Type currently works with tensorflow 1.9.0
TensorflowRegressor - (Concrete MLLeafPipe) This Type allows the user to store a tensorflow estimator-based model, specifically a regressor. This Type currently works with tensorflow 1.9.0
XgBoostPipe - (Concrete MLLeafPipe) This Type implements the sklearn-compatible part of the Xgboost library.
Usage Examples
Example Notebooks
Several jupyter notebooks exist which demonstrate the usage of these Pipeline types. We list directions to each here.
C3.ai developed notebooks:
- https://<vanity_url>/jupyter/notebooks/tutorials/TutorialIntroMLPipeline.ipynb
- https://<vanity_url>/jupyter/notebooks/tutorials/TutorialKerasPipe.ipynb
- https://<vanity_url>/jupyter/notebooks/tutorials/TutorialMLDataStreams.ipynb
- https://<vanity_url>/jupyter/notebooks/tutorials/TutorialProphetPipe.ipynb
- https://<vanity_url>/jupyter/notebooks/tutorials/TutorialStatsModelsTsaPipe.ipynb
- https://<vanity_url>/jupyter/notebooks/tutorials/TutorialTensorflowPipe.ipynb
- https://<vanity_url>/jupyter/notebooks/tutorials/TutorialXgBoostMLPipeline.ipynb
DTI developed notebooks:
- MNIST example: https://github.com/c3aidti/mnistExample
C3.ai Resources related to Machine Learning
- Developer Documentation
- C3.ai Academy Videos