You have been given a grant as part of the new C3 Digital Transformation Institute (DTI)!
To make the start of your DTI experience as painless as possible, we have assembled a set of resources to
If you have questions not covered by this guide, please contact the DTI team at the email help+c3ai@ncsa.illinois.edu
C3 is a Java-based data analytics engine designed to make the ingestion and analysis of heterogeneous data sources
as painless as possible. The C3 system joins data from multiple sources into a single unified federated data image.
With the federated data image defined, C3 then provides an API to access that data, and in the case of time-series data,
perform numerous transformations and computations all producing normalized time-series data at regular intervals.
C3 also supports R and Python Jupyter notebook analysis of the federated data image. These notebooks provide a
great way for researchers to analyze data close to where the data is stored. While C3 supports many data science
capabilities familiar to the researcher, some expected functionality may be missing. For these cases, C3 supports
implementing new data processing functions in python and javascript.
Like any other API porting your own workflows will take some care and time to learn properly. Please leverage this guide
to make understanding C3's platform and porting your workflow as quick and easy as possible.
Use this guide to determine what training you need to utilize C3's resources effectively. We have separated
researchers into four levels based on what level of interaction with C3's resources they require. We include
basic examples of workflows which might fall into that level, pros and cons of operating on that level, and
a list of training resources we recommend resources researchers completing on the DTI training environment
before starting their C3 allocations. This will ensure researchers will be able to use their allocation as
efficiently as possible.
Examine the high level overviews of each level below, then click the section titles to go to more in-depth
discussions related to that level, like the recommended training.
For many researchers, they will simply want to leverage the C3 COVID-19 Federated Data Image.
Pros:
Cons:
C3 provides a wonderful GUI based interfaced to the C3 system with their Integrated Development Studio. Such
an environment is likely to be attractive to many researchers.
Pros:
Cons:
Some researchers will want to write their own C3 package and leverage more of the AI Suite through Jupyter notebooks.
C3 allows researchers to define their own types, methods, and use R and python to perform analysis on Datalake data.
Pros:
Cons:
Some researchers will want to bring state-of-the-art ML workflows to C3. C3 can support such workflows, but
extra work may be needed.
Pros:
Cons:
This section introduces the process to access C3. Generally speaking, once you receive your grant,
the DTI team will reach out and discuss with you what your needs are.
C3 is quite different from traditional HPC resources. We have written an introduction to C3 from the
perspective of a scientific researcher. We go over several important C3 concepts and relate them to
what scientists are more familiar with.
This section introduces How researchers will be expected to manage their allocation while on the C3 platform.
This section will be expanded once the DTI team understands how this procedure will look to the researcher.
See the above link for a comprehensive list and categorization of the available training
materials. This includes C3 Documentation, DTI introductions, and DTI created examples and exercises.
No problem! You're not alone! Please send an email to help+c3ai@ncsa.illinois.edu with a description of your issue
and one of our team will work with you to resolve your issue.
If you feel aspects of this guide are incomplete or Inaccurate, please send an email to help+c3ai@ncsa.illinois.edu with the
issue or suggestion, and we will work to incorporate it to make the documentation better. We appreciate the new perspective
More eyes can bring to a software project!
Jay Roloff - Executive Director
Matthew Krafczyk - Data Analyst
Yifang Zhang - Data Analyst
Larry Rohrbach - Executive Director
Eric Fraser
Greg Merritt
Matt Podolsky