...

See the official C3 AI documentation here.

Checking SourceFile Integration And Troubleshooting

When a .csv file is sent to C3, it is broken into chunks, and processing of the chunks proceeds in parallel. These chunks are stored in the SourceChunk Type and their status can be checked with the SourceChunkStatus Type. When the source file is being processed, we can examine the InvalidationQueue to see that the SourceQueue is processing jobs.

c3Grid(InvalidationQueue.countAll())

If processing has finished, or we want a closer look, we can inspect the chunk status like so:

c3Grid(SourceChunkStatus.fetch())

The SourceChunkStatus type has several useful fields for tracking processing progress.

state, count, successful, skipped, failed, totalErrorCount

Most useful here is the 'state' field. This field will be 'completed' if processing of the chunk is completed. Sometimes processing of chunks does not start properly. In this situation, the state will be stuck on 'initial'.

We can trigger a chunk to be reprocessed, or force a chunk to start processing as follows:

SourceChunk.forId(<chunk_id>).process(true)

Try to force your failed or frozen chunks in this way.

Complex Data Sources

Custom External Database

...