🤩Run your first workflow

Create a project

Coretex is a project-oriented system, meaning everything you do you need to do inside of a Project. Every Dataset, Workflow, Model or Endpoint you create will be created in a scope of a Project. You can create a new Project as soon as you login to your account:

For this walkthrough we will train a computer vision model for document analysis to find relevant text in images of ID documents. You can find out more about supported project types here.

Create a dataset

The data can be uploaded to Coretex either through the web UI or through the Coretex CLI. For more information about it, visit this page.

Let's create a new Dataset either using the quick action button directly on your project dashboard or from the datasets screen which you open through the side menu on the left.

Computer Vision data samples have to be annotated before using them for training, so the neural network know which objects to look for in the image. You can do this by adding classes and then annotating the samples using the bounding box or polygon tools.

Build workflow

You can create a new Workflow from the project dashboard or from the workflow screen, and you can choose between creating from a template with pre-made tasks or an empty workflow into which you can insert new tasks. Let's create an empty workflow for now.

You define what your Workflow will do by inserting and chaining some tasks into it. You can create a task in couple of ways:

• cloning an existing one from your project

• using a task shared by a community member

• from your public Git repository

• manually uploading the archive with the code

We will insert two tasks from Coretex official Git repo, one for image augmentation (link here) and another one for object detection using Yolo V10 (link here).

Inserting image augmentation task from Coretex Git repo

Inserting object detection task from Coretex Git repo

Once our Tasks are part of the Workflow, we need to provide the missing parameters to each one of them. Coretex helps us by validating all of the parameters and highlighting the ones missing in red. Parameters can be provided either directly by value or chaining the outputs of previous tasks as inputs to the next task. This creates a chain of ordered execution which Coretex uses to plan the runs efficiently and provide outputs to the tasks when they are available.

So we need to tell our augmentation task to take our dataset as input, and then connect the output of the augmentation task as input of the object detection task.

Our workflow is ready for final review and run.

Run workflow

After reviewing the workflow, we need to select a Node to start the run. When executing your runs, you have two options when it comes to choosing which computers (called Nodes) will perform your runs: a self-managed Node or a
Coretex Cloud Node. For more details on installing a Node, visit this page.

This time we will select a self-managed Node and as you can see, the workflow is successfully started.

After the workflow is started, the augmentation task, as the first task in the workflow, starts executing, while the object detection task will be executed immediately after the first task’s execution is done. Coretex will track the graph of task dependencies and try to execute tasks as rapidly as possible assuming you have multiple nodes which can run the tasks in parallel.

Coretex will be logging machine and experiment metrics, artifacts and console output in real time while each task is being executed. You don't need to make any changes to your code to make this possible, which is a unique feature to our experiment tracking module of our platform.

Once both task runs have completed successfully we are ready for reviewing the results.

Workflow Result preview

After the task execution is complete, you can view the metrics, console, artifacts and output parameters of executed tasks in the Runs screen by double clicking each task.

Metrics

As shown in the screenshots below, you can see and compare different graphs of metrics.

Object detection task run's metrics

Comparison of the metrics of the object detection task run

Console

Console logs can be viewed by opening Task Run and clicking on the Console tab.

You can download the complete console log by clicking the Download icon.

Console preview of the object detection task run

Console preview of the augmentation task run

In the event of an error during the execution of workflow runs, Coretex AI Log Analyzer will give you a more detailed explanation of what actually happened and guide you on how you can independently fix the error that occurred.

Artifacts

Task run artifacts can be viewed by opening Task Run and clicking on the Artifacts tab.

In this particular case, the augmentation task run had no artifacts and you can only go through the artifacts of the object detection task run.

Object detection task run's artifacts

Output parameters

The output parameter of the augmentation task run is a dataset that you can find in the datasets list.

The output parameter of the object detection task run is the model, which you can find in the model list.

Conclusion

Coretex provides a smooth way for creating workflows, managing, and executing tasks, offering extensive tools for tracking and reviewing experiment metrics, logs, and artifacts. This makes it a powerful platform for developing and deploying models efficiently, especially in scenarios requiring complex task dependencies and large datasets.