đ§Kubernetes Cluster Node
How to run Coretex worker node on a third party Kubernetes cluster managed by you for a maximum and transparent cost control, cloud-agnostic and private environments
Last updated
How to run Coretex worker node on a third party Kubernetes cluster managed by you for a maximum and transparent cost control, cloud-agnostic and private environments
Last updated
Kubernetes has been an industry standard for many years now when it comes to elastic, scalable compute resources. Many teams use it for dynamically allocating expensive short-lived containers to ML engineering team members on demand for model training. It is heavily used for inference endpoints as well, i.e. in KServe.
The beauty of Kubernetes as a container orchestration platform is it abstracts away the underlying infrastructure (computers, networking, and so on). All major cloud providers have their Kubernetes Engine: Amazon Elastic Kubernetes Service (EKS), Azure Kubernetes Service, Google Kubernetes Engine, just to name a few. You can set up your own local, on-premise, air-gapped Kubernetes cluster if you will, and the deployment process would not change at all. Without the loss of generality, in this guide we'll set up our nodes on DigitalOcean.
You have a valid Coretex account. Create one for free here.
You have installed kubectl, the CLI tool for managing Kubernetes clusters. Get it here.
(Optional) You have installed doctl, the CLI tool for managing Digital Ocean infrastructure. Get it here.
You have a valid Digital Ocean account.
The first step is to create a Kubernetes cluster control plane by logging into Digital Ocean dashboard, choosing Kubernetes on the left hand menu and clicking Create.
Most of the default cluster settings will do well for our purpose, but usually this is the moment where you would choose the balance of your infrastructure costs vs. the capability. You can choose between fixed amount of instances or use the autoscaler, setting the min and max instance count. You can also choose the instance type which suits your application the most, specifically choosing the amount of vCPUs and RAM each instance in the node pool will have.
Bear in mind this step will incur charges. Make sure to decommission your resources after use to control costs.
Once you cluster is provisioned, you should set up your local kubectl to talk to your neqwly created cluster. This is usually done by downloading the .yaml config file, but if you have doctl installed Kubernetes will give you a bash one-liner you can copy paste into your terminal, so it will take care of setting up your kubectl to talk to the cluster for you.
After successful cluster connection you should see an output like this in your console:
Now each standard kubectl command will be executed against your cluster. We're ready to deploy our Coretex worker nodes there!
Containers (known as "pods" in Kubernetes) are defined and spawned using Kubernetes Deployments, usually defined in a .yaml configuration file. This file describes various things about the service you want Kubernetes to run, such as:
required resources (vCPU, RAM)
Docker image to run
network settings and port mappings
number of replicas
...
You can download the official Coretex Node Kubernetes Deployment YAML file on this link:
https://github.com/coretex-ai/coretexpylib/raw/develop/kubernetes/node_deployment_template.yaml
The next thing would be applying the Deployment defined in node_deployment_template.yaml file you just downloaded:
But before doing that, if you inspect the deployment configuration you'll notice the Coretex Node registration protocol requires username and password encoded as base64 strings, so to streamline this process our team has provided a simple bash script to ask the user for their credentials and generate a node_deployment.yaml for you automatically, with proper base64 encoded credentials.
Download this setup script into the same folder where node_deployment_template.yaml is:
https://github.com/coretex-ai/coretexpylib/raw/develop/kubernetes/k8s_setup.sh
Running it and providing necessary input you should see something like this:
Now we are ready to apply the deployment and actually create our Coretex Nodes:
If everything went well, your nodes should be visible in Coretex Web UI, under the Nodes tab from the left-hand menu.
Assuming you have sufficient amount of hardware resources in your pool, you could change the number of replicas in the node_deployment.yaml file to parallelise your runs or inference. The only thing you need to do is change the number of replicas from 1 to 3 for instance and re-apply the deployment. There should be 3 nodes available for executing your runs or to be set up as inference endpoints, automatically load-balanced by Coretex infrastructure.
Full support for Kubernetes integration is coming to Coretex CLI. Essentially, it will wrap and automate the steps outlines in this guide to simplify cluster management for you - there will be no need to manually download any files nor run any bash commands.
Make sure to destroy resources you don't use and consult the pricing of Kubernetes service you're using. Control plane usually fixed rate charge even with no/minimum node pools.
