MLFlow is a tool for managing the training and publishing Machine Learning Models. We have deployed an instance of the tracking server on the software-dev Radiant cluster at
https://mlflow-demo.software-dev.ncsa.illinois.edu
Access to the server is through NCSA LDAP. You must have an NCSA account and be a member of the sd_mlflow or ncsa_employe groups. You can create an NCSA account or join your existing account to the sd_mlflow LDAP group by following this link:
https://identity.ncsa.illinois.edu/join/SYHDMNC8N7
There are other MLFlow instances running at NCSA. You may receive a different link to gain access to your project's specific instance.
Once you have your account set up, you can visit the tracking server. You will be asked to authenticate using your NCSA username and password.
API Access
During training your will be using the MLFlow Python or R API - the SDK assumes that you have an environment variable set to the tracking server URL:
% export MLFLOW_TRACKING_URI=https://mlflow-demo.software-dev.ncsa.illinois.edu
You also need a JWT to access the token. The SDK picks this up from the environment variable MLFLOW_TRACKING_TOKEN. To generate this you can use an NCSA python tool called mlflow-token.
% pip install mlflow-token
When you run this command it uses the tracking server URL environment variable to start a device flow with the associated authentication provider and display you with a link to complete the login. Once successfully logged in, it returns a fresh token, formatted for setting your environment variable.
The easiest way to do this is
% export $(mlflow-token)
This token is valid for 24 hours. You can now start your training and tracking.
MLFlow in Jupyter
You can access the MLFlow tracking server from a Jupyter notebook, but the above method is a not terribly useful. Instead, you can create a cell in your notebook with the following:
import mlflow_token
mlflow_token.setup_mlflow_environment("https://mlflow-demo.software-dev.ncsa.illinois.edu")
You should put the URL for your tracking server. The library gets confused if there is a trailing slash in the URL and will throw an exception if present.
This function call will set environment variables inside the current Jupyter kernel. The token is valid for several hours, but you may need to re-execute this cell from time to time to refresh.
MLFlow on Delta
There is a Slurm backend for MLFlow which can be used to run your MLFlow projects on Delta. Please see MLFlow-Slurm for details.
Questions? Email bengal1@illinois.edu, or on NCSA slack