Datalayer from CLI
Datalayer provides a Command Line Interface (CLI) that supports the platform features.
Installation
Check you have python>=3.9 and pip and install Datalayer with the following command in your terminal.
# Use --upgrade if you want to update an existing installation to the latest version.
pip install datalayer-core
Environment setup with conda
If you are using conda, you can create a new environment and install the required packages as follows. Note that conda, miniconda, mamba, micromamba are interchangeable.
ENV_NAME=datalayer
miniconda create -y -n $ENV_NAME
miniconda activate $ENV_NAME
miniconda install -c conda-forge -y python=3.11 pip
pip install datalayer-core
Once installed you can invoke the datalayer command.
datalayer
Datalayer - Version 1.0.0 - Connected as None on https://prod1.datalayer.run
[DatalayerCLI] One of `about` `benchmarks` `envs` `runtimes` `login` `logout` `web` `who` `whoami` must be specified.
Ask details adding --help to any of the commands.
datalayer --help
The Datalayer CLI application.
Subcommands
===========
Subcommands are launched as `datalayer_core cmd [args]`. For information on
using subcommand 'cmd', do: `datalayer_core cmd -h`.
about
benchmarks
envs
runtimes
login
logout
...
You can use dla or even d as shortcut, instead of datalayer if you prefer.
The about command is available to explain you the features
datalayer about
Datalayer - Version 1.0.0 - Connected as None on https://prod1.datalayer.run
About
Datalayer provides a command line tool allowing to list,
create, terminate and open a console against with Runtimes.
Read more on https://datalayer.io
All command lines support an option --run-url to set the URL of the Datalayer RUN cluster providing the Datalayer services.
datalayer runtimes list --run-url https://prod1.datalayer.run
Alternatively, you can set environment variables to avoid adding parameters to every command.
export DATALAYER_RUN_URL="https://prod1.datalayer.run"
export DATALAYER_API_KEY="MY_API_KEY"
export DATALAYER_EXTERNAL_TOKEN="MY_EXTERNAL_TOKEN"
datalayer whoami
datalayer envs list
datalayer runtimes list
We also made it easy for you to get the values via a User Interface (on the Platform, in JupyterLab...). Go to the action menu of the Kernel list and select the Copy terminal command line action. This will copy in your clipboard the command to run from CLI with the needed environment variables.
Account
To login, you can use the following command.
datalayer login
This command will open a new tab in your web browser to authenticate you.
Once authenticated, you can close your browser.
If your cached credentials are obsolete, you will be prompted to login when you run any of the command lines.
If you prefer not to use your web browser, you can add the --no-browser option to any command. When using the --no-browser option, you will be prompted to either:
- Provide a Username and Password.
- Provide a Token.
datalayer login --no-browser --run-url=...
? How do you want to login? (Use arrow keys)
» Username/Password
Token
Ask help to get all the login options.
datalayer login --help
An application to log into a Runtime provider.
datalayer login
Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
<cmd> --help-all
--debug
set log level to logging.DEBUG (maximize logging output)
Equivalent to: [--Application.log_level=10]
...
If you have a Token, you can use it to authenticate. The Token can be a Datalayer Token or an External Token in case of integration with an External IAM Provider.
Just use the --token option to provide the Token directly in any command.
datalayer envs list --token TOKEN --run-url RUN_URL
Once logged in, you can view the details of your profile.
datalayer whoami
Profile
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━┳━━━━━━━━┳━━━━━━━━┓
┃ ID ┃ H… ┃ First… ┃ Last … ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━╇━━━━━━━━╇━━━━━━━━┩
│ 01J2E2WQ28HS4DM0AXVEY… │ e… │ Eric │ Charl… │
└────────────────────────┴────┴────────┴────────┘
You can logout at any time.
datalayer logout
Runtimes
Refer to the Runtimes feature documentation for more information.
To launch a Runtime, use datalayer runtimes create ENV_ID --given-name GIVEN_NAME --credits-limit CREDITS_LIMIT.
Refer to the Launch a Runtime feature documentation for more information on the different options available.
datalayer runtimes create python-cpu-env --given-name r1 --credits-limit 3
Jupyter Runtimes
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Runtime ID ┃ Runtime Name┃ Environment ┃ Expired At ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ runtime-01j2m0zt2kaheyy3yzn021g032 │ r1 │ python-cpu-env │ 2024-08-10T17:29:19.355000+02:… │
└────────────────────────────────────┴─────────────┴───────────────────┴─────────────────────────────────┘
Use datalayer runtimes list to list all the running Runtimes.
datalayer runtimes list
Jupyter Runtimes
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Runtime ID ┃ Runtime Name┃ Environment ┃ Expired At ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ runtime-01j2m0zt2kaheyy3yzn021g032 │ r1 │ python-cpu-env │ 2024-08-10T17:29:19.355000+02:… │
│ runtime-01j2m1kc0zgq3kjcne4p5avhd8 │ r2 │ python-cpu-env │ 2024-07-27T06:52:44.704000+02:… │
└────────────────────────────────────┴─────────────┴───────────────────┴─────────────────────────────────┘
To execute code in a console on a Runtime, use datalayer runtimes console --runtime RUNTIME_ID
datalayer console --runtime runtime-01j2krcmskpye0sn0nv9a7afhp
[RuntimesConsoleApp] RuntimeManager using existing runtime runtime-01k6ba06dgfrztbvfbjy8yzd1c expiring at 2025-09-30T08:07:38+02:00
Jupyter Konsole 0.8.0
Python 3.12.9 | packaged by conda-forge | (main, Mar 4 2025, 22:48:41) [GCC 13.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.34.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: print('👋 Hello from console Runtime')
👋 Hello from console Runtime
In [2]:
The Runtime ID is optional. If not provided, it will take the first Runtime found.
To keep the Runtime alive when you exit the console, use CTRL-D or quit(keep_kernel=True).
To execute a Python file on a Runtime, use datalayer exec FILE_PATH with the following command options:
--runtime- Runtime ID of the Runtime to use.--raise- Stop the execution one an exception occurred. By default, all cells of the notebook will be executed even if one cell fails.--timeout seconds--verbose- Display all cell outputs. By default, only cell outputs targeting the error stream will be displayed.
cat > hello.py <<EOF
print('👋 Hello from Python Runtime')
EOF
datalayer exec hello.py
rm hello.py
To execute a notebook on a Runtime, use datalayer exec NOTEBOOK_PATH with the following command options:
--runtime- Runtime ID of the Runtime to use.--raise- Stop the execution one an exception occurred. By default, all cells of the notebook will be executed even if one cell fails.--timeout seconds--verbose- Display all cell outputs. By default, only cell outputs targeting the error stream will be displayed.
cat > hello.ipynb <<EOF
{
"cells": [
{
"id": "cell-1",
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print('👋 Hello from Notebook Runtime')"
]
}
],
"metadata": {
},
"nbformat": 4,
"nbformat_minor": 4
}
EOF
datalayer exec hello.ipynb
rm hello.ipynb
To terminate a runtime, use datalayer runtimes terminate RUNTIME_ID. Refer to the Terminate a Runtime feature documentation for more information.
datalayer runtimes terminate runtime-01j2ks4xzbr0vwe8zfzcv5ksm2
[RuntimesTerminateApp] Runtime 'runtime-01j2ks4xzbr0vwe8zfzcv5ksm2' deleted.
Environments
You can list the available environments using the following CLI command:
datalayer envs list
Environments
┏━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ID ┃ Cost p… ┃ Name ┃ Description ┃ ┃ Resources ┃
┡━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ python-s… │ 0.01 │ Python Simple En… │ A Python environment for simp… │ │ {"cpu": "250m", "memory… │
└───────────┴─────────┴───────────────────┴────────────────────────────────┴──┴──────────────────────────┘
Refer to the Environments feature documentation for more information.
Secrets
You can manage secrets using the following CLI command.
datalayer secrets --help
The Runtimes CLI application.
Subcommands
===========
Subcommands are launched as `datalayer_core cmd [args]`. For information on
using subcommand 'cmd', do: `datalayer_core cmd -h`.
create
list
ls
delete
You can create a secret using the datalayer secrets create command. The command requires a name, description and value for the secret.
datalayer secrets create MY_SECRET "My secret description" --value "my-secret-value"
To list all secrets, use the datalayer secrets list command.
datalayer secrets list
Secrets
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┓
┃ ID ┃ Name ┃ Description ┃ Variant ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━┩
│ 01J....................... │ AWS_ACCESS_KEY_ID │ AWS_ACCESS_KEY_ID │ generic │
│ 01J....................... │ AWS_SECRET_ACCESS_KEY │ AWS_SECRET_ACCESS_KEY │ generic │
│ 01J....................... │ AWS_DEFAULT_REGION │ AWS_DEFAULT_REGION │ generic │
└────────────────────────────┴───────────────────────┴───────────────────────┴─────────┘