Datalayer from CLI

Datalayer provides a Command Line Interface (CLI) that supports the platform features.

Installation

Check you have python>=3.9 and pip and install Datalayer with the following command in your terminal.

# Use --upgrade if you want to update an existing installation to the latest version.
pip install datalayer

Environment setup with conda

If you are using conda, you can create a new environment and install the required packages as follows. Note that conda, miniconda, mamba, micromamba are interchangeable.

ENV_NAME=datalayer
miniconda create -y -n $ENV_NAME
miniconda activate $ENV_NAME
miniconda install -c conda-forge -y python=3.11 pip
pip install datalayer

Once installed you can invoke the datalayer command.

datalayer

Datalayer - Version 1.0.0 - Connected as None on https://prod1.datalayer.run

[DatalayerCLI] One of `about` `benchmarks` `envs` `runtimes` `login` `logout` `web` `who` `whoami` must be specified.

Ask details adding --help to any of the commands.

datalayer --help
The Datalayer CLI application.

Subcommands
===========
Subcommands are launched as `datalayer_core cmd [args]`. For information on
using subcommand 'cmd', do: `datalayer_core cmd -h`.

about
benchmarks
envs
runtimes
login
logout
...

tip

You can use dla or even d as shortcut, instead of datalayer if you prefer.

The about command is available to explain you the features

datalayer about

Datalayer - Version 1.0.0 - Connected as None on https://prod1.datalayer.run

About                                                                                                       

Datalayer provides a command line tool allowing to list, 
create, terminate and open a console against with Runtimes.                                                                                             

Read more on https://datalayer.io

All command lines support an option --run-url to set the URL of the Datalayer RUN cluster providing the Datalayer services.

datalayer runtimes list --run-url https://prod1.datalayer.run

Alternatively, you can set environment variables to avoid adding parameters to every command.

export DATALAYER_RUN_URL="https://prod1.datalayer.run"
export DATALAYER_TOKEN="MY_TOKEN"
export DATALAYER_EXTERNAL_TOKEN="MY_EXTERNAL_TOKEN"
datalayer whoami
datalayer envs list
datalayer runtimes list

We also made it easy for you to get the values via a User Interface (on Platform, JupyterLab...). Go to the action menu of the Kernel list and select the Copy terminal command line action. This will copy in your clipboard the command to run from CLI with the needed environment variables.

Account

To login, you can use the following command.

datalayer login

This command will open a new tab in your web browser to authenticate you.

Once authenticated, you can close your browser.

info

If your cached credentials are obsolete, you will be prompted to login when you run any of the command lines.

If you prefer not to use your web browser, you can add the --no-browser option to any command. When using the --no-browser option, you will be prompted to either:

Provide a Username and Password.
Provide a Token.

datalayer login --no-browser --run-url=...

? How do you want to login? (Use arrow keys)
 » Username/Password
   Token

Ask help to get all the login options.

datalayer login --help
An application to log into a Runtime provider.

datalayer login

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to logging.DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=10]
...

If you have a Token, you can use it to authenticate. The Token can be a Datalayer Token or an External Token in case of integration with an External IAM Provider.

Just use the --token option to provide the Token directly in any command.

datalayer envs list --token TOKEN --run-url RUN_URL

Once logged in, you can view the details of your profile.

datalayer whoami

                     Profile                     
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━┳━━━━━━━━┳━━━━━━━━┓
┃ ID                     ┃ H… ┃ First… ┃ Last … ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━╇━━━━━━━━╇━━━━━━━━┩
│ 01J2E2WQ28HS4DM0AXVEY… │ e… │ Eric   │ Charl… │
└────────────────────────┴────┴────────┴────────┘

You can logout at any time.

datalayer logout

Runtimes

Refer to the Runtimes feature documentation for more information.

To launch a Runtime, use datalayer runtimes create ENV_ID --given-name GIVEN_NAME --credits-limit CREDITS_LIMIT. Refer to the Launch a Runtime feature documentation for more information on the different options available.

datalayer runtimes create python-simple-env --given-name k1 --credits-limit 3

                                            Jupyter Runtimes                                              
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Runtime ID                         ┃ Runtime Name┃ Environment       ┃ Expired At                      ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ jupyter-01j2m0zt2kaheyy3yzn021g032 │ k1          │ python-simple-env │ 2024-08-10T17:29:19.355000+02:… │
└────────────────────────────────────┴─────────────┴───────────────────┴─────────────────────────────────┘

Use datalayer runtimes list to list all the running Runtimes.

datalayer runtimes list

                                             Jupyter Runtimes                                              
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Runtime ID                         ┃ Runtime Name┃ Environment       ┃ Expired At                      ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ jupyter-01j2m0zt2kaheyy3yzn021g032 │ k1          │ python-simple-env │ 2024-08-10T17:29:19.355000+02:… │
│ jupyter-01j2m1kc0zgq3kjcne4p5avhd8 │ k2          │ python-simple-env │ 2024-07-27T06:52:44.704000+02:… │
└────────────────────────────────────┴─────────────┴───────────────────┴─────────────────────────────────┘

To execute a notebook on a Runtime, use datalayer runtimes exec NOTEBOOK_PATH with the following command options:

--runtime - Runtime ID of the Runtime to use.
--raise - Stop the execution one an exception occurred. By default, all cells of the notebook will be executed even if one cell fails.
--timeout seconds
--verbose - Display all cell outputs. By default, only cell outputs targeting the error stream will be displayed.

datalayer runtimes exec hello.ipynb --runtime jupyter-01j2krcmskpye0sn0nv9a7afhp

To excecute a Python file on a Runtime, use datalayer runtimes exec FILE_PATH with the following command options:

--runtime - Runtime ID of the Runtime to use.
--raise - Stop the execution one an exception occurred. By default, all cells of the notebook will be executed even if one cell fails.
--timeout seconds
--verbose - Display all cell outputs. By default, only cell outputs targeting the error stream will be displayed.

cat > hello.py <<EOF
print('👋 Hello from Runtime')
EOF
datalayer runtimes exec hello.py --runtime jupyter-01j2krcmskpye0sn0nv9a7afhp
rm hello.py

To execute code in a console on a Runtime, use datalayer runtimes console --runtime RUNTIME_ID

datalayer runtimes console --runtime jupyter-01j2krcmskpye0sn0nv9a7afhp

[KernelConsoleApp] Datalayer - Version 1.1.9 - Connected as eric on https://prod1.datalayer.run
[KernelConsoleApp] KernelManager using existing kernel server jupyter-01j2krbpy5nh72xcgc8r7jwjh7 expiring at 2024-08-10T17:16:58.139000+02:00

JupyterKernels console 1.1.9

Python 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) [GCC 12.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.22.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: print('👋 Hello from Runtime')
👋 Hello from Runtime

In [2]: 

The Runtime ID is optional. If not provided, it will take the first Runtime found.

warning

To keep the Runtime alive when you exit the console, use CTRL-D or quit(keep_kernel=True).

To terminate a runtime, use datalayer runtimes terminate RUNTIME_ID. Refer to the Terminate a Runtime feature documentation for more information.

datalayer runtimes terminate jupyter-01j2ks4xzbr0vwe8zfzcv5ksm2

[RuntimesTerminateApp] Jupyter Kernels - Version 1.1.9 - Connected as eric on https://prod1.datalayer.run
[RuntimesTerminateApp] Runtime 'jupyter-01j2ks4xzbr0vwe8zfzcv5ksm2' deleted.

Environments

You can list the available environments using the following CLI command:

datalayer envs list

                                               Environments                                           
┏━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ID        ┃ Cost p… ┃ Name              ┃ Description                    ┃  ┃                Resources ┃
┡━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ python-s… │    0.01 │ Python Simple En… │ A Python environment for simp… │  │ {"cpu": "250m", "memory… │
└───────────┴─────────┴───────────────────┴────────────────────────────────┴──┴──────────────────────────┘

Refer to the Environments feature documentation for more information.

Installation​

Account​

Runtimes​

Environments​

Installation

Account

Runtimes

Environments