Skip to content

User Guide

ScienceApps allows you to interactively run and manage ScienceCluster sessions from the browser.

Interactive Apps

To create a new interactive session, click "Interactive Apps" on the top menu bar then select which App you would like to start.

The following interactive apps are available, where you can analyze data, develop algorithms, and create models:

Beta Interactive Apps

The following apps are available in Beta, so only with very limited support:

Launching a Session

Once you have selected the App, complete the web form to create your session:

  • Version: Application version
  • Hours: The number of hours your interactive session should be available. You can always delete your interactive session at any point to stop the allocation. The maximum duration for a single session is one week (168 hours).
  • Cores: Number of vCPUs to allocate for your session.
  • RAM (system memory): Amount of memory to allocate for your session.
  • GPU: (Jupyter and VSCode only) Allows you to request a GPU either first available or of a specific type. Note: GPUs have their own memory, all of which is allocated to the session, independent of the above RAM setting.
  • Project (tenant): (Optional) In most cases, this field should be left blank. If you are a member of multiple research groups, and the cost contribution needs to be assigned to a non-default project, you would then specify the name of the Science IT project that will fund your cost contribution.
  • Partition: (Jupyter and VSCode only). Select the ScienceCluster partition that you want to use. This value is only applied when a GPU is requested. See lowprio for more info.
  • Email notifications: Check the box "Receive email on all job state changes" if you want to receive email notifications when your job starts, fails, or ends.

My Interactive Sessions

This gives an overview of currently running interactive Apps. Here you can do the following:

  • Connect to the web interface of an existing session
  • View and manage queued sessions
  • Delete running sessions to release the allocated resources

Files

You can interact with the filesystem through the web browser.

  • /home is where you store your configuration files and potentially other small files (limited in size and file number).
  • /data is where you store datasets and output files.
  • /scratch is for temporary storage of potentially big data sets. Data that is not accessed or modified in more than 30 days will be automatically removed.

A full description of the ScienceCluster filesystem is available here. Reminder: Backing up or archiving your files to protect against data loss is the responsibility of the user.

Job Interaction

Here you can view and manage your current cluster jobs (active or in the queue).

With the Job Composer you can create jobs based on templates.

Advanced Topics

Cluster Shells

Start an interactive ssh shell on the frontend node of the cluster, similar to having an interactive session like in this article. You can use this tool to create custom Jupyter kernels as described above.

Note

This will not work on Safari or Internet Explorer. Please use Chrome or Firefox.

Custom Kernels in Jupyter

If you need to install packages or want to use Jupyter with a non-python programming language, you will need to first create a Conda virtual environment. To do this, ssh to the cluster from a terminal, or open an interactive terminal under the Cluster shells section.

# Load mamba (conda) module
module load mamba

# Create and activate your environment
mamba create --name myenv
source activate myenv

# Add packages to your environment
mamba install --name myenv numpy

# Install the tools to add a custom kernel
mamba install --name myenv ipykernel

# Add your environment to the kernel list

ipython kernel install --user --name myenv

The custom package installation step can also use pip, for example:

pip install tf-models-official

Warning

Only use pip within a Conda environment when the package of interest cannot be installed using conda install. Install all packages that require pip after you've installed all packages that use conda install. Furthermore, you will need a specific version of Python in your Conda environment in order for pip install to work. For additional details on using Conda, see this how-to article.

In case you need to remove a kernel, you can use the following command from within the same environment:

jupyter kernelspec remove myenv

After running these commands, you will be able to switch to your custom kernel when you start an interactive session with Jupyter. Once the kernel is registered, it is also possible to install additional custom packages directly within Jupyter. However, it is not recommended. It's best practice to use conda install when installing packages into Conda environments.

Tensorboard

Tensorboard can be used to monitor the status of a Tensorflow model either in real-time or after a workflow has been completed.

For example, if you run the code from this page on the cluster, you'll create a logs folder within your current working directory. At that point (or at any point thereafter) you can begin a Tensorboard ScienceApps session by providing the absolute path to the logs folder via the "Log Directory" input (e.g., /scratch/$USER/logs if the logs directory is located in scratch).

Code Server

Code Server launches VS Code sessions which allows to develop directly on a compute node.

Note

If VS Code in ScienceApps does not satisfy your needs, the alternative would be to install VS Code on your machine along with all the necessary extensions, connect to a login node using remote SSH and then from the terminal of VS Code, start an interactive session to a compute node, i.e. to a GPU compute node.

FAQs

My ScienceApps homepage won't load. What should I do?

If your ScienceApps homepage won't load (e.g., if you receive a message saying "Size of a request header field exceeds server limit."), consider using an incognito/private window in your browser. If the page loads in the private mode, you should be able to resolve the problem by deleting cookies that originate from apps.s3it.uzh.ch.