Questions? Feedback? powered by Olark live chat software
Resources Home

Rescale CLI

A command-line app for submitting jobs to Rescale

The Rescale Client App ("The App") is a Java-based client app for submitting jobs to Rescale without having to explicitly know the Rescale API. Aside from providing a simple command line utility for the Rescale API, it also integrates the multi-threaded transfer library for uploading files directly to the Rescale Data Store. This also handles any authentication related to uploading/downloading files from the Rescale Data Store. The App has several main functions:

  • Submit: submitting a job in the current working directory to Rescale
  • Status: obtaining the status of a running job on Rescale
  • Sync: syncing output files of jobs to your hard drive
  • Retrieve Info: Retrieve metadata from the Rescale API

Contents

Getting Started

Getting the Rescale CLI

First, download the CLI tool by going to Settings > API > CLI, and then clicking the Download button.

Download-CLI

Please move the downloaded file from the 'Downloads' folder into a directory in your path. For Linux and Mac OS X systems, we recommend putting the file in the /usr/local/bin/ directory. Also, make sure this directory is included in your $PATH environment variable.

Getting an API Token

An API token is required to use the API. Users can provision an API key for their account by visiting https://platform.rescale.com/user/settings/profile/ and clicking the Generate button.

Submitting a Job using the Rescale Client App

A job can be submitted using the following command (assuming the client app jar-file is named rescale.jar):

java -jar rescale.jar submit -p <api-token> -i <input-script>

The input script in this case contains the commands the user may want to run on Rescale and can be any kind of .sh script that can be executed from the command line using the ./ prefix. The script may be a long submit script or a simple one-liner script. Regardless of the type of script, the App will try to parse the script to minimize inadvertent issues related to environment setup and un-supported commands.

Please refer to the CLI FAQ page on how to simplify this command as well as the API key integration.

Example Job - Hello World!

Let's look at a very simple example. This example assumes that you have set up a wrapper script called rescale-submit (refer here).

To run a Hello World job, on the local machine open a shell terminal:

mkdir helloworld
cd helloworld
vi submit.sh

Put the following in submit.sh:

#! /bin/sh -f
#RESCALE_NAME="Hello World"
#RESCALE_CORES=1
echo "Hello World!" > hello_world
sleep 120

Create some input files:

echo "file 0" > file0
mkdir inputs
echo "file 1" > inputs/file1
echo "file 2" > inputs/file2

Submit the job:

rescale-submit submit.sh

The user can now login to the Rescale Web UI to tail and monitor the job. The job list will show the job as user: "Hello World"

VPN-image

The status page will show the progress of the job and the files can be tailed from within the UI by clicking on the file:

VPN-image

The log will look something like this:

Running SGE version of Rescale Client App
Executing Command.
Parsing Input Files
No existing files to include
Did not find explicitly specified analysis, checking implicitly selected analyses
No implicitly selected analyses, assuming user-included analysis
No core type specified by user. Using default core type 'Standard'
Zipping Files
Creating temporary encrypted zip at /Users/rescale/helloworld/input.zip
Finished writing encrypted file
Uploading Files
Uploading: /Users/rescale/helloworld/run.sh
Uploading: /Users/rescale/helloworld/input.zip
Job qMCOdb: Saved
Job qMCOdb: Submitting
Job qMCOdb: Starting polling cycle
Job qMCOdb: Status - Validated
Job qMCOdb: Status - Executing
Job qMCOdb: Status - Executing
Job qMCOdb: Status - Executing
Job qMCOdb: Status - Executing
Job qMCOdb: Status - Completed
Job qMCOdb: Finished...
Job qMCOdb: Downloading files to output/
Downloading output/submit.sh
Downloading output/run.sh
Downloading output/process_output.log
Downloading output/inputs/file2
Downloading output/inputs/file1
Downloading output/hello_world
Downloading output/file0
Finished downloading files.

Example Job - LS-DYNA

To submit a job that runs on a particular software (LS-DYNA in this case), you need to specify the software in your submit.sh file.

#!/bin/bash
#RESCALE_NAME="LS-DYNA Neon Example"
#RESCALE_ANALYSIS=ls_dyna
#RESCALE_CORE_TYPE=Onyx
#RESCALE_CORES=2
#RESCALE_WALLTIME=1
#USE_RESCALE_LICENSE
ls-dyna -n $RESCALE_CORES_PER_SLOT -i neon.refined.rev01.k -p single

Note: You can find the flags for the input script here. Also, the command on the last line is the same as you would use on the Rescale platform; you can find the command on the Rescale Web UI or using the list-info command (see here).

Assuming you have the wrapper script set up (refer here), run the job using the command:

rescale-submit submit.sh

The log will look something like this:

Running SGE version of Rescale Client App
Executing Command.
Parsing Input Files
No existing files to include
No core type specified by user. Using default core type 'Nickel'
Found Analysis: ls_dyna
Zipping Files
Creating temporary encrypted zip at /Users/rescale/lsdyna_job/input.zip
Finished writing encrypted file
Uploading Files
Uploading: /Users/rescale/lsdyna_job/run.sh
Uploading run.sh:
##############################| 176B / 176B
Uploading: /Users/rescale/lsdyna_job/input.zip
Uploading input.zip:
##############################| 9.91MB / 9.91MB
Job: Saving Job
Job npnLT: Saved
Job npnLT: Submitting
Job npnLT: Starting polling cycle
Job npnLT: Status - Validated
Job npnLT: Status - Executing
.
.
.
Job npnLT: Status - Completed
Job npnLT: Finished...
Job npnLT: Downloading files to /Users/rescale/lsdyna_job/output
.
.
.
Finished downloading files.

Overview of Running an End-to-End Job

The App will zip up all files in the current working directory into a temporary file called input.zip. It will then upload both run.sh and input.zip to Rescale and include them as input files to the job. It will also include any files listed in the #RESCALE_EXISTING_FILES line of the input script. For example, if your directory looks like this:

materials/material1
materials/material2
mesh.sim
submit.sh

And you submit your job using rescale-submit submit.sh the app will temporarily create the zip file, as well as parse the submit.sh script into run.sh. Your directory will look like this:

input.zip
materials/material1
materials/material2
mesh.sim
run.sh
submit.sh

The app will then upload the input.zip file to Rescale along with the run.sh script. The App will then remove the zip file from the local file system and attempt to save/submit the job. If the job passes validation it will be saved and submitted to Rescale. The App will then start a polling cycle and wait for the status of the job to reach completed. At this point, the App will also print out a job id which can be used for reference.

Once the job has completed, it will download all the files in the remote Rescale working directory to the local file system in a directory name output. And your working directory may look something like this:

materials/material1
materials/material2
mesh.sim
output/output1
output/output2
run.sh
submit.sh

The App will exit once all the output files have been downloaded. While the job is running the user may run a status using the job id printed out by the app. To obtain the status the user would either visit https://platform.rescale.com/jobs/<job-id>/ or from the command line run:

java -jar rescale.jar status -p <api-token> -j <job-id>

Commands and Options

Command Line Commands

The app supports the following commands:

"status" requests the specified job's status. Place in a loop to monitor continuously. It is recommended that an exponential backoff algorithm is used to monitor jobs that take longer.

java -jar /usr/local/bin/rescale.jar status [-p <api-token>] -j <job-id>

"stop" submits a request to cleanly shutdown the currently running job.

java -jar /usr/local/bin/rescale.jar stop [-p <api-token>] -j <job-id>

"sync" downloads the results files.

java -jar /usr/local/bin/rescale.jar sync [-p <api-token>] -j <job-id>

"list-info" retrieves useful metadata from the Rescale API.

Core types (-c):

java -jar /usr/local/bin/rescale.jar list-info -c [-p <api-token>]

Softwares (-a):

java -jar /usr/local/bin/rescale.jar list-info -a [-p <api-token>]

Please find a list of additional command line options/flags on the CLI FAQ page. Also, if you have set up the API key according to methods 2 or 3 in the FAQ Section, you may omit [-p <api-token>] from the above commands.

Job Configuration Using the Input Script

The Input Parser

The App uses the SGE input parser by default to parse information from the script to be used as settings on Rescale, and to parse out any unwanted cluster-specific settings in the submit script. For example, the user may have a script named submit.sh that looks like this:

#!/bin/bash
#RESCALE_NAME="Hello World!"
module load hello-world
source /home/rescale/environments/hello_world
/usr/lib64/openmpi/bin/mpirun echo "hello-world" > $(hostname)

The parser will try to make sure it parses out the cluster specific line module load... to make sure the script doesn't error on this line. It will then try to remove any commands that change the environment. In this case the lines after source will be parsed out so it doesn't interfere with the Rescale environment. Additionally, the parser will try to remove absolute references to executables/binaries as it is very likely that the executable will not reside in the same location on Rescale as on the local cluster. In this case, the reference to mpirun will be referenced through the PATH environment variable in the Rescale environment. The resulting script will then be saved to a file named run.sh and will look like this:

#!/bin/bash
#RESCALE_NAME="Hello World!"
#module load hello-world
#source /home/rescale/environments/hello_world
mpirun echo "hello-world" > $(hostname)

The Rescale analysis command will be set to ./run.sh. This script will be uploaded to Rescale along with the input files.

Rescale Specific Flags for the Input Script

The App supports the use of Rescale-specific settings flags inside the submit script to allow the user to configure their jobs exactly the way they want it. The currently available flags are listed below. If these flags are not included, the fallback value will be used first.

#RESCALE_CORES=<integer>
{1|2|4|8|16|32|64|128|...}

The number of cores to use. Depending on the core type, the allowed values is a subset of the listed options. The fallback value for this option is the SGE #$ -pe value.

#RESCALE_CORE_TYPE=<string>
{Marble|Nickel|Onyx|Iron|Gold|Obsidian|...}

The Rescale core type to use. There is no fallback value for this option. See the Metadata section in this document for instructions on how to retrieve a list of valid values for this option.

#RESCALE_NAME=<string>

The name of your job. The fallback value for this option is the SGE #$ -N value.

#RESCALE_ANALYSIS=<string>

{abaqus|adina|aermod|ansys_cfx|ansys_fluent|...|user_included|...} The analysis used. This should be set to an Analysis "code" value. The Metadata section in this document contains instructions on how to retrieve a list of valid values for this option. The fallback value is any analysis name that can be parsed and is specified in the module or source lines of the script.

#RESCALE_ANALYSIS_VERSION=<string>

The specific version of the selected analysis to use. This is an optional setting. If omitted, the latest version will be used. If specified, this should be set to the "versionCode" value for a specific analysis version. The Metadata section in this document contains instructions on how to retrieve a list of valid values for this option.

#RESCALE_EXISTING_FILES=<comma delimited list of strings>

for example: #RESCALE_EXISTING_FILES=aAbBcC,dDeEfF,gGhHiI Include the listed files already stored on Rescale in this job. If the file(s) listed does not exist, the job will fail on validation. There is no fallback value for this option.

#USE_RESCALE_LICENSE

Use the Rescale provided licenses for the analysis code. If already existing licenses are to be used, those can be set up using environment variables, using the execution script.

#RESCALE_ENV_<varname>=<value>

Used to set license server information on the Rescale cluster. Replace with the name of the environment variable name and with the license server location. For example: #RESCALE_ENV_RLM_LICENSE=8112@license-proxy.rescale.com will create a RLM_LICENSE environment variable set to 8112@license-proxy.rescale.com on the cluster nodes.

#RESCALE_WALLTIME=<# of hours>

An optional value used to set the maximum number of hours that this job will be allowed to run. Jobs that exceed this number of hours will be terminated. If not specified, the default Max Job Hours value listed in your Rescale user account settings will be used.

#RESCALE_LOW_PRIORITY=<boolean>
{true, false}

An optional value used to run your job in low priority mode. With low priority mode enabled, jobs can be terminated and restarted from the beginning at most one time. If this happens, jobs may take up to double the runtime. If not specified, the default value is false.

Advanced Settings

There are some additional advanced settings that can be used to enable SSH access into the master node of the Rescale cluster. Keep in mind that while this can be useful for examining and modifying the results of a job, it negatively impacts the reproducibility of the results should you ever clone and re-run the job in the future.

Currently, there is no way to retrieve the IP address of the master node from the CLI. You will need to use the Job status page on the Web UI to find this information.

#RESCALE_INBOUND_SSH_CIDR=<string>
for example: #RESCALE_INBOUND_SSH_CIDR=50.123.22.112/32

Used to define an IP range that will be allowed inbound SSH access to the master node of the Rescale cluster. If this value is omitted, the default value listed in your Rescale user account settings will be used.

#RESCALE_PUBLIC_KEY=<string>
for example: #RESCALE_PUBLIC_KEY=ssh-rsa AAAA...

Used to set the public key that will be added to the authorized_keys file of the master node on the Rescale cluster. This will allow you to SSH into the master node of the Rescale cluster with the associated private key. If this value is omitted, the default value listed in your Rescale user account settings will be used.

Example Scripts

An example script that will both work on SGE and Rescale could look something like this (the module line will be commented out, and the absolute reference to mpirun will be removed):

#!/bin/bash
#RESCALE_NAME="Converge Sample"
#RESCALE_CORES=32
#RESCALE_CORE_TYPE=Nickel
#RESCALE_ANALYSIS=converge_open_mpi
module load converge
/usr/bin/mpirun -np 32 converge-2.1.0

An example script that will only work on Rescale could look something like below. The command for the command line can be found in the Analysis Code Table (separate attachment).

#!/bin/bash
#RESCALE_NAME="Converge Sample"
#RESCALE_CORES=32
#RESCALE_CORE_TYPE=Nickel
#RESCALE_ANALYSIS=converge_open_mpi
converge-mpi -n all -v 2.1

Metadata

There are several Rescale-specific flags that need to be filled out when submitting a job. The following metadata retrieval commands can be used to retrieve a JSON blob containing the data to use.

The following command will retrieve a list of core types:

java -jar /usr/local/bin/rescale.jar list-info -c -p <api-token>

The value listed for the name or code can be used in the RESCALE_CORE_TYPE flag. Note that the returned list includes deprecated types which can change over time so the user should make sure that the isDeprecated value is set to false for the desired core type.

The following command will retrieve a list of analyses and their versions:

java -jar /usr/local/bin/rescale.jar list-info -a -p <api-token>

Each item in the returned list represents an analysis. The value listed for "code" can be used in the RESCALE_ANALYSIS option. Within every analysis is a list of versions. The "versionCode" field can be used in the RESCALE_ANALYSIS_VERSION option.

Note: If you have set the API key according to methods 2 or 3 in the FAQ Section, you may omit [-p <api-token>] from the above commands.

File Transfer

Syncing (Downloading)

The user may want to sync remote output files with a local file system. One of the reasons the user may want to sync output files through the app is not having to manually go through jobs and downloading the files through the browser. There is a very basic implementation of syncing files built into the app which stores sync states in a .rescale file.

The sync commands must be run from a file system with enough disk space to accommodate the downloaded files.

If the user just wants to sync a single job the user would then run:

java -jar /usr/local/bin/rescale.jar sync -p <api-token> -j <job-id>

Also, the user may want to sync all jobs newer than a specific job, the user would then use the command:

java -jar /usr/local/bin/rescale.jar sync -p <api-token> -n <job-id>

For continuous syncing, the user may specify a sync time. The time in seconds would determine the delay time between two sync attempts:

java -jar /usr/local/bin/rescale.jar sync -p <api-token> -d <sync-time>

Given the working directory, the job output files will be synced to individual directories for each job. For example, the user may have the following jobs in order of creation date with the newest first: zzzZZZ, yyyYYY, xxxXXX, cccCCC, bbbBBB, aaaAAA.

If the user runs the command

java -jar /usr/local/bin/rescale.jar sync -p <api-token> -n xxxXXX

the App will create the following directories: rescale_job_zzzZZZ, rescale_job_yyyYYY, rescale_job_xxxXXX and sync the output files inside these directories. If the syncing of zzzZZZ, and yyyYYY succeed, but the syncing of xxxXXX fails, the next time the user runs the same command the App will not try to sync zzzZZZ and yyyYYY again, but will try to sync xxxXXX again.

The sync state files are saved in the respective output file directories, so if the user tries to run the sync command from a different working directory it will re-sync the files into that directory.

Output File Filtering

Both the submit and sync commands take an optional -f flag which can be used to restrict the output files that are downloaded to the ones that match the specified file name filters. Here are a couple of usage examples:

java -jar rescale.jar submit -p <api-token> -i <input-script> -E -f *.out process_output.log

java -jar rescale.jar sync -p <api-token> -j <job-id> -f image?.png

In the submit command, the file filtering option is only applicable when running an end-to-end job.

One or more file name matchers may be specified after the -f argument. Matchers are separated by spaces. * is a wildcard that will match any string of characters and ? is a wildcard that will match any single character.

Note that all matching is done against just the filename and does not match against the full path of the output file.

Uploading

The user may want to upload one or several files from local file system to Rescale's cloud storage using the following command:

java -jar /usr/local/bin/rescale.jar upload -p <api-token> -f <file1> ... <fileN>

This command will check if all the files specified are actually files or not, and will stop if any of them is not. Note that the cli tool doesn't support uploading a directory yet.

During the uploading, after each file is successfully uploaded there will be a standard output indicating so. So the user can also redirect the output into a log file so all the successful uploads will be recorded.

java -jar /usr/local/bin/rescale.jar upload -p <api-token> -f <file1> ... <fileN> > upload_20141201.log

By default, the upload command will return a limited set of metadata for the uploaded file. To see an extended set of metadata, use the -e flag in the upload call:

java -jar /usr/local/bin/rescale.jar upload -p <api-token> -f <file1> ... <fileN> -e

Finally, if the CLI is being embedded in a larger script, then it is useful to use the --quiet flag to suppress non-json output from the command. This makes the parsing the response easier:

java -jar /usr/local/bin/rescale.jar --quiet upload -p <api-token> -f <file1> ... <fileN> -e
Last Updated: Jun 22, 2017 10:02AM PDT

Powered by Desk.com
c63bcf49f4eed122cd69ef16269a7198@rescale.desk-mail.com
https://cdn.desk.com/
false
desk
Loading
seconds ago
a minute ago
minutes ago
an hour ago
hours ago
a day ago
days ago
about
false
Invalid characters found
/customer/en/portal/articles/autocomplete