Links

YAML Schema Reference

YAML Schema reference for serving

Revision YAML Field Types

Message

Write a message for the Serving Revision. We recommend writing an identical message for each revision to distinguish them.
Name
Type
Required
Description
message
str
Requried
Description of the revision.
message: vessl-serve-using-yaml

Launch_immediately

Determines whether the revision will be deployed immediately.
Name
Type
Required
Description
launch_immediately
boolean
Requried
True if revision is launch immediately.
launch_immediately: true

Image

The name of the docker image that will be used for inference. You can also use a custom docker image.
Name
Type
Required
Description
image
string
Requried
Docker image url.
image: quay.io/vessl-ai/ngc-pytorch-kernel:22.10-py3-202306140422

Resources

Write down the compute resources you want to use for Serving. You can specify the resources you want to use in the Cluster settings.
Name
Type
Required
Description
cluster
string
Optional
The cluster to be used for the run. (default: VESSL-managed cluster)
name
string
Optional
The resource spec name that specified in VESSL. If the name is not specified, we will offer the best option for you based on cpu, memory, and accelerators.
cpu
string
Optional
The number of cpu cores.
memory
string
Optional
The memory size in GB.
accelerators
string
Optional
The type and quanity of the GPU to be used for the run.
spot
boolean
Optional
Whether to use spot instances for the run or not.
resources:
cluster: vessl-tmap-gi-aiml-stg
accelerators: T4:1 # using T4 with 1 GPU
spot: true
💡 You can list available clusters or resource specs with the CLI command: `vessl cluster list` or `vessl resource list`.

Volumes

Write the datasets and volumes mounted in the Revision container when the Revision is deployed.
Prefix
Type
Required
Description
git://
string
Optional
Mount a git repository into your container. The repository will be cloned into the specified mount path when container starts.
vessl-dataset://
string
Optional
Mount a dataset stored in VESSL. Replace {organizationName} with the name of your organization and {datasetName} with the name of the dataset.
s3://
string
Optional
Mount an AWS S3 bucket into your container. Replace {bucketName} with the name of your S3 bucket and {path} with the path to te file or folder you want to mount.
local://
string
Optional
Mount a file or directory from the machine where you are running the command. This can be useful for using configuration files or other data that is not in your Docker image.
hostpath://
string
Optional
Mount a file or directory from the host node’s filesystem into your container. Replace {path} with the path to the file or folder you want to mount.
nfs://
string
Optional
Mount a Network File System(NFS) into your container. Replace {ip} with the IP address of your NFS server and {path} with the path to the file or folder you want to mount.
cifs://
string
Optional
Mount a Command Internet File System(CIFS) into your contianer. Replace {ip} with the IP address of your NFS server and {path} with the path to the file or folder you want to mount.
volumes:
/root/git-examples: git://github.com/vessl-ai/examples
/input/data1: hostpath:///opt/data1
/input/config: local://config.yml
/input/data2: nfs://192.168.10.2:~/
/input/data3: vessl-dataset://{organization_name}/{dataset_name}
/output:
artifact: true
You can also add an artifact flag to indicate whether the directory /output should be treated as an output artifact. Typically, volumes store model checkpoints or key metrics.

Run

Write down what commands you want to run on the service container when the Revision is deployed.
Name
Type
Required
Description
workdir
string
Optional
The working directory for the command.
command
string
Required
The command to be run.
run:
- workdir: /root/git-examples
command: |
python train.py --learning_rate=$learning_rate --batch_size=$batch_size

Env

Write down the environment variables that will be set in the Revision Service container.
Name
Type
Required
Description
env
map
Optional
Key-value pairs for environment variables in the run container.
env:
learning_rate: 0.001
batch_size: 64
optimizer: sgd

Ports

Write down the ports and protocols that the Revision Service container should open.
Name
Type
Required
Description
name
string
Required
The name for the opening port.
type
string
Required
The protocol the port will use.
port
int
Required
The number of the port.
ports:
- name: web-service
type: http
port: 8000
- name: web-service-2
type: http
port: 8001
...

Autoscaling

Sets the value for how the Revision Pod will autoscale.
Name
Type
Required
Description
min
string
Required
Minimum number of Pods to autoscale.
max
string
Required
Maximum number of Pods to autoscale.
metric
int
Required
Determine what conditions you want to autoscale under. You can select cpu, gpu, memory, and custom
target
int
Required
A metric threshold percentage. If the metric is above the target, then the Autoscaler automatically scale-out.
autoscaling:
min: 1
max: 3
metric: cpu
target: 50

Simple YAML example for revision

message: vessl-yaml-serve-test
launch_immediately: true
image: quay.io/vessl-ai/kernels:py39
resources:
accelerators: T4:1
spot: true
volumes:
/root/examples:
git:
clone: https://github.com/vessl-ai/examples
revision: 33a49398fc6f87265ac490b1cf587912b337741a
run:
- workdir: /code/examples
command: |
python3 mnist.py
env:
- key: TEST_ENV
value: test
ports:
- name: http
type: http
port: 8000
autoscaling:
min: 1
max: 1
metric: cpu
target: 50

Gateway YAML Field Types

Enabled

Name
Type
Required
Description
enabled
boolean
Required
Whether gateway is enabled or not.
enabled: true

Targets

Name
Type
Required
Description
number
string
Required
The revision number that the Gateway will use for routing.
port
string
Required
The port number that the gateway will use for routing.
weight
int
Required
The weight to determine how much traffic should be distributed.
targets:
- number: 1
port: 8000
weight: 50
- number: 2
port: 8001
weight: 50

Sample Gateway YAML Schema

enabled: true
targets:
- number: 1
port: 8000
weight: 10
- number: 2
port: 8000
weight: 90

Serving example with YAML

MNIST model mount example

message: Example serving from YAML
image: quay.io/vessl-ai/kernels:py310-202301160626
resources:
name: cpu-m6i-large
volumes:
/root:
model:
repo: vessl-mnist-example
version: 2
run: vessl model serve vessl-mnist-example 2 --install-reqs --remote
env:
- key: VESSL_LOG
value: DEBUG
autoscaling:
min: 1
max: 3
metric: cpu
target: 60
ports:
- port: 8000
name: service
type: http