YAML Schema Reference

YAML Schema reference for serving
Revision YAML Field Types
Message
Write a message for the Serving Revision. We recommend writing an identical message for each revision to distinguish them.
Name
Type
Required
Description
message
str
Requried
Description of the revision.
message: vessl-serve-using-yaml
Launch_immediately
Determines whether the revision will be deployed immediately.
Name
Type
Required
Description
launch_immediately
boolean
Requried
True if revision is launch immediately.
launch_immediately: true
Image
The name of the docker image that will be used for inference. You can also use a custom docker image.
Name
Type
Required
Description
image
string
Requried
Docker image url.
image: quay.io/vessl-ai/ngc-pytorch-kernel:22.10-py3-202306140422
Resources
Write down the compute resources you want to use for Serving. You can specify the resources you want to use in the Cluster settings.
Name
Type
Required
Description
cluster
string
Optional
The cluster to be used for the run. (default: VESSL-managed cluster)
name
string
Optional
The resource spec name that specified in VESSL. If the name is not specified, we will offer the best option for you based on cpu, memory, and accelerators.
cpu
string
Optional
The number of cpu cores.
memory
string
Optional
The memory size in GB.
accelerators
string
Optional
The type and quanity of the GPU to be used for the run.
spot
boolean
Optional
Whether to use spot instances for the run or not.
resources:
  cluster: vessl-tmap-gi-aiml-stg
  accelerators: T4:1 # using T4 with 1 GPU
  spot: true
💡 You can list available clusters or resource specs with the CLI command: `vessl cluster list` or `vessl resource list`.
Volumes
Write the datasets and volumes mounted in the Revision container when the Revision is deployed.
Prefix
Type
Required
Description
git://
string
Optional
Mount a git repository into your container. The repository will be cloned into the specified mount path when container starts.
vessl-dataset://
string
Optional
Mount a dataset stored in VESSL. Replace {organizationName} with the name of your organization and {datasetName} with the name of the dataset.
s3://
string
Optional
Mount an AWS S3 bucket into your container. Replace {bucketName} with the name of your S3 bucket and {path} with the path to te file or folder you want to mount.
local://
string
Optional
Mount a file or directory from the machine where you are running the command. This can be useful for using configuration files or other data that is not in your Docker image.
hostpath://
string
Optional
Mount a file or directory from the host node’s filesystem into your container. Replace {path} with the path to the file or folder you want to mount.
nfs://
string
Optional
Mount a Network File System(NFS) into your container. Replace {ip} with the IP address of your NFS server and {path} with the path to the file or folder you want to mount.
cifs://
string
Optional
Mount a Command Internet File System(CIFS) into your contianer. Replace {ip} with the IP address of your NFS server and {path} with the path to the file or folder you want to mount.
volumes:
	/root/git-examples: git://github.com/vessl-ai/examples
	/input/data1: hostpath:///opt/data1
	/input/config: local://config.yml
	/input/data2: nfs://192.168.10.2:~/
	/input/data3: vessl-dataset://{organization_name}/{dataset_name}
	/output:
	  artifact: true
You can also add an artifact flag to indicate whether the directory /output should be treated as an output artifact. Typically, volumes store model checkpoints or key metrics.
Run
Write down what commands you want to run on the service container when the Revision is deployed.
Name
Type
Required
Description
workdir
string
Optional
The working directory for the command.
command
string
Required
The command to be run.
run:
  - workdir: /root/git-examples
    command: |
      python train.py --learning_rate=$learning_rate --batch_size=$batch_size
Env
Write down the environment variables that will be set in the Revision Service container.
Name
Type
Required
Description
env
map
Optional
Key-value pairs for environment variables in the run container.
env:
  learning_rate: 0.001
  batch_size: 64
  optimizer: sgd
Ports
Write down the ports and protocols that the Revision Service container should open.
Name
Type
Required
Description
name
string
Required
The name for the opening port.
type
string
Required
The protocol the port will use.
port
int
Required
The number of the port.
ports:
  - name: web-service
    type: http
    port: 8000
  - name: web-service-2
    type: http
    port: 8001
...
Autoscaling
Sets the value for how the Revision Pod will autoscale.
Name
Type
Required
Description
min
string
Required
Minimum number of Pods to autoscale.
max
string
Required
Maximum number of Pods to autoscale.
metric
int
Required
Determine what conditions you want to autoscale under. You can select cpu, gpu, memory, and custom
target
int
Required
A metric threshold percentage. If the metric is above the target, then the Autoscaler automatically scale-out.
autoscaling:
  min: 1
  max: 3
  metric: cpu
  target: 50
Simple YAML example for revision
message: vessl-yaml-serve-test
launch_immediately: true
​
image: quay.io/vessl-ai/kernels:py39
​
resources:
  accelerators: T4:1
  spot: true
​
volumes:
  /root/examples:
    git:
      clone: https://github.com/vessl-ai/examples
      revision: 33a49398fc6f87265ac490b1cf587912b337741a
​
run:
  - workdir: /code/examples
    command: |
      python3 mnist.py
​
env:
  - key: TEST_ENV
    value: test
​
ports:
  - name: http
    type: http
    port: 8000
​
autoscaling:
  min: 1
  max: 1
  metric: cpu
  target: 50
Gateway YAML Field Types
Enabled
Name
Type
Required
Description
enabled
boolean
Required
Whether gateway is enabled or not.
enabled: true
Targets
Name
Type
Required
Description
number
string
Required
The revision number that the Gateway will use for routing.
port
string
Required
The port number that the gateway will use for routing.
weight
int
Required
The weight to determine how much traffic should be distributed.
targets:
  - number: 1
    port: 8000
    weight: 50
  - number: 2
    port: 8001
    weight: 50
Sample Gateway YAML Schema
enabled: true
targets:
  - number: 1
    port: 8000
    weight: 10
  - number: 2
    port: 8000
    weight: 90
Serving example with YAML
MNIST model mount example
message: Example serving from YAML
image: quay.io/vessl-ai/kernels:py310-202301160626
resources:
  name: cpu-m6i-large
volumes:
  /root:
    model:
      repo: vessl-mnist-example
      version: 2
run: vessl model serve vessl-mnist-example 2 --install-reqs --remote
env:
  - key: VESSL_LOG
    value: DEBUG
autoscaling:
  min: 1
  max: 3
  metric: cpu
  target: 60
ports:
  - port: 8000
    name: service
    type: http

Name	Type	Required	Description
message	str	Requried	Description of the revision.

Name	Type	Required	Description
launch_immediately	boolean	Requried	True if revision is launch immediately.

Name	Type	Required	Description
image	string	Requried	Docker image url.

Name	Type	Required	Description
cluster	string	Optional	The cluster to be used for the run. (default: VESSL-managed cluster)
name	string	Optional	The resource spec name that specified in VESSL. If the name is not specified, we will offer the best option for you based on cpu, memory, and accelerators.
cpu	string	Optional	The number of cpu cores.
memory	string	Optional	The memory size in GB.
accelerators	string	Optional	The type and quanity of the GPU to be used for the run.
spot	boolean	Optional	Whether to use spot instances for the run or not.

Prefix	Type	Required	Description
git://	string	Optional	Mount a git repository into your container. The repository will be cloned into the specified mount path when container starts.
vessl-dataset://	string	Optional	Mount a dataset stored in VESSL. Replace {organizationName} with the name of your organization and {datasetName} with the name of the dataset.
s3://	string	Optional	Mount an AWS S3 bucket into your container. Replace {bucketName} with the name of your S3 bucket and {path} with the path to te file or folder you want to mount.
local://	string	Optional	Mount a file or directory from the machine where you are running the command. This can be useful for using configuration files or other data that is not in your Docker image.
hostpath://	string	Optional	Mount a file or directory from the host node’s filesystem into your container. Replace {path} with the path to the file or folder you want to mount.
nfs://	string	Optional	Mount a Network File System(NFS) into your container. Replace {ip} with the IP address of your NFS server and {path} with the path to the file or folder you want to mount.
cifs://	string	Optional	Mount a Command Internet File System(CIFS) into your contianer. Replace {ip} with the IP address of your NFS server and {path} with the path to the file or folder you want to mount.

Name	Type	Required	Description
workdir	string	Optional	The working directory for the command.
command	string	Required	The command to be run.

Name	Type	Required	Description
env	map	Optional	Key-value pairs for environment variables in the run container.

Name	Type	Required	Description
name	string	Required	The name for the opening port.
type	string	Required	The protocol the port will use.
port	int	Required	The number of the port.

Name	Type	Required	Description
min	string	Required	Minimum number of Pods to autoscale.
max	string	Required	Maximum number of Pods to autoscale.
metric	int	Required	Determine what conditions you want to autoscale under. You can select cpu, gpu, memory, and custom
target	int	Required	A metric threshold percentage. If the metric is above the target, then the Autoscaler automatically scale-out.

Name	Type	Required	Description
enabled	boolean	Required	Whether gateway is enabled or not.

Name	Type	Required	Description
number	string	Required	The revision number that the Gateway will use for routing.
port	string	Required	The port number that the gateway will use for routing.
weight	int	Required	The weight to determine how much traffic should be distributed.

Last modified 25d ago