YAML Schema Reference
Write a message for the Serving Revision. We recommend writing an identical message for each revision to distinguish them.
Name | Type | Required | Description |
---|---|---|---|
message | str | Requried | Description of the revision. |
message: vessl-serve-using-yaml
Determines whether the revision will be deployed immediately.
Name | Type | Required | Description |
---|---|---|---|
launch_immediately | boolean | Requried | True if revision is launch immediately. |
launch_immediately: true
The name of the docker image that will be used for inference. You can also use a custom docker image.
Name | Type | Required | Description |
---|---|---|---|
image | string | Requried | Docker image url. |
image: quay.io/vessl-ai/ngc-pytorch-kernel:22.10-py3-202306140422
Write down the compute resources you want to use for Serving. You can specify the resources you want to use in the Cluster settings.
Name | Type | Required | Description |
---|---|---|---|
cluster | string | Optional | The cluster to be used for the run. (default: VESSL-managed cluster) |
name | string | Optional | The resource spec name that specified in VESSL. If the name is not specified, we will offer the best option for you based on cpu, memory, and accelerators. |
cpu | string | Optional | The number of cpu cores. |
memory | string | Optional | The memory size in GB. |
accelerators | string | Optional | The type and quanity of the GPU to be used for the run. |
spot | boolean | Optional | Whether to use spot instances for the run or not. |
resources:
cluster: vessl-tmap-gi-aiml-stg
accelerators: T4:1 # using T4 with 1 GPU
spot: true
💡 You can list available clusters or resource specs with the CLI command: `vessl cluster list` or `vessl resource list`.
Write the datasets and volumes mounted in the Revision container when the Revision is deployed.
Prefix | Type | Required | Description |
---|---|---|---|
git:// | string | Optional | Mount a git repository into your container. The repository will be cloned into the specified mount path when container starts. |
vessl-dataset:// | string | Optional | Mount a dataset stored in VESSL. Replace {organizationName} with the name of your organization and {datasetName} with the name of the dataset. |
s3:// | string | Optional | Mount an AWS S3 bucket into your container. Replace {bucketName} with the name of your S3 bucket and {path} with the path to te file or folder you want to mount. |
local:// | string | Optional | Mount a file or directory from the machine where you are running the command. This can be useful for using configuration files or other data that is not in your Docker image. |
hostpath:// | string | Optional | Mount a file or directory from the host node’s filesystem into your container. Replace {path} with the path to the file or folder you want to mount. |
nfs:// | string | Optional | Mount a Network File System(NFS) into your container. Replace {ip} with the IP address of your NFS server and {path} with the path to the file or folder you want to mount. |
cifs:// | string | Optional | Mount a Command Internet File System(CIFS) into your contianer. Replace {ip} with the IP address of your NFS server and {path} with the path to the file or folder you want to mount. |
volumes:
/root/git-examples: git://github.com/vessl-ai/examples
/input/data1: hostpath:///opt/data1
/input/config: local://config.yml
/input/data2: nfs://192.168.10.2:~/
/input/data3: vessl-dataset://{organization_name}/{dataset_name}
/output:
artifact: true
You can also add an artifact flag to indicate whether the directory
/output
should be treated as an output artifact. Typically, volumes store model checkpoints or key metrics.Write down what commands you want to run on the service container when the Revision is deployed.
Name | Type | Required | Description |
---|---|---|---|
workdir | string | Optional | The working directory for the command. |
command | string | Required | The command to be run. |
run:
- workdir: /root/git-examples
command: |
python train.py --learning_rate=$learning_rate --batch_size=$batch_size
Write down the environment variables that will be set in the Revision Service container.
Name | Type | Required | Description |
---|---|---|---|
env | map | Optional | Key-value pairs for environment variables in the run container. |
env:
learning_rate: 0.001
batch_size: 64
optimizer: sgd
Write down the ports and protocols that the Revision Service container should open.
Name | Type | Required | Description |
---|---|---|---|
name | string | Required | The name for the opening port. |
type | string | Required | The protocol the port will use. |
port | int | Required | The number of the port. |
ports:
- name: web-service
type: http
port: 8000
- name: web-service-2
type: http
port: 8001
...
Sets the value for how the Revision Pod will autoscale.
Name | Type | Required | Description |
---|---|---|---|
min | string | Required | Minimum number of Pods to autoscale. |
max | string | Required | Maximum number of Pods to autoscale. |
metric | int | Required | Determine what conditions you want to autoscale under. You can select cpu, gpu, memory, and custom |
target | int | Required | A metric threshold percentage. If the metric is above the target, then the Autoscaler automatically scale-out. |
autoscaling:
min: 1
max: 3
metric: cpu
target: 50
message: vessl-yaml-serve-test
launch_immediately: true
image: quay.io/vessl-ai/kernels:py39
resources:
accelerators: T4:1
spot: true
volumes:
/root/examples:
git:
clone: https://github.com/vessl-ai/examples
revision: 33a49398fc6f87265ac490b1cf587912b337741a
run:
- workdir: /code/examples
command: |
python3 mnist.py
env:
- key: TEST_ENV
value: test
ports:
- name: http
type: http
port: 8000
autoscaling:
min: 1
max: 1
metric: cpu
target: 50
Name | Type | Required | Description |
---|---|---|---|
enabled | boolean | Required | Whether gateway is enabled or not. |
enabled: true
Name | Type | Required | Description |
---|---|---|---|
number | string | Required | The revision number that the Gateway will use for routing. |
port | string | Required | The port number that the gateway will use for routing. |
weight | int | Required | The weight to determine how much traffic should be distributed. |
targets:
- number: 1
port: 8000
weight: 50
- number: 2
port: 8001
weight: 50
enabled: true
targets:
- number: 1
port: 8000
weight: 10
- number: 2
port: 8000
weight: 90
message: Example serving from YAML
image: quay.io/vessl-ai/kernels:py310-202301160626
resources:
name: cpu-m6i-large
volumes:
/root:
model:
repo: vessl-mnist-example
version: 2
run: vessl model serve vessl-mnist-example 2 --install-reqs --remote
env:
- key: VESSL_LOG
value: DEBUG
autoscaling:
min: 1
max: 3
metric: cpu
target: 60
ports:
- port: 8000
name: service
type: http
Last modified 25d ago