YAML Reference
Serve YAML
YAML Reference
Serve YAML
VESSL Serve is configured through a single YAML file.
Revision YAML Field Types
Name
Service name created by the user.
| Name | Type | Required | Description |
|---|---|---|---|
| name | string | Requried | Service name |
name: vessl-test-service
Message
Write a message for the Service Revision. We recommend writing an identical message for each revision to distinguish them.
| Name | Type | Required | Description |
|---|---|---|---|
| message | string | Requried | Description of the revision. |
message: vessl-yaml-revision
Image
The name of the docker image that will be used for inference. You can also use a custom docker image.
| Name | Type | Required | Description |
|---|---|---|---|
| image | string | Requried | Docker image url. |
image: quay.io/vessl-ai/ngc-pytorch-kernel:22.10-py3-202306140422
Resources
Write down the compute resources you want to use for Service. You can specify the resources you want to use in the Cluster settings.
| Name | Type | Required | Description |
|---|---|---|---|
| cluster | string | Optional | The cluster to be used for the run. (default: VESSL-managed cluster) |
| preset | string | Optional | The preset to be used for the run. |
| spot | one of (empty, SpotOnly, SpotFirst, OnDemandOnly) | Optional | Whether to use spot instances for the run or not. |
resources:
cluster: vessl-gcp-oregon
preset: gpu-l4-small-spot
Import
Write the datasets and volumes imported in the Revision container when the Revision is deployed.
| Prefix | Type | Required | Description |
|---|---|---|---|
| git:// | string | Optional | Mount a git repository into your container. The repository will be cloned into the specified mount path when container starts. |
| hf:// | string | Optional | Mount a huggingface repository into your container. The repository will be cloned into the specified mount path when container starts. |
| vessl-dataset:// | string | Optional | Mount a dataset stored in VESSL. Replace organizationName with the name of your organization and datasetName with the name of the dataset. |
| s3:// | string | Optional | Mount an AWS S3 bucket into your container. Replace bucketName with the name of your S3 bucket and path with the path to te file or folder you want to mount. |
| gs:// | string | Optional | Mount an GCP GCS bucket into your container. Replace bucketName with the name of your GCS bucket and path with the path to te file or folder you want to mount. |
| vessl-model:// | string | Optional | Mount a VESSL Model into your container. |
| vessl-artifact:// | string | Optional | Mount a VESSL Artifact into your container. |
import:
/root/git-examples: git://github.com/vessl-ai/examples
/root/hf: hf:///opt/data1
/input/config: gs://config.yaml
/input/data2: s3://192.168.10.2:~/
/input/data3: vessl-dataset://{organization_name}/{dataset_name}
Mount
Write down the volumes that will be mounted in the Revision Service container when the Revision is deployed.
| Prefix | Type | Required | Description |
|---|---|---|---|
| hf:// | string | Optional | Mount a huggingface repository into your container. The repository will be cloned into the specified mount path when container starts. |
| hostpath:// | string | Optional | Mount a file or directory from the host node’s filesystem into your container. Replace path with the path to the file or folder you want to mount. |
| nfs:// | string | Optional | Mount a Network File System(NFS) into your container. Replace ip with the IP address of your NFS server and path with the path to the file or folder you want to mount. |
| cifs:// | string | Optional | Mount a Command Internet File System(CIFS) into your contianer. Replace ip with the IP address of your NFS server and path with the path to the file or folder you want to mount. |
| gcs-fuse:// | string | Optional | Mount a GCS with FUSE csi driver. |
Run
Write down what commands you want to run on the service container when the Revision is deployed.
| Name | Type | Required | Description |
|---|---|---|---|
| workdir | string | Optional | The working directory for the command. |
| command | string | Required | The command to be run. |
run:
- workdir: /root/git-examples
command: |
vessl model launch service.py:Service -p 3000
Env
Write down the environment variables that will be set in the Revision Service container.
| Name | Type | Required | Description |
|---|---|---|---|
| env | map | Optional | Key-value pairs for environment variables in the run container. |
env:
learning_rate: 0.001
batch_size: 64
optimizer: sgd
Ports
Write down the ports and protocols that the Revision Service container should open.
| Name | Type | Required | Description |
|---|---|---|---|
| name | string | Required | The name for the opening port. |
| type | string | Required | The protocol the port will use. |
| port | int | Required | The number of the port. |
ports:
- name: web-service
type: http
port: 8000
- name: web-service-2
type: http
port: 8001
Autoscaling
Sets the value for how the Revision Pod will autoscale.
| Name | Type | Required | Description |
|---|---|---|---|
| min | string | Required | Minimum number of Pods to autoscale. |
| max | string | Required | Maximum number of Pods to autoscale. |
| metric | int | Required | Determine what conditions you want to autoscale under. You can select cpu, gpu, memory, and custom |
| target | int | Required | A metric threshold percentage. If the metric is above the target, then the Autoscaler automatically scale-out. |
autoscaling:
min: 1
max: 3
metric: cpu
target: 50