ℹ️ This documentation is for the VESSL MLOps product. For the new VESSL Cloud docs, see docs.cloud.vessl.ai.
YAML Reference
CLI Reference
SDK Reference
Serverless Inference API Reference
ℹ️ This documentation is for the VESSL MLOps product. For the new VESSL Cloud docs, see docs.cloud.vessl.ai.
Serverless Inference API Reference
Send a request
POST
{base_url}
/
request
/
{path}
POST /predictions/my-model
(...)
Content-Type: application/json
Content-Length: 23
{"question": "1+1 = ?"}
200 OK
(...)
Content-Type: application/json
Content-Length: 43
{"answer": "The answer is 3. No, it's 11."}
Documentation Index
Fetch the complete documentation index at: https://docs.vessl.ai/llms.txt
Use this file to discover all available pages before exploring further.
Overview
Send a request and fetch the result directly. In contrast to asynchronous APIs, this API will return the result in the same connection, and there is no JSON wrapping in either input or outputs. Thus, you can use this API as if you are directly accessing your service.When the service is in a cold state (i.e. there are no running replicas due to service
idleness) and a new request is made, a new replica will be started immediately.In such case, the first few requests may get aborted due to timeouts,
until the replica becomes up and running. Please consult your HTTP client’s timeout configuration.
Interaction code example
$ curl \
-H "Authorization: Bearer ${TOKEN}" \
"${BASE_URL}/request/predictions/my-model" \
--json '{"question": "1+1 = ?"}'
{"answer": "The answer is 3. No, it's 11."}
Request
Authorization
You must provide a token inAuthorization header with Bearer scheme, as:Authorization: Bearer <token>
Path parameters
Base URL for your service. This value can be found in the web UI (in service overview’s
Request dialog).Typical value:
https://serve-api.dev2.vssl.ai/api/v1/services/<slug>Path to use to make a request to your service.Your service should provide corresponding endpoint. Common path values used for inference include:
/v2/models/my-model/infer/predictions/my-model/v1/completions
Response
Response from your service will be relayed. Thus, there is no fixed form of response. The response will be streamed with low latency, so it can be used in live streamed applications, e.g. chatting or text completions using large language models (LLMs).HTTP response headers from your service will be generally stripped out.
Only the following headers will be passed along:
Content-TypeContent-Length
POST /predictions/my-model
(...)
Content-Type: application/json
Content-Length: 23
{"question": "1+1 = ?"}
200 OK
(...)
Content-Type: application/json
Content-Length: 43
{"answer": "The answer is 3. No, it's 11."}
Was this page helpful?
⌘I
POST /predictions/my-model
(...)
Content-Type: application/json
Content-Length: 23
{"question": "1+1 = ?"}
200 OK
(...)
Content-Type: application/json
Content-Length: 43
{"answer": "The answer is 3. No, it's 11."}