nip.language_model_server.server.start_vllm_server

nip.language_model_server.server.start_vllm_server#

async nip.language_model_server.server.start_vllm_server(request: VllmStartRequest) VllmStartResponse[source]#

Start the vLLM server with the specified model.

Parameters:

request (VllmStartRequest) – A request containing the model name to be served by the vLLM server.

Returns:

response (VllmStartResponse) – A data structure containing the success message, model name, and port on which the vLLM server is running.

Raises:

HTTPException – If the vLLM server fails to start.