nip.language_model_server.server.start_vllm_server#
- async nip.language_model_server.server.start_vllm_server(request: VllmStartRequest) VllmStartResponse[source]#
Start the vLLM server with the specified model.
- Parameters:
request (VllmStartRequest) – A request containing the model name to be served by the vLLM server.
- Returns:
response (VllmStartResponse) – A data structure containing the success message, model name, and port on which the vLLM server is running.
- Raises:
HTTPException – If the vLLM server fails to start.