nip.language_model_server.types.VllmStartRequest

class nip.language_model_server.types.VllmStartRequest(*, model_name: str, quantization: Literal['bitsandbytes', 'none'] = 'none')

A request to start the vLLM server with a specific model.

Attributes

__fields_set__
model_computed_fields
model_config	Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
model_extra	Get extra fields set during validation.
model_fields
model_fields_set	Returns the set of fields that have been explicitly set on this model instance.
model_name	The name of the model to be served by the vLLM server.
quantization	The quantization method to use for the model.

Methods