nip.language_model_server.client.LanguageModelClient

class nip.language_model_server.client.LanguageModelClient(server_url: str = 'http://localhost:5000')

A client for interacting with the language model server.

This client provides methods to interact with the language model server, allowing for controlling the vLLM server and performing language model training tasks.

Parameters:

server_url (str, default=”http://localhost:5000”) – The URL of the language model server. This should include the protocol (http or https) and the port number if applicable.

Methods Summary

__init__([server_url])
cancel_training_job(job_id)	Cancel a training job by its ID.
create_training_job(training_config, dataset)	Create a new training job with the specified configuration.
get_server_version()	Get the version of the language model server.
get_training_job(job_id)	Get the details of a specific training job by its ID.
get_training_jobs()	Get the list of training jobs currently managed by the server.
get_vllm_server_status()	Get the current status of the vLLM language model server.
start_vllm_server(model_name)	Start the vLLM language model server with the specified model.
stop_vllm_server([ignore_not_running])	Stop the vLLM language model server.
wait_for_vllm_server([timeout])	Wait for the vLLM server to be online.

Methods

__init__(server_url: str = 'http://localhost:5000')

async cancel_training_job(job_id: str)

Cancel a training job by its ID.

Parameters:

job_id (str) – The ID of the training job to cancel.

Raises:

HTTPStatusError – If the server returns an error status code while cancelling the training job.

async create_training_job(training_config: LmTrainingConfig, dataset: list[DpoDatasetItem], job_id_suffix: str | None = None) → TrainingJobInfo

Create a new training job with the specified configuration.

Parameters:

• training_config (LmTrainingConfig) – The configuration for the training job, including model name and training parameters.

• dataset (list[DpoDatasetItem]) – The dataset to be used for training. This should be a list of dictionaries where each dictionary represents a single data point in the dataset.

• job_id_suffix (Optional[str], default=None) – An optional suffix to append to the job ID, to make it more recognizable.

Returns:

training_job (TrainingJobInfo) – An object containing the details of the created training job, including its ID, status, and configuration.

Raises:

• HTTPStatusError – If the server returns an error status code while creating the training job.

• BadResponseError – If the server returns an invalid response or if the response does not contain the expected data.

async get_server_version() → str

Get the version of the language model server.

Returns:

version (str) – The version of the language model server, as a string.

Raises:

BadResponseError – If the server returns an invalid response or if the response does not contain the expected ‘version’ field.

async get_training_job(job_id: str) → TrainingJobInfo

Get the details of a specific training job by its ID.

Parameters:

job_id (str) – The ID of the training job to retrieve.

Returns:

training_job (TrainingJobInfo) – An object containing the details of the training job, including its ID, status, and configuration.

Raises:

• HTTPStatusError – If the server returns an error status code while creating the training job.

• BadResponseError – If the server returns an invalid response or if the response does not contain the expected data.

async get_training_jobs() → list[TrainingJobInfo]

Get the list of training jobs currently managed by the server.

Returns:

training_jobs (list[TrainingJobInfo]) – A list of TrainingJobInfo objects, each containing information about a training job, including its ID, status, and configuration.

Raises:

• HTTPStatusError – If the server returns an error status code while creating the training job.

• BadResponseError – If the server returns an invalid response or if the response does not contain the expected data.

async get_vllm_server_status() → Literal['online', 'not_started', 'crashed', 'not_accepting_connections', 'server_error', 'other_error']

Get the current status of the vLLM language model server.

Returns:

vllm_server_status (ServerStatus) – The current status of the vLLM server. See the documentation for ServerStatus for possible values.

Raises:

BadResponseError – If the server returns an invalid response or if the response does not contain the expected ‘status’ field, or if the status is not a valid ServerStatus.

async start_vllm_server(model_name: str) → str

Start the vLLM language model server with the specified model.

Parameters:

model_name (str) – The name of the model to be served by vLLM. This should match a model that is available in the vLLM installation.

Returns:

success_message (str) – A message indicating that the vLLM server has been started successfully, or was already running.

Raises:

BadResponseError – If the server returns an invalid response or if the response does not contain the expected data.

async stop_vllm_server(ignore_not_running: bool = False)

Stop the vLLM language model server.

Parameters:

ignore_not_running (bool, default=False) – If True, the server will not raise an error if it is not running. Instead, it will log a warning and return a success message indicating that the server was not running and is being ignored.

Raises:

HTTPStatusError – If the server returns an error status code while stopping the vLLM server.

async wait_for_vllm_server(timeout: float = 300)

Wait for the vLLM server to be online.

Parameters:

timeout (float, default=300) – The maximum time to wait for the vLLM server to be online, in seconds.

Raises:

ClientTimeoutError – If the vLLM server does not become online within the specified timeout.