yellowdog_client.scheduler – Scheduler

All functionality is accessed via:

class yellowdog_client.scheduler.PredicatedWorkSubscriptionEventListener(future, predicate, work_client)
Parameters:
subscription_cancelled()

Invoked if the server cancels the subscription associated with this listener

Return type:

None

subscription_error(error)

Invoked if an error occurs on the subscription associated with this listener.

Parameters:

error (Exception) – the error that occurred

Return type:

None

updated(obj)

Invoked when an updated instance of the object is received from YellowDog Platform.

Parameters:

obj (WorkRequirement) – an updated instance of the object for which the listener was registered

Return type:

None

class yellowdog_client.scheduler.PredicatedWorkerPoolSubscriptionEventListener(future, predicate, worker_pool_client)
Parameters:
subscription_cancelled()

Invoked if the server cancels the subscription associated with this listener

Return type:

None

subscription_error(error)

Invoked if an error occurs on the subscription associated with this listener.

Parameters:

error (Exception) – the error that occurred

Return type:

None

updated(obj)

Invoked when an updated instance of the object is received from YellowDog Platform.

Parameters:

obj (WorkerPool) – an updated instance of the object for which the listener was registered

Return type:

None

class yellowdog_client.scheduler.WorkClient

Client interface containing methods for accessing YellowDog Scheduler functions.

abstract add_tasks_to_task_group(task_group, tasks)

Submits NEW tasks to the YellowDog Scheduler service to be added to the specified task group.

Parameters:

  • task_group (TaskGroup) – the task group to add the tasks to

  • tasks (List[Task]) – the submitted tasks

Return type:

List[Task]

Returns:

the latest state of the tasks after they have been submitted

abstract add_tasks_to_task_group_by_id(task_group_id, tasks)

Submits NEW tasks to the YellowDog Scheduler service to be added to the specified task group.

Parameters:

  • task_group_id (str) – the ID of the task group to add the tasks to

  • tasks (List[Task]) – the submitted tasks

Return type:

List[Task]

Returns:

the latest state of the tasks after they have been submitted

abstract add_tasks_to_task_group_by_name(namespace, work_requirement_name, task_group_name, tasks)

Submits NEW tasks to the YellowDog Scheduler service to be added to the specified task group.

Parameters:

  • namespace (str) – the namespace

  • work_requirement_name (str) – the work requirement name

  • task_group_name (str) – the task group name

  • tasks (List[Task]) – the submitted tasks

Return type:

List[Task]

Returns:

the latest state of the tasks after they have been submitted

abstract add_work_requirement(work_requirement)

Submits a NEW work requirement to the YellowDog Scheduler service to be initialised and started.

Parameters:

work_requirement (WorkRequirement) – the work requirement to submit

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after it has been submitted

abstract add_work_requirement_listener(work_requirement, listener)

Adds an event listener to receive notifications of changes for the specified work requirement. The client manages subscriptions to YellowDog Scheduler such that the first listener created for a requirement will cause a Server-Sent Events subscription to be initiated; additional listeners for the same requirement share that subscription.

Parameters:

  • work_requirement (WorkRequirement) – the work requirement for which to receive notifications

  • listener (SubscriptionEventListener[WorkRequirement]) – the event listener that will be invoked for notifications

Return type:

None

abstract add_work_requirement_listener_by_id(work_requirement_id, listener)

Adds an event listener to receive notifications of changes for the specified work requirement. The client manages subscriptions to YellowDog Scheduler such that the first listener created for a requirement will cause a Server-Sent Events subscription to be initiated; additional listeners for the same requirement share that subscription.

Parameters:

  • work_requirement_id (str) – the ID of the work requirement for which to receive notifications

  • listener (SubscriptionEventListener[WorkRequirement]) – the event listener that will be invoked for notifications

Return type:

None

abstract cancel_task(task, abort)

Instructs the Scheduler to cancel the specified task.

Parameters:

  • task (Task) – the task to cancel

  • abort (bool) – if the task should be aborted if it has been allocated to a worker

Return type:

Task

Returns:

the latest state of the task after the cancel instruction was submitted

abstract cancel_task_by_id(task_id, abort)

Instructs the Scheduler to cancel the specified task.

Parameters:

  • task_id (str) – the ID of the task to cancel

  • abort (bool) – if the task should be aborted if it has been allocated to a worker

Return type:

Task

Returns:

the latest state of the task after the cancel instruction was submitted

abstract cancel_work_requirement(work_requirement, abort=None)

Instructs the Scheduler to cancel the supplied work requirement, no further tasks will be executed and all workers shall be released.

Parameters:

  • work_requirement (WorkRequirement) – the work requirement to cancel

  • abort (Optional[bool]) – if tasks should be aborted if they have been allocated to a worker

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the cancel instruction was submitted

abstract cancel_work_requirement_by_id(work_requirement_id, abort=None)

Instructs the Scheduler to cancel the supplied work requirement, no further tasks will be executed and all workers shall be released.

Parameters:

  • work_requirement_id (str) – the ID of the work requirement to cancel

  • abort (Optional[bool]) – if tasks should be aborted if they have been allocated to a worker

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the cancel instruction was submitted

abstract find_all_work_requirements()

Returns summaries of all existing work requirements within the system for the requesting user.

Return type:

List[WorkRequirementSummary]

Returns:

a list of work requirement summaries

@deprecated use #getWorkRequirements(WorkRequirementSearch) instead to search tasks.

abstract find_tasks(search)

Returns tasks within the system that match the specified search. WARNING: If your search matches too many tasks to fit into your application’s memory limits, consider using #streamTasks(TaskSearch) or #findTasks(TaskSearch, SliceReference).

Parameters:

search (TaskSearch) – the search

Return type:

List[Task]

Returns:

a list of tasks

@deprecated use #getTasks(TaskSearch) instead to search tasks.

abstract find_tasks_slice(search, slice_reference)

Returns a slice of tasks that match the specified search and slice reference.

Parameters:
Return type:

Slice[Task]

Returns:

a slice of tasks

@deprecated use #getTasks(TaskSearch) instead to search tasks.

abstract finish_work_requirement(work_requirement)

Instructs the Scheduler not to accept further tasks for the supplied work requirement and to finish it after all tasks are finished.

Parameters:

work_requirement (WorkRequirement) – the work requirement to finish

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the finish instruction was submitted

abstract finish_work_requirement_by_id(work_requirement_id)

Instructs the Scheduler not to accept further tasks for the supplied work requirement and to finish it after all tasks are finished.

Parameters:

work_requirement_id (str) – the ID of the work requirement to finish

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the finish instruction was submitted

abstract get_task(task)

Gets the latest state of the supplied task.

Parameters:

task (Task) – the task for which to get the latest state

Return type:

Task

Returns:

the latest state of the task

abstract get_task_by_id(task_id)

Gets the latest state of the supplied task.

Parameters:

task_id (str) – the ID of the task

Return type:

Task

Returns:

the latest state of the task

abstract get_tasks(search)

Returns a SearchClient that offers the ability to search tasks.

Parameters:

search (TaskSearch) – the search

Return type:

SearchClient[Task]

Returns:

the search client

abstract get_work_requirement(work_requirement)

Gets the latest state of the supplied work requirement.

Parameters:

work_requirement (WorkRequirement) – the work requirement for which to get the latest state

Return type:

WorkRequirement

Returns:

the latest state of the work requirement

abstract get_work_requirement_by_id(work_requirement_id)

Gets the latest state of the supplied work requirement.

Parameters:

work_requirement_id (str) – the ID of the work requirement

Return type:

WorkRequirement

Returns:

the latest state of the work requirement

abstract get_work_requirement_by_name(namespace, work_requirement_name)

Gets the latest state of the supplied work requirement.

Parameters:

  • namespace (str) – the namespace containing the requirement

  • work_requirement_name (str) – the name of the requirement

Return type:

WorkRequirement

Returns:

the latest state of the work requirement

abstract get_work_requirement_helper(work_requirement)

Constructs a new work requirement helper for the specified requirement.

Parameters:

work_requirement (WorkRequirement) – the work requirement for which the helper will be constructed

Return type:

WorkRequirementHelper

Returns:

a new work requirement helper

abstract get_work_requirement_helper_by_id(work_requirement_id)

Constructs a new work requirement helper for the specified requirement.

Parameters:

work_requirement_id (str) – the ID of the work requirement for which the helper will be constructed

Return type:

WorkRequirementHelper

Returns:

a new work requirement helper

abstract get_work_requirements(search)

Returns a SearchClient that offers the ability to search work requirements.

Parameters:

search (WorkRequirementSearch) – the search

Return type:

SearchClient[WorkRequirementSummary]

Returns:

the search client

abstract hold_work_requirement(work_requirement)

Instructs the Scheduler to hold the supplied work requirement, no further tasks will be executed until the work requirement is started again.

Parameters:

work_requirement (WorkRequirement) – the work requirement to hold

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the hold instruction was submitted

abstract hold_work_requirement_by_id(work_requirement_id)

Instructs the Scheduler to hold the supplied work requirement, no further tasks will be executed until the work requirement is started again.

Parameters:

work_requirement_id (str) – the ID of the work requirement to hold

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the hold instruction was submitted

abstract remove_work_requirement_listener(listener)

Removes the specified event listener. The client manages subscriptions to YellowDog Scheduler such that when the last listener for a requirement is removed, the associated Server-Sent Events subscription is cancelled.

Parameters:

listener (SubscriptionEventListener[WorkRequirement]) – the event listener that will no longer be invoked for notifications

Return type:

None

abstract start_work_requirement(work_requirement)

Instructs the Scheduler to start the supplied work requirement after it was held.

Parameters:

work_requirement (WorkRequirement) – the work requirement to start after being held

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the start instruction was submitted

abstract start_work_requirement_by_id(work_requirement_id)

Instructs the Scheduler to start the supplied work requirement after it was held.

Parameters:

work_requirement_id (str) – the ID of the work requirement to start after being held

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the start instruction was submitted

abstract update_work_requirement(work_requirement)

Submits an existing work requirement to the YellowDog Scheduler service to be updated.

Parameters:

work_requirement (WorkRequirement) – the work requirement to submit

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after it has been submitted

class yellowdog_client.scheduler.WorkClientImpl(service_proxy)
Parameters:

service_proxy (WorkServiceProxy)

add_tasks_to_task_group(task_group, tasks)

Submits NEW tasks to the YellowDog Scheduler service to be added to the specified task group.

Parameters:

  • task_group (TaskGroup) – the task group to add the tasks to

  • tasks (List[Task]) – the submitted tasks

Return type:

List[Task]

Returns:

the latest state of the tasks after they have been submitted

add_tasks_to_task_group_by_id(task_group_id, tasks)

Submits NEW tasks to the YellowDog Scheduler service to be added to the specified task group.

Parameters:

  • task_group_id (str) – the ID of the task group to add the tasks to

  • tasks (List[Task]) – the submitted tasks

Return type:

List[Task]

Returns:

the latest state of the tasks after they have been submitted

add_tasks_to_task_group_by_name(namespace, requirement_name, task_group_name, tasks)

Submits NEW tasks to the YellowDog Scheduler service to be added to the specified task group.

Parameters:

  • namespace (str) – the namespace

  • work_requirement_name – the work requirement name

  • task_group_name (str) – the task group name

  • tasks (List[Task]) – the submitted tasks

  • requirement_name (str)

Return type:

List[Task]

Returns:

the latest state of the tasks after they have been submitted

add_work_requirement(work_requirement)

Submits a NEW work requirement to the YellowDog Scheduler service to be initialised and started.

Parameters:

work_requirement (WorkRequirement) – the work requirement to submit

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after it has been submitted

add_work_requirement_listener(work_requirement, listener)

Adds an event listener to receive notifications of changes for the specified work requirement. The client manages subscriptions to YellowDog Scheduler such that the first listener created for a requirement will cause a Server-Sent Events subscription to be initiated; additional listeners for the same requirement share that subscription.

Parameters:

  • work_requirement (WorkRequirement) – the work requirement for which to receive notifications

  • listener (SubscriptionEventListener[WorkRequirement]) – the event listener that will be invoked for notifications

Return type:

None

add_work_requirement_listener_by_id(work_requirement_id, listener)

Adds an event listener to receive notifications of changes for the specified work requirement. The client manages subscriptions to YellowDog Scheduler such that the first listener created for a requirement will cause a Server-Sent Events subscription to be initiated; additional listeners for the same requirement share that subscription.

Parameters:

  • work_requirement_id (str) – the ID of the work requirement for which to receive notifications

  • listener (SubscriptionEventListener[WorkRequirement]) – the event listener that will be invoked for notifications

Return type:

None

cancel_task(task, abort)

Instructs the Scheduler to cancel the specified task.

Parameters:

  • task (Task) – the task to cancel

  • abort (bool) – if the task should be aborted if it has been allocated to a worker

Return type:

Task

Returns:

the latest state of the task after the cancel instruction was submitted

cancel_task_by_id(task_id, abort)

Instructs the Scheduler to cancel the specified task.

Parameters:

  • task_id (str) – the ID of the task to cancel

  • abort (bool) – if the task should be aborted if it has been allocated to a worker

Return type:

Task

Returns:

the latest state of the task after the cancel instruction was submitted

cancel_work_requirement(work_requirement, abort=False)

Instructs the Scheduler to cancel the supplied work requirement, no further tasks will be executed and all workers shall be released.

Parameters:

  • work_requirement (WorkRequirement) – the work requirement to cancel

  • abort (bool) – if tasks should be aborted if they have been allocated to a worker

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the cancel instruction was submitted

cancel_work_requirement_by_id(work_requirement_id, abort=False)

Instructs the Scheduler to cancel the supplied work requirement, no further tasks will be executed and all workers shall be released.

Parameters:

  • work_requirement_id (str) – the ID of the work requirement to cancel

  • abort (bool) – if tasks should be aborted if they have been allocated to a worker

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the cancel instruction was submitted

close()
Return type:

None

find_all_work_requirements()

Returns summaries of all existing work requirements within the system for the requesting user.

Return type:

List[WorkRequirementSummary]

Returns:

a list of work requirement summaries

@deprecated use #getWorkRequirements(WorkRequirementSearch) instead to search tasks.

find_tasks(search)

Returns tasks within the system that match the specified search. WARNING: If your search matches too many tasks to fit into your application’s memory limits, consider using #streamTasks(TaskSearch) or #findTasks(TaskSearch, SliceReference).

Parameters:

search (TaskSearch) – the search

Return type:

List[Task]

Returns:

a list of tasks

@deprecated use #getTasks(TaskSearch) instead to search tasks.

find_tasks_slice(search, slice_reference)

Returns a slice of tasks that match the specified search and slice reference.

Parameters:
Return type:

Slice

Returns:

a slice of tasks

@deprecated use #getTasks(TaskSearch) instead to search tasks.

finish_work_requirement(work_requirement)

Instructs the Scheduler not to accept further tasks for the supplied work requirement and to finish it after all tasks are finished.

Parameters:

work_requirement (WorkRequirement) – the work requirement to finish

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the finish instruction was submitted

finish_work_requirement_by_id(work_requirement_id)

Instructs the Scheduler not to accept further tasks for the supplied work requirement and to finish it after all tasks are finished.

Parameters:

work_requirement_id (str) – the ID of the work requirement to finish

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the finish instruction was submitted

get_task(task)

Gets the latest state of the supplied task.

Parameters:

task (Task) – the task for which to get the latest state

Return type:

Task

Returns:

the latest state of the task

get_task_by_id(task_id)

Gets the latest state of the supplied task.

Parameters:

task_id (str) – the ID of the task

Return type:

Task

Returns:

the latest state of the task

get_tasks(search)

Returns a SearchClient that offers the ability to search tasks.

Parameters:

search (TaskSearch) – the search

Return type:

SearchClient[Task]

Returns:

the search client

get_work_requirement(work_requirement)

Gets the latest state of the supplied work requirement.

Parameters:

work_requirement (WorkRequirement) – the work requirement for which to get the latest state

Return type:

WorkRequirement

Returns:

the latest state of the work requirement

get_work_requirement_by_id(work_requirement_id)

Gets the latest state of the supplied work requirement.

Parameters:

work_requirement_id (str) – the ID of the work requirement

Return type:

WorkRequirement

Returns:

the latest state of the work requirement

get_work_requirement_by_name(namespace, work_requirement_name)

Gets the latest state of the supplied work requirement.

Parameters:

  • namespace (str) – the namespace containing the requirement

  • work_requirement_name (str) – the name of the requirement

Return type:

WorkRequirement

Returns:

the latest state of the work requirement

get_work_requirement_helper(work_requirement)

Constructs a new work requirement helper for the specified requirement.

Parameters:

work_requirement (WorkRequirement) – the work requirement for which the helper will be constructed

Return type:

WorkRequirementHelper

Returns:

a new work requirement helper

get_work_requirement_helper_by_id(work_requirement_id)

Constructs a new work requirement helper for the specified requirement.

Parameters:

work_requirement_id (str) – the ID of the work requirement for which the helper will be constructed

Return type:

WorkRequirementHelper

Returns:

a new work requirement helper

get_work_requirements(search)

Returns a SearchClient that offers the ability to search work requirements.

Parameters:

search (WorkRequirementSearch) – the search

Return type:

SearchClient[WorkRequirementSummary]

Returns:

the search client

hold_work_requirement(work_requirement)

Instructs the Scheduler to hold the supplied work requirement, no further tasks will be executed until the work requirement is started again.

Parameters:

work_requirement (WorkRequirement) – the work requirement to hold

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the hold instruction was submitted

hold_work_requirement_by_id(work_requirement_id)

Instructs the Scheduler to hold the supplied work requirement, no further tasks will be executed until the work requirement is started again.

Parameters:

work_requirement_id (str) – the ID of the work requirement to hold

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the hold instruction was submitted

remove_work_requirement_listener(listener)

Removes the specified event listener. The client manages subscriptions to YellowDog Scheduler such that when the last listener for a requirement is removed, the associated Server-Sent Events subscription is cancelled.

Parameters:

listener (SubscriptionEventListener[WorkRequirement]) – the event listener that will no longer be invoked for notifications

Return type:

None

start_work_requirement(work_requirement)

Instructs the Scheduler to start the supplied work requirement after it was held.

Parameters:

work_requirement (WorkRequirement) – the work requirement to start after being held

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the start instruction was submitted

start_work_requirement_by_id(work_requirement_id)

Instructs the Scheduler to start the supplied work requirement after it was held.

Parameters:

work_requirement_id (str) – the ID of the work requirement to start after being held

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after the start instruction was submitted

update_work_requirement(work_requirement)

Submits an existing work requirement to the YellowDog Scheduler service to be updated.

Parameters:

work_requirement (WorkRequirement) – the work requirement to submit

Return type:

WorkRequirement

Returns:

the latest state of the work requirement after it has been submitted

class yellowdog_client.scheduler.WorkRequirementHelper(work_requirement, work_service_client_impl)

This class provides a number of methods that return a concurrent.futures.Future allowing consumers to simply wait for the required state of a work requirement before continuing on.

Constructor accepts the following arguments:

Parameters:

See also

Use yellowdog_client.scheduler.WorkClientImpl.get_work_requirement_helper() for easier access to the yellowdog_client.scheduler.WorkRequirementHelper

helper = client.work_client.get_work_requirement_helper(work_requirement)
# WorkRequirementHelper

Added in version 0.4.0.

when_requirement_matches(predicate)

Returns a concurrent.futures.Future that is completed when the specified predicate evaluates to true.

Parameters:

predicate (Callable[[yellowdog_client.model.WorkRequirement], bool]) – The predicate to test for each work requirement changed event received.

Returns:

A concurrent.futures.Future containing the matching work requirement.

Return type:

concurrent.futures.Future

from concurrent import futures
from yellowdog_client.model import WorkRequirementStatus

future = helper.when_requirement_matches(lambda req:  req.status == WorkRequirementStatus.COMPLETED)

futures.wait(fs=(future,))
future_work_requirement = future.result()
# WorkRequirement
when_requirement_status_is(status)

Returns a task that is completed when the work requirement status matches the specified status.

Parameters:

status (yellowdog_client.model.WorkRequirementStatus) – The work requirement status to wait for.

Returns:

A concurrent.futures.Future containing the matching work requirement.

Return type:

concurrent.futures.Future

from concurrent import futures
from yellowdog_client.model import WorkRequirementStatus

future = helper.when_requirement_status_is(WorkRequirementStatus.COMPLETED)

futures.wait(fs=(future,))
future_work_requirement = future.result()
# WorkRequirement
class yellowdog_client.scheduler.WorkServiceProxy(proxy)
Parameters:

proxy (Proxy)

add_tasks_to_task_group(task_group_id, tasks)
Parameters:
Return type:

List[Task]

add_tasks_to_task_group_by_name(namespace, requirement_name, task_group_name, tasks)
Parameters:
Return type:

List[Task]

add_work_requirement(requirement)
Parameters:

requirement (WorkRequirement)

Return type:

WorkRequirement

cancel_task(task_id, abort)
Parameters:
Return type:

Task

find_work_requirements(search, slice_reference)
Parameters:
Return type:

Slice[WorkRequirementSummary]

get_task_by_id(task_id)
Parameters:

task_id (str)

Return type:

Task

get_work_requirement_by_id(work_requirement_id)
Parameters:

work_requirement_id (str)

Return type:

WorkRequirement

get_work_requirement_by_name(namespace, work_requirement_name)
Parameters:

  • namespace (str)

  • work_requirement_name (str)

Return type:

WorkRequirement

search_tasks(search, slice_reference)
Parameters:
Return type:

Slice[Task]

stream_work_requirement_updates(requirement_id)
Parameters:

requirement_id (str)

transition_work_requirement(requirement_id, next_status, abort_if_cancelling)
Parameters:
Return type:

WorkRequirement

update_work_requirement(requirement)
Parameters:

requirement (WorkRequirement)

Return type:

WorkRequirement

class yellowdog_client.scheduler.WorkerPoolClient

Client interface containing methods for accessing YellowDog Scheduler functions.

abstract add_configured_worker_pool(request)

Submits a NEW configured worker pool to the YellowDog Scheduler service to be initialised.

Parameters:

request (AddConfiguredWorkerPoolRequest) – the request containing details of the worker pool

Return type:

AddConfiguredWorkerPoolResponse

Returns:

the response containing the worker pool and the token

abstract add_node_actions(worker_pool, *actions)

Adds the specified node actions to be performed for the specified worker pool.

Parameters:
Return type:

None

abstract add_node_actions_by_id(worker_pool_id, *actions)

Adds the specified node actions to be performed for the specified worker pool.

Parameters:

  • worker_pool_id (str) – the ID of the worker pool

  • actions (NodeAction) – the node actions

Return type:

None

abstract add_node_actions_for_node(node, *actions)

Adds node actions to be performed for the specified node. Sets the nodeIdFilter and nodeTypes properties of any actions to apply to the specified node.

Parameters:
Return type:

None

abstract add_node_actions_for_node_by_id(worker_pool_id, node_id, *actions)

Adds node actions to be performed for the specified node. Sets the nodeIdFilter and nodeTypes properties of any actions to apply to the specified node.

Parameters:

  • worker_pool_id (str) – the ID of the worker pool containing the node

  • node_id (str) – the ID of the node

  • actions (NodeAction) – the node actions

Return type:

None

abstract add_node_actions_grouped(worker_pool, action_groups=None, node_id_filter_list=None)

Adds the specified groups of node actions to be performed for the specified worker pool.

Parameters:
Return type:

None

abstract add_node_actions_grouped_by_id(worker_pool_id, action_groups=None, node_id_filter_list=None)

Adds the specified groups of node actions to be performed for the specified worker pool.

Parameters:

  • worker_pool_id (str) – the ID of the worker pool

  • action_groups (Optional[List[NodeActionGroup]]) – the node action groups

  • node_id_filter_list (Optional[List[str]]) – a list of node IDs that can be used to filter recipients of node actions

Return type:

None

abstract add_worker_pool_listener(worker_pool, listener)

Adds an event listener to receive notifications of changes for the specified worker pool. The client manages subscriptions to YellowDog Scheduler such that the first listener created for a worker pool will cause a Server-Sent Events subscription to be initiated; additional listeners for the same worker pool share that subscription.

Parameters:

  • worker_pool (WorkerPool) – the worker pool for which to receive notifications

  • listener (SubscriptionEventListener[WorkerPool]) – the event listener that will be invoked for notifications

Return type:

None

abstract add_worker_pool_listener_by_id(worker_pool_id, listener)

Adds an event listener to receive notifications of changes for the specified worker pool. The client manages subscriptions to YellowDog Scheduler such that the first listener created for a worker pool will cause a Server-Sent Events subscription to be initiated; additional listeners for the same worker pool share that subscription.

Parameters:

  • worker_pool_id (str) – the ID of the worker pool for which to receive notifications

  • listener (SubscriptionEventListener[WorkerPool]) – the event listener that will be invoked for notifications

Return type:

None

abstract find_all_worker_pools()

Returns summaries of all existing worker pools within the system for the requesting user.

Return type:

List[WorkerPoolSummary]

Returns:

a list of worker pool summaries

@deprecated use #getWorkerPools(WorkerPoolSearch) instead to search worker pools.

abstract find_nodes(search)

Returns worker pool nodes within the system that match the specified search. WARNING: If your search matches too many workers to fit into your application’s memory limits, consider using #streamNodes(NodeSearch) or #findNodes(NodeSearch, SliceReference).

Parameters:

search (NodeSearch) – the search

Return type:

List[Node]

Returns:

a list of nodes

@deprecated use #getNodes(NodeSearch) instead to search nodes.

abstract find_nodes_slice(search, slice_reference)

Returns a slice of nodes that match the specified search and slice reference.

Parameters:
Return type:

Slice[Node]

Returns:

a slice of nodes

@deprecated use #getNodes(NodeSearch) instead to search nodes.

abstract get_configured_worker_pool_token(worker_pool)

Get a configured worker pool token.

Parameters:

worker_pool (ConfiguredWorkerPool) – the worker pool

Return type:

WorkerPoolToken

Returns:

the worker pool token

abstract get_configured_worker_pool_token_by_id(worker_pool_id)

Get a configured worker pool token.

Parameters:

worker_pool_id (str) – the worker pool ID

Return type:

WorkerPoolToken

Returns:

the worker pool token

abstract get_node(node)

Gets the latest state of the node with the specified ID.

Parameters:

node (Node) – the node

Return type:

Node

Returns:

the latest state of the node

abstract get_node_actions(node)

Gets the current state of the specified node’s action queue.

Parameters:

node (Node) – the node

Return type:

NodeActionQueueSnapshot

Returns:

the current state of the action queue

abstract get_node_actions_by_id(node_id)

Gets the current state of the specified node’s action queue.

Parameters:

node_id (str) – the ID of the node

Return type:

NodeActionQueueSnapshot

Returns:

the current state of the action queue

abstract get_node_by_id(node_id)

Gets the latest state of the node with the specified ID.

Parameters:

node_id (str) – the ID of the node

Return type:

Node

Returns:

the latest state of the node

abstract get_node_by_worker_id(worker_id)

Gets the latest state of the node with a worker with the specified ID.

Parameters:

worker_id (str) – the ID of the worker

Return type:

Node

Returns:

the latest state of the node

abstract get_nodes(search)

Returns a SearchClient that offers the ability to search nodes.

Parameters:

search (NodeSearch) – the search

Return type:

SearchClient[Node]

Returns:

the search client

abstract get_worker_pool(worker_pool)

Gets the latest state of the supplied worker pool.

Parameters:

worker_pool (~T) – the worker pool for which to get the latest state

Return type:

~T

Returns:

the latest state of the worker pool

abstract get_worker_pool_by_id(worker_pool_id)

Gets the latest state of the worker pool with the specified ID.

Parameters:

worker_pool_id (str) – the ID of the worker pool

Return type:

WorkerPool

Returns:

the latest state of the worker pool

abstract get_worker_pool_by_name(namespace, name)

Gets the latest state of the worker pool with the specified namespace and name.

Parameters:

  • namespace (str) – the namespace of the worker pool

  • name (str) – the name of the worker pool

Return type:

WorkerPool

Returns:

the latest state of the worker pool

abstract get_worker_pool_helper(worker_pool)

Constructs a new worker pool helper for the specified worker pool.

Parameters:

worker_pool (WorkerPool) – the worker pool for which the helper will be constructed

Return type:

WorkerPoolHelper

Returns:

a new worker pool helper

abstract get_worker_pool_helper_by_id(worker_pool_id)

Constructs a new worker pool helper for the specified worker pool.

Parameters:

worker_pool_id (str) – the ID of the worker pool for which the helper will be constructed

Return type:

WorkerPoolHelper

Returns:

a new worker pool helper

abstract get_worker_pools(search)

Returns a SearchClient that offers the ability to search worker pools.

Parameters:

search (WorkerPoolSearch) – the search

Return type:

SearchClient[WorkerPoolSummary]

Returns:

the search client

abstract provision_worker_pool(compute_requirement_template_usage, provisioned_properties=None)

Requests that the supplied compute requirement template request is used to provision a worker pool.

Parameters:
Return type:

ProvisionedWorkerPool

Returns:

the provisioned worker pool

abstract refresh_configured_worker_pool_token(worker_pool, token_ttl=None)

Refresh a configured worker pool token with the specified time-to-live.

Parameters:
Return type:

WorkerPoolToken

Returns:

the worker pool token

abstract refresh_configured_worker_pool_token_by_id(worker_pool_id, token_ttl=None)

Refresh a configured worker pool token with the specified time-to-live.

Parameters:

  • worker_pool_id (str) – the worker pool ID

  • token_ttl (Optional[timedelta]) – the token time-to-live.

Return type:

WorkerPoolToken

Returns:

the worker pool token

abstract regenerate_configured_worker_pool_token(worker_pool, token_ttl=None)

Regenerate a configured worker pool token with the specified time-to-live.

Parameters:
Return type:

WorkerPoolToken

Returns:

the worker pool token

abstract regenerate_configured_worker_pool_token_by_id(worker_pool_id, token_ttl=None)

Regenerate a configured worker pool token with the specified time-to-live.

Parameters:

  • worker_pool_id (str) – the worker pool ID

  • token_ttl (Optional[timedelta]) – the token time-to-live.

Return type:

WorkerPoolToken

Returns:

the worker pool token

abstract remove_worker_pool_listener(listener)

Removes the specified event listener. The client manages subscriptions to YellowDog Scheduler such that when the last listener for a worker pool is removed, the associated Server-Sent Events subscription is cancelled.

Parameters:

listener (SubscriptionEventListener[WorkerPool]) – the event listener that will no longer be invoked for notifications

Return type:

None

abstract resize_worker_pool(worker_pool, size)

Resizes the specified worker pool to the given number of nodes.

Parameters:

  • worker_pool (ProvisionedWorkerPool) – the worker pool to resize

  • size (int) – the target number of nodes when resizing the worker pool

Return type:

ProvisionedWorkerPool

Returns:

the latest state of the worker pool

abstract resize_worker_pool_by_id(worker_pool_id, size)

Resizes the specified worker pool to the given number of nodes.

Parameters:

  • worker_pool_id (str) – the ID of the worker pool to resize

  • size (int) – the target number of nodes when resizing the worker pool

Return type:

ProvisionedWorkerPool

Returns:

the latest state of the worker pool

abstract shutdown_node(node)

Shuts down the specified node gracefully, allowing any running tasks to finish first.

Parameters:

node (Node) – the node

Return type:

Node

Returns:

the latest state of the node

abstract shutdown_node_by_id(node_id)

Shuts down the specified node gracefully, allowing any running tasks to finish first.

Parameters:

node_id (str) – the ID of the node

Return type:

Node

Returns:

the latest state of the node

abstract shutdown_worker_pool(worker_pool)

Shuts down the specified worker pool.

Parameters:

worker_pool (WorkerPool) – the worker pool to shutdown

Return type:

None

abstract shutdown_worker_pool_by_id(worker_pool_id)

Shuts down the specified worker pool.

Parameters:

worker_pool_id (str) – the ID of the worker pool to shutdown

Return type:

None

class yellowdog_client.scheduler.WorkerPoolClientImpl(service_proxy)
Parameters:

service_proxy (WorkerPoolServiceProxy)

add_configured_worker_pool(request)

Submits a NEW configured worker pool to the YellowDog Scheduler service to be initialised.

Parameters:

request (AddConfiguredWorkerPoolRequest) – the request containing details of the worker pool

Return type:

AddConfiguredWorkerPoolResponse

Returns:

the response containing the worker pool and the token

add_node_actions(worker_pool, *actions)

Adds the specified node actions to be performed for the specified worker pool.

Parameters:
Return type:

None

add_node_actions_by_id(worker_pool_id, *actions)

Adds the specified node actions to be performed for the specified worker pool.

Parameters:

  • worker_pool_id (str) – the ID of the worker pool

  • actions (NodeAction) – the node actions

Return type:

None

add_node_actions_for_node(node, *actions)

Adds node actions to be performed for the specified node. Sets the nodeIdFilter and nodeTypes properties of any actions to apply to the specified node.

Parameters:
Return type:

None

add_node_actions_for_node_by_id(worker_pool_id, node_id, *actions)

Adds node actions to be performed for the specified node. Sets the nodeIdFilter and nodeTypes properties of any actions to apply to the specified node.

Parameters:

  • worker_pool_id (str) – the ID of the worker pool containing the node

  • node_id (str) – the ID of the node

  • actions (NodeAction) – the node actions

Return type:

None

add_node_actions_grouped(worker_pool, action_groups=None, node_id_filter_list=None)

Adds the specified groups of node actions to be performed for the specified worker pool.

Parameters:
Return type:

None

add_node_actions_grouped_by_id(worker_pool_id, action_groups=None, node_id_filter_list=None)

Adds the specified groups of node actions to be performed for the specified worker pool.

Parameters:

  • worker_pool_id (str) – the ID of the worker pool

  • action_groups (Optional[List[NodeActionGroup]]) – the node action groups

  • node_id_filter_list (Optional[List[str]]) – a list of node IDs that can be used to filter recipients of node actions

Return type:

None

add_worker_pool_listener(worker_pool, listener)

Adds an event listener to receive notifications of changes for the specified worker pool. The client manages subscriptions to YellowDog Scheduler such that the first listener created for a worker pool will cause a Server-Sent Events subscription to be initiated; additional listeners for the same worker pool share that subscription.

Parameters:

  • worker_pool (WorkerPool) – the worker pool for which to receive notifications

  • listener (SubscriptionEventListener[WorkerPool]) – the event listener that will be invoked for notifications

Return type:

None

add_worker_pool_listener_by_id(worker_pool_id, listener)

Adds an event listener to receive notifications of changes for the specified worker pool. The client manages subscriptions to YellowDog Scheduler such that the first listener created for a worker pool will cause a Server-Sent Events subscription to be initiated; additional listeners for the same worker pool share that subscription.

Parameters:

  • worker_pool_id (str) – the ID of the worker pool for which to receive notifications

  • listener (SubscriptionEventListener[WorkerPool]) – the event listener that will be invoked for notifications

Return type:

None

close()
Return type:

None

find_all_worker_pools()

Returns summaries of all existing worker pools within the system for the requesting user.

Return type:

List[WorkerPoolSummary]

Returns:

a list of worker pool summaries

@deprecated use #getWorkerPools(WorkerPoolSearch) instead to search worker pools.

find_nodes(search)

Returns worker pool nodes within the system that match the specified search. WARNING: If your search matches too many workers to fit into your application’s memory limits, consider using #streamNodes(NodeSearch) or #findNodes(NodeSearch, SliceReference).

Parameters:

search (NodeSearch) – the search

Return type:

List[Node]

Returns:

a list of nodes

@deprecated use #getNodes(NodeSearch) instead to search nodes.

find_nodes_slice(search, slice_reference)

Returns a slice of nodes that match the specified search and slice reference.

Parameters:
Return type:

Slice[Node]

Returns:

a slice of nodes

@deprecated use #getNodes(NodeSearch) instead to search nodes.

get_configured_worker_pool_token(worker_pool)

Get a configured worker pool token.

Parameters:

worker_pool (ConfiguredWorkerPool) – the worker pool

Return type:

WorkerPoolToken

Returns:

the worker pool token

get_configured_worker_pool_token_by_id(worker_pool_id)

Get a configured worker pool token.

Parameters:

worker_pool_id (str) – the worker pool ID

Return type:

WorkerPoolToken

Returns:

the worker pool token

get_node(node)

Gets the latest state of the node with the specified ID.

Parameters:

node (Node) – the node

Return type:

Node

Returns:

the latest state of the node

get_node_actions(node)

Gets the current state of the specified node’s action queue.

Parameters:

node (Node) – the node

Return type:

NodeActionQueueSnapshot

Returns:

the current state of the action queue

get_node_actions_by_id(node_id)

Gets the current state of the specified node’s action queue.

Parameters:

node_id (str) – the ID of the node

Return type:

NodeActionQueueSnapshot

Returns:

the current state of the action queue

get_node_by_id(node_id)

Gets the latest state of the node with the specified ID.

Parameters:

node_id (str) – the ID of the node

Return type:

Node

Returns:

the latest state of the node

get_node_by_worker_id(worker_id)

Gets the latest state of the node with a worker with the specified ID.

Parameters:

worker_id (str) – the ID of the worker

Return type:

Node

Returns:

the latest state of the node

get_nodes(search)

Returns a SearchClient that offers the ability to search nodes.

Parameters:

search (NodeSearch) – the search

Return type:

SearchClient[Node]

Returns:

the search client

get_worker_pool(worker_pool)

Gets the latest state of the supplied worker pool.

Parameters:

worker_pool (~T) – the worker pool for which to get the latest state

Return type:

~T

Returns:

the latest state of the worker pool

get_worker_pool_by_id(worker_pool_id)

Gets the latest state of the worker pool with the specified ID.

Parameters:

worker_pool_id (str) – the ID of the worker pool

Return type:

WorkerPool

Returns:

the latest state of the worker pool

get_worker_pool_by_name(namespace, name)

Gets the latest state of the worker pool with the specified namespace and name.

Parameters:

  • namespace (str) – the namespace of the worker pool

  • name (str) – the name of the worker pool

Return type:

WorkerPool

Returns:

the latest state of the worker pool

get_worker_pool_helper(worker_pool)

Constructs a new worker pool helper for the specified worker pool.

Parameters:

worker_pool (WorkerPool) – the worker pool for which the helper will be constructed

Return type:

WorkerPoolHelper

Returns:

a new worker pool helper

get_worker_pool_helper_by_id(worker_pool_id)

Constructs a new worker pool helper for the specified worker pool.

Parameters:

worker_pool_id (str) – the ID of the worker pool for which the helper will be constructed

Return type:

WorkerPoolHelper

Returns:

a new worker pool helper

get_worker_pools(search)

Returns a SearchClient that offers the ability to search worker pools.

Parameters:

search (WorkerPoolSearch) – the search

Return type:

SearchClient[WorkerPoolSummary]

Returns:

the search client

provision_worker_pool(requirement_template_usage, provisioned_properties=None)

Requests that the supplied compute requirement template request is used to provision a worker pool.

Parameters:
Return type:

ProvisionedWorkerPool

Returns:

the provisioned worker pool

refresh_configured_worker_pool_token(worker_pool, token_ttl=None)

Refresh a configured worker pool token with the specified time-to-live.

Parameters:
Return type:

WorkerPoolToken

Returns:

the worker pool token

refresh_configured_worker_pool_token_by_id(worker_pool_id, token_ttl=None)

Refresh a configured worker pool token with the specified time-to-live.

Parameters:

  • worker_pool_id (str) – the worker pool ID

  • token_ttl (Optional[timedelta]) – the token time-to-live.

Return type:

WorkerPoolToken

Returns:

the worker pool token

regenerate_configured_worker_pool_token(worker_pool, token_ttl=None)

Regenerate a configured worker pool token with the specified time-to-live.

Parameters:
Return type:

WorkerPoolToken

Returns:

the worker pool token

regenerate_configured_worker_pool_token_by_id(worker_pool_id, token_ttl=None)

Regenerate a configured worker pool token with the specified time-to-live.

Parameters:

  • worker_pool_id (str) – the worker pool ID

  • token_ttl (Optional[timedelta]) – the token time-to-live.

Return type:

WorkerPoolToken

Returns:

the worker pool token

remove_worker_pool_listener(listener)

Removes the specified event listener. The client manages subscriptions to YellowDog Scheduler such that when the last listener for a worker pool is removed, the associated Server-Sent Events subscription is cancelled.

Parameters:

listener (SubscriptionEventListener[WorkerPool]) – the event listener that will no longer be invoked for notifications

Return type:

None

resize_worker_pool(worker_pool, size)

Resizes the specified worker pool to the given number of nodes.

Parameters:

  • worker_pool (ProvisionedWorkerPool) – the worker pool to resize

  • size (int) – the target number of nodes when resizing the worker pool

Return type:

ProvisionedWorkerPool

Returns:

the latest state of the worker pool

resize_worker_pool_by_id(worker_pool_id, size)

Resizes the specified worker pool to the given number of nodes.

Parameters:

  • worker_pool_id (str) – the ID of the worker pool to resize

  • size (int) – the target number of nodes when resizing the worker pool

Return type:

ProvisionedWorkerPool

Returns:

the latest state of the worker pool

shutdown_node(node)

Shuts down the specified node gracefully, allowing any running tasks to finish first.

Parameters:

node (Node) – the node

Return type:

Node

Returns:

the latest state of the node

shutdown_node_by_id(node_id)

Shuts down the specified node gracefully, allowing any running tasks to finish first.

Parameters:

node_id (str) – the ID of the node

Return type:

Node

Returns:

the latest state of the node

shutdown_worker_pool(worker_pool)

Shuts down the specified worker pool.

Parameters:

worker_pool (WorkerPool) – the worker pool to shutdown

Return type:

None

shutdown_worker_pool_by_id(worker_pool_id)

Shuts down the specified worker pool.

Parameters:

worker_pool_id (str) – the ID of the worker pool to shutdown

Return type:

None

class yellowdog_client.scheduler.WorkerPoolHelper(worker_pool, worker_pool_client)
Parameters:
when_worker_pool_matches(predicate)
Parameters:

predicate (Callable[[WorkerPool], bool])

Return type:

Future

when_worker_pool_status_is(status)
Parameters:

status (WorkerPoolStatus)

Return type:

Future

class yellowdog_client.scheduler.WorkerPoolServiceProxy(proxy)
Parameters:

proxy (Proxy)

add_configured_worker_pool(request)
Parameters:

request (AddConfiguredWorkerPoolRequest)

Return type:

AddConfiguredWorkerPoolResponse

add_node_actions_grouped(worker_pool_id, action_groups, node_id_filter_list=None)
Parameters:
Return type:

None

find_worker_pools(search, slice_reference)
Parameters:
Return type:

Slice[WorkerPoolSummary]

get_configured_worker_pool_token_by_id(worker_pool_id)
Parameters:

worker_pool_id (str)

Return type:

WorkerPoolToken

get_node_actions(node_id)
Parameters:

node_id (str)

Return type:

NodeActionQueueSnapshot

get_node_by_id(node_id)
Parameters:

node_id (str)

Return type:

Node

get_node_by_worker_id(worker_id)
Parameters:

worker_id (str)

Return type:

Node

get_worker_pool_by_id(worker_pool_id)
Parameters:

worker_pool_id (str)

Return type:

WorkerPool

get_worker_pool_by_name(namespace, name)
Parameters:
Return type:

WorkerPool

provision_worker_pool(template_usage, worker_pool_properties)
Parameters:
Return type:

ProvisionedWorkerPool

refresh_configured_worker_pool_token(worker_pool_id, token_ttl=None)
Parameters:
Return type:

WorkerPoolToken

regenerate_configured_worker_pool_token(worker_pool_id, token_ttl=None)
Parameters:
Return type:

WorkerPoolToken

resize_worker_pool(worker_pool_id, size)
Parameters:

  • worker_pool_id (str)

  • size (int)

Return type:

ProvisionedWorkerPool

search_nodes(search, slice_reference)
Parameters:
Return type:

Slice[Node]

shutdown_node(node_id)
Parameters:

node_id (str)

Return type:

Node

shutdown_worker_pool(worker_pool_id)
Parameters:

worker_pool_id (str)

Return type:

None

stream_worker_pool_updates(worker_pool_id)
Parameters:

worker_pool_id (str)

Return type:

EventSource