Managing queues

A queue is a named scheduling lane. It does two jobs at once: it routes work to a cluster pool (and, optionally, specific clusters within it), and it governs that work with concurrency, depth, priority, and fairness limits.

This page covers creating and managing queues administratively, from either the CLI or Python. For how workflow authors target a queue from task code, see Queues in Configure tasks.

How a queue routes

A queue lives inside one cluster pool and routes work to one or more clusters within that pool. By default (the * selector) it spreads across the pool’s healthy, enabled clusters; you can also pin it to specific clusters. It can never reach a cluster in another pool: pools are isolation boundaries.

            flowchart TD
    R(["Runs & actions"])

    subgraph Pdef["Cluster pool: default"]
        direction TB
        QD["Queue: default<br/>selector: *"]
        CA["Cluster A"]
        CB["Cluster B"]
        QD --> CA
        QD --> CB
    end

    subgraph Pprod["Cluster pool: prod"]
        direction TB
        QP["Queue: prod-queue<br/>selector: *"]
        QG["Queue: gpu-queue<br/>pinned: Cluster C"]
        CC["Cluster C"]
        CD["Cluster D"]
        QP --> CC
        QP --> CD
        QG --> CC
    end

    R --> QD
    R --> QP
    R --> QG
        

Users submit to a queue, never to a pool or a cluster directly. Each queue sits inside exactly one pool:

• default spreads across the eligible clusters in the default pool.
• prod-queue spreads across the eligible clusters in the prod pool.
• gpu-queue lives in the same prod pool but is pinned to a single cluster.

Queues you get for free

You don’t have to create a queue to have one. Two exist without any action on your part:

• The org-wide default queue, in the default pool with the * selector. Anything that doesn’t explicitly target a queue goes here. (If the default queue is drained, untargeted submissions are rejected until it is reactivated.)
• A co-named queue for every cluster: registering a cluster creates a queue with the same name as the cluster, in that cluster’s pool, whose selector names that one cluster explicitly rather than using *. Register prod-us-east-1 and you get a prod-us-east-1 queue that routes only to prod-us-east-1 — so any cluster can be targeted by name immediately, without setting up a queue for it. See The co-named queue.

Both are ordinary queues: they show up in flyte get queue and take the same settings and updates as queues you create yourself — with one exception. A co-named queue’s cluster selector and pool are managed by its cluster and cannot be edited directly: the queue follows its cluster if the cluster is reassigned to another pool, and is deleted with it. Its other settings (concurrency, depth, priority, fairness) stay editable like any queue’s. Listings make the distinction visible: cluster-managed queues are flagged in the flyte get queue table (cluster_managed, exposed as Queue.cluster_managed in Python), and flyte update queue --edit says so at the top of the edit buffer.

The selector (which clusters within the pool) is mutable and can be changed at any time. The pool a queue lives in can only change once the queue is fully drained: a pool move on an active or draining queue is rejected, because moving a queue that still holds work would cross an isolation boundary. See Move work to another pool.

Create a queue

run_concurrency and action_concurrency are required; everything else has a sensible default. With no cluster selector, a queue spreads work across all healthy clusters in its pool. The name default is reserved, and a queue cannot share a name with a cluster — every cluster already owns its co-named queue.

flyte create queue my-queue \
  --run-concurrency 100 \
  --action-concurrency 1000

flyte create queue gpu-queue \
  --cluster-pool prod \
  --cluster prod-us-east-1 \
  --run-concurrency 50 \
  --action-concurrency 500 \
  --depth 5000 \
  --priority max \
  --fairness round_robin

from flyteplugins.union.remote import Queue

queue = Queue.create(
    "my-queue",
    run_concurrency=100,
    action_concurrency=1000,
)

print(queue.to_dict())

queue = Queue.create(
    "gpu-queue",
    cluster_pool="prod",
    clusters=["prod-us-east-1"],
    run_concurrency=50,
    action_concurrency=500,
    depth=5000,
    priority="max",
    fairness="round_robin",
)

What each setting controls

• cluster_pool / --cluster-pool: the pool this queue lives in. A queue can only route to clusters in its own pool. Omit to bind the queue to the default pool.
• clusters / --cluster: pin the queue to one or more clusters in the pool. Omit to use all clusters in the pool. In the API, ["*"] means all enabled and healthy clusters in the pool (see Wildcard routing), and * must be the only entry if used.
• run_concurrency / --run-concurrency: maximum number of runs active on the queue at once. Children of an active run aren’t counted; use this to stop a job from overlapping with a previous invocation of itself. 0 means no limit.
• action_concurrency / --action-concurrency: maximum number of actions (tasks) running at once. A cap of 1 serializes the queue; higher values bound the burst rate. 0 means no limit.
• depth / --depth: total in-flight plus waiting items the queue will hold (default 10000). 0 means no limit.
• priority / --priority: min, medium (default), or max. Among queues contending for the same pool’s capacity, higher-priority work is scheduled first. Under the hood these map to enum values 1, 50, and 100; use max for a priority higher than 50. Priority controls ordering, not preemption.
• fairness / --fairness: round_robin (default) or shuffle_interleave. This controls how actions from different projects sharing the queue are interleaved.

Inspect queues

# List all queues
flyte get queue

# List only queues in a given state: active, draining, or drained
flyte get queue --state active

# Inspect one queue's settings and status
flyte get queue gpu-queue

# Stream live metrics — runs in-flight, actions in-flight, queue depth
flyte get queue gpu-queue --watch

# List soft-deleted queues
flyte get queue --deleted

from flyteplugins.union.remote import Queue

for queue in Queue.listall(limit=100):
    print(queue.name, queue.status, queue.priority, queue.cluster_pool, queue.clusters)

# Narrow the listing to one state: "active", "draining", or "drained"
for queue in Queue.listall(state="draining"):
    print(queue.name)

queue = Queue.get("gpu-queue")
print(queue.to_dict())

metrics = Queue.details("gpu-queue")
print(metrics)

for metrics in Queue.watch("gpu-queue"):
    print(metrics)

Change a queue’s settings

You can update limits, priority, fairness, or cluster pinning. The update API replaces the full queue spec; the Python wrapper handles this by reading the current queue first, changing only the fields you pass, and writing the complete spec back.

flyte update queue gpu-queue --edit

from flyteplugins.union.remote import Queue

Queue.update(
    "gpu-queue",
    run_concurrency=75,
    action_concurrency=750,
    priority="max",
    clusters=["prod-us-east-1"],
)

Changing the cluster selector within the same pool (which clusters the queue pins to) takes effect immediately because every cluster in the pool shares the same data plane.

Changing the queue’s pool (cluster_pool in the YAML or Python) is allowed only when the queue is drained, the destination pool exists, and every cluster in the queue’s selector is a member of the destination pool — see Move work to another pool. A cluster’s co-named queue rejects selector and pool changes entirely: those are managed by its cluster.

Drain and reactivate a queue

Draining takes a queue out of rotation without losing in-flight work: the queue stops admitting new submissions, work already in flight runs to completion, and once nothing is left the queue settles into the drained state. Draining is how you quiesce a queue: before deleting the cluster behind it, before maintenance, or as part of moving work to another pool.

A queue is in one of three states:

active --[drain]--> draining --[in-flight work completes]--> drained
  ^                    |                                        |
  +----[activate]------+----------------[activate]-------------+

flyte update queue gpu-queue --drain      # stop new submissions; let in-flight work finish
flyte update queue gpu-queue --activate   # put the queue back in rotation

from flyteplugins.union.remote import Queue

Queue.drain("gpu-queue")     # stop new submissions; let in-flight work finish
Queue.activate("gpu-queue")  # put the queue back in rotation

To see where things stand across the organization, filter listings by state: flyte get queue --state draining shows the queues still finishing in-flight work, and --state drained the ones that are done.

Draining is the prerequisite for every disruptive queue operation: a queue must be drained before it can be deleted or moved to another pool, and a cluster’s co-named queue must be drained before its cluster can be deleted or moved.

The default queue can be drained like any other — since it cannot be deleted, draining is how you stop scheduling on it. One guard applies to any queue: a queue configured as the default run queue (run.default_queue) in your organization’s settings cannot be drained until that setting points elsewhere.

Delete a queue

A queue must be drained before it can be deleted; deleting an active or draining queue is rejected. The default queue cannot be deleted — drain it instead. A cluster’s co-named queue can be deleted on its own while the cluster lives, and is deleted automatically when its cluster is deleted.

flyte delete queue gpu-queue
flyte delete queue gpu-queue --yes   # skip the confirmation prompt

# List deleted queues, restore one
flyte get queue --deleted
flyte undelete queue gpu-queue

from flyteplugins.union.remote import Queue

Queue.delete("gpu-queue")
Queue.undelete("gpu-queue")   # restore a deleted queue

Deletion is a soft delete: the queue disappears from flyte get queue, but its name stays reserved and it can be restored with flyte undelete queue <name>. Undeleting requires the queue’s pool and its pinned clusters to still be live, and the restored queue comes back drained — reactivate it to resume routing.

Move work to another pool

Moving work to a different pool crosses an isolation boundary. In-flight runs have already landed their data, containers, code, and secrets in the old pool’s data plane, and a different pool’s clusters cannot read them. So work in flight never moves: the migration is always drain-first, in one of two shapes.

Move the queue itself. A queue can change pools once it holds no work:

1. Drain the queue and wait for it to reach the drained state.
2. Update the queue’s pool — flyte update queue <name> --edit and change cluster_pool in the YAML, or Queue.update(name, cluster_pool=...) — and set its cluster selector to clusters that are members of the destination pool (or *).
3. Reactivate the queue.

Anything targeting the queue by name follows it to the new pool. New submissions made while it is drained are rejected, so coordinate the window with the queue’s users.

Replace the queue. Alternatively, keep the old queue’s pool binding and shift traffic over:

1. Create a new queue in the destination pool.
2. Update workflows, launch plans, triggers, or run overrides to target the new queue.
3. Drain the old queue to shut out straggler submissions and let in-flight work finish.
4. Delete the drained queue, or leave it idle — an idle queue costs nothing.

See also

• Queues in Configure tasks: routing work to a queue from task code, triggers, and per-run context.
• Cluster pools and Clusters: the routing targets a queue points at.