dstack - Browse /0.20.29 at SourceForge.net

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
0.20.29 source code.tar.gz	2026-07-24	20.8 MB	0
0.20.29 source code.zip	2026-07-24	21.8 MB	0
README.md	2026-07-24	6.7 kB	0
Totals: 3 Items		42.6 MB	0

Presets

This release introduces dstack preset, an agent-driven inference optimization toolkit. A headless agent runs sequential trials on your fleets, benchmarks serving configurations, and saves the best one as a reusable preset.

To create a preset, define a preset configuration and pass it to dstack preset create:

:::yaml
type: preset
name: qwen3-coder

# The agent picks a compatible variant of the base model
base: Qwen/Qwen3-Coder-30B-A3B-Instruct

# Consider only the specified fleets
fleets: [h200-fleet]

env:

  - HF_TOKEN

# The number of benchmarked trials
max_trials: 5

By default, the agent may choose the hardware among the fleet's offers, the serving framework and its flags, and, when base is specified, any compatible variant of the model, including a different precision or quantization. The prompt property can change the objectives and extend what the agent is allowed to do, from custom target metrics to experiments it would not attempt on its own.

:::shell
$ dstack preset create -f preset.dstack.yml
...
Create the preset qwen3-coder? [y/n]: y
[2026-07-23 09:11:40] Submitted trial task qwen3-coder-4c9a17d5-1 (vllm/vllm-openai:latest, H100:80GB). Trial 1: bf16, defaults, max-model-len 32768, establishing the baseline.
[2026-07-23 09:46:02] Trial 1 complete. Baseline: 780 tok/s, TTFT 610ms (concurrency 8, 4096 in / 512 out). Decode underutilizes the card; bf16 weights cap context at 32K.
[2026-07-23 10:12:29] Trial 2 complete (official FP8 variant, chunked prefill tuned). 1240 tok/s, TTFT 340ms; max-model-len now 131072.
[2026-07-23 10:41:55] Trial 3 complete (same config on H200 141GB). 1685 tok/s, TTFT 295ms; KV headroom allows the full native 262144 context at concurrency 8.
[2026-07-23 11:07:13] Trial 4 complete (prefix caching + cascade attention). 1608 tok/s (-5%). Rejected: random benchmark prompts get zero cache hits and pay the hashing overhead.
[2026-07-23 11:34:47] Trial 5 complete (batched tokens 16384, full CUDA graphs, async scheduling). 2140 tok/s, TTFT 262ms. Best config: trial 5, with 2.7x throughput over baseline.
[2026-07-23 11:52:24] Final service verified: readiness probe passed, context length confirmed at 262144. Final benchmark through the service URL: 2050 tok/s, TTFT p50 318ms. Report written.

dstack preset lists saved presets along with in-progress creations:

:::shell
$ dstack preset list
 BASE                                ID        GPU                  BENCHMARK                     STATUS          SUBMITTED    NAME
 Qwen/Qwen3-Coder-30B-A3B-Instruct
                                     4c9a17d5  H200:141GB:1         con=8 2140 tok/s TTFT 262ms   verified (5/5)  3 min ago    qwen3-coder
 Qwen/Qwen3.5-27B
                                     b7e42f19  RTXPRO4500:32GB:1..  con=8 342 tok/s TTFT 389ms    verified (5/5)  yesterday    qwen35-27b

To deploy a preset as a standard dstack service, pass its ID to dstack preset apply:

:::shell
$ dstack preset apply -f preset.dstack.yml --id 4c9a17d5

Presets are an experimental feature. See the documentation for more details, and share feedback via GitHub issues.

Backends

AWS

NVIDIA RTX PRO 4500

dstack now supports AWS G7 instance types with NVIDIA RTX PRO 4500 Blackwell GPUs, from the single-GPU g7.2xlarge up to the 8-GPU g7.48xlarge, including spot:

:::shell
$ dstack offer -b aws --gpu RTXPRO4500
 #   BACKEND          RESOURCES                                                  INSTANCE TYPE  PRICE
 1   aws (us-east-2)  cpu=8 mem=32GB disk=100GB gpu=RTXPRO4500:32GB:1 (spot)     g7.2xlarge     $0.3963
 2   aws (us-east-2)  cpu=32 mem=128GB disk=100GB gpu=RTXPRO4500:32GB:1 (spot)   g7.8xlarge     $0.4086
 3   aws (us-east-2)  cpu=16 mem=64GB disk=100GB gpu=RTXPRO4500:32GB:1 (spot)    g7.4xlarge     $0.4965
 4   aws (us-east-2)  cpu=48 mem=192GB disk=100GB gpu=RTXPRO4500:32GB:2 (spot)   g7.12xlarge    $0.7128
 5   aws (us-east-2)  cpu=96 mem=384GB disk=100GB gpu=RTXPRO4500:32GB:4 (spot)   g7.24xlarge    $1.4257
 6   aws (us-east-2)  cpu=192 mem=768GB disk=100GB gpu=RTXPRO4500:32GB:8 (spot)  g7.48xlarge    $2.8513
 7   aws (us-east-1)  cpu=8 mem=32GB disk=100GB gpu=RTXPRO4500:32GB:1            g7.2xlarge     $2.52
 8   aws (us-east-2)  cpu=16 mem=64GB disk=100GB gpu=RTXPRO4500:32GB:1           g7.4xlarge     $3.0421