3+ day, 8+ hour ago   (47+ words) vLLM CPU DeepSeek-V4 DSpark (speculative decoding) — deferred, not implemented. DSparkDeepseekV4ForCausalLM is only instantiated when the speculative- decoding registry resolves it by name for a DSpark speculative config; a plain (non-spec) DeepSeek-V4 CPU model never constructs this class. This stub exists solely…...

5+ day, 18+ hour ago   (580+ words) This is an example of doing DeepSeek R1 RL training using 128xH100 GPUs. We will use bf16 for training, and an fp8 format with 128x128 blockwise quantization for inference. The maximum response length is 32k, and dynamic sampling will be used to filter data during training....

5+ day, 17+ hour ago   (685+ words) An external rollout engine is a vLLM engine that is not launched by the vime training job. Another system deploys and owns the engine lifecycle; vime connects to those engines during training, registers a router, and syncs updated actor weights…...

5+ day, 17+ hour ago   (309+ words) vime can attach lightweight execution traces to each rollout sample. These traces capture span-style events such as generation and reward-model calls, and they can be inspected later from a saved rollout debug dump. To inspect traces later, save rollout debug…...

5+ day, 17+ hour ago   (288+ words) PD Disaggregation separates Prefill and Decode workers in vLLM rollout. This is especially useful for multi-turn, long-context, and agentic RL workloads where prompt processing and token generation have very different compute and memory profiles. Use PD Disaggregation when: rollout contexts…...

5+ day, 18+ hour ago   (364+ words) The environment setup, data, and checkpoint conversion are the same as for the Qwen3-4B model. You can refer to Example: Qwen3-4B Model, replacing mentions of Qwen3-4B with GLM-4.7-Flash. To convert the Hugging Face checkpoint to torch_dist format: Execute the training script: Here, we…...

6+ day, 8+ hour ago   (920+ words) DeepSeek-V4-Flash-Vision-Exp is a multimodal mixture-of-experts model in the DeepSeek-V4 family. It combines the DeepSeek-V4 language model with a vision encoder and aligner, and accepts text, single-image, and multi-image requests through the OpenAI-compatible chat API. Support on vLLM Ascend is…...

1+ week, 6+ day ago   (41+ words) vLLM Per-request speculative-decoding acceptance metrics. Apply response_format overrides to structured_outputs. Validate structural tags before they are sent to the engine. Engine-side validation reports malformed structural tags as generation failures. OpenAI request parsing should classify them as bad requests....

1+ week, 6+ day ago   (122+ words) Return whether a 64-bit integer is prime. Build deterministic hash multipliers for one PLE layer. Build per-head vocabulary sizes, offsets, and total row count. Return the count-th prime strictly greater than start. Mix an integer into a deterministic unsigned 64-bit…...

1+ week, 6+ day ago   (20+ words) vLLM Qwen4Exp low-latency GEMM hook for AMD ROCm. Keep the standard vLLM linear methods on AMD ROCm....