From AFD Experiment to an Evidence-Driven vLLM Roadmap

Why split Attention from FFN?

What the first release establishes

The plugin integrates through vllm.general_plugins and --additional-config, without modifying the vLLM source tree. Its current matrix includes NVIDIA GPU and Ascend NPU paths, synchronous decode and asynchronous prefill connectors, and wrappers for DeepSeek V2/V3-family and GLM MoE models.

Reading the architecture from the code

AFD is not implemented as one large fork of vLLM. It is a chain of deliberately small interception points: plugin registration selects AFD-aware workers and model classes; the Attention runner injects execution metadata; the model wrapper introduces the layer boundary; and a connector-driven FFN runner consumes that boundary.

1. Bootstrap: change the model lifecycle, not the API

vllm serve
  └─ general_plugins entry point
      └─ register AFD model architectures
          └─ parse additional_config["afd"]
              ├─ role="attention" → AFDAttentionWorker
              └─ role="ffn"       → AFDFFNWorker
                  └─ rewrite model architecture → AFD model wrapper

2. Role-specific construction removes unused state

The DeepSeek decoder wrapper does more than skip half of a normal forward pass. During construction, each role creates only the modules it needs. The Attention layer builds self-attention and request-facing state; the FFN layer builds dense MLP or MoE modules. On the FFN worker, get_kv_cache_spec() returns an empty mapping and sampling is explicitly unsupported.

One subtlety: normalization and other shared lifecycle components may exist on both role-specific wrappers where the upstream model-loading contract requires them. “Split” means role-required construction, not a blanket claim that every non-expert parameter exists on only one side.

3. One layer, end to end

The actual split point lives inside the model wrapper's layer loop. Attention computes through post-attention normalization, sends that hidden state to FFN, and later receives the expert result. The residual stays on the Attention side, preserving request-local transformer state.

# Attention process
for layer in layers:
    previous_ffn = receive_if_pending()
    attn_state, residual = layer.compute_attention(previous_ffn, residual)
    send_to_ffn(attn_state, layer_id, microbatch_id)
final_state = receive_last_ffn()
sample(final_norm(final_state, residual))

# FFN process: driven by connector metadata, not request scheduling
for layer_id in layers:
    for microbatch_id in active_stages:
        aggregate, transfer = receive_attention_states()
        expert_output = model.compute_ffn_output(aggregate, layer_id)
        split_and_return(expert_output, transfer.peer_token_lengths)

4. The connector is a two-plane protocol

The connector abstraction separates tensor movement from execution coordination. The data plane has four symmetric operations; the control plane carries per-stage DP token counts and warmup or graph-capture state so the FFN side can allocate the correct buffers before tensors arrive.

Each transfer carries backend-neutral layer_idx, stage_idx, and per-peer sequence lengths. Backends can attach their own transfer state without changing the model-facing call sites. That is what lets NCCL P2P, CAMP2P/HCCL, and asynchronous CAM share one layer loop.

5. GPU topology: fan in, compute, fan out

For P2pNcclAFDConnector, global ranks are ordered FFN first, then Attention. The current topology requires at least as many Attention ranks as FFN ranks and an integer ratio. Each FFN rank owns one subgroup of consecutive Attention peers.

The important systems effect is aggregation: FFN sees tokens from several independently scheduled Attention lanes as one expert batch. The cost is a round trip at every split layer. AFD wins only when improved expert utilization and independent capacity planning outweigh communication, synchronization, and the extra FFN devices.

6. Ubatching turns the round trip into a pipeline

A synchronous implementation that executes Attention, sends, waits for FFN, receives, and only then starts the next slice would serialize the new boundary. AFD's stage metadata and ubatch wrapper create multiple in-flight slices so Attention work for one stage can overlap FFN work for another.

More stages are not automatically better. They create overlap but also multiply small-kernel launches, metadata handling, and graph shapes. The currently validated DBO path is intentionally limited to exactly two ubatches; the async CAM prefill path owns a separate ubatching mechanism and currently does not support graph execution.

7. The code makes invalid ownership fail fast

Early performance signals—and their limits

The announcement reports two focused experiments. They are useful because they expose both the opportunity and the main systems lesson: disaggregation only helps when the Attention-to-FFN allocation matches the workload.

Support is a stack, not a checkbox

The roadmap's most important idea is to replace a global “supported” label with three evidence layers. A successful import does not validate a model, and a successful launch does not validate a production topology.

Six workstreams, not six release phases

The roadmap is organized by dependency rather than calendar. Compatibility and delivery are foundational, while model, prefill, hardware, feasibility, and contributor-tooling work can advance in parallel when their prerequisites exist.

Measure realistic prefill, not just launch success

References

1. 1.Announcing vLLM AFD Plugin: Disaggregating Attention and FFN for Flexible MoE Serving
2. 2.[Draft] [RFC]: afd-plugin project roadmap
3. 3.vLLM AFD Plugin repository
4. 4.FastAFD architecture and performance analysis