If you want to achieve Prefill decode separation, is it necessary to keep identical copies of the cache tree and kv cache pool on both nodes?

[hcyz33]

I am thinking about how to achieve PD separation, but I am a bit confused about the transmission of the kv cache.

[zhyncs]

I am a bit confused about the transmission of the kv cache

You can refer to the implementation in the Mooncake paper, which uses a Distributed KV Cache Pool and RDMA for inter-node transmission.