[WIP][Feature] Integrate Suffix Spec Decoding

[fluctlux]

What this PR does / why we need it?

This PR integrate suffix decoding (https://arxiv.org/abs/2411.04975) from vllm (vllm-project/vllm#25784)

Suffix Decoding is a dynamic n-gram matching method that:

1. Uses suffix trees to generate speculative tokens quickly using branch frequency counts.
2. Can keep a history of prior model responses, which tends to work very well with repetitive agentic use cases.
3. Can be dynamically updated with newly generated tokens, and FIFO eviction of older requests.

Does this PR introduce any user-facing change?

This feature should be implemented as opt-in and remain seamless for users who do not require suffix speculative decoding.

For users who wish to enable it, they must first install arctic-inference:
pip install arctic-inference

After installation, the suffix speculative decoding feature can be enabled using the following speculative config:
--speculative_config '{"method": "suffix", "num_speculative_tokens": 5}'

How was this patch tested?

This PR is currently being tested on vLLM main:vllm-project/vllm@83f478b
with PR vllm-project/vllm#25784

In our previous testing, suffix decoding achieved a 13%-30% throughput improvement over n-gram on the sonnet dataset, tested on vllm-ascend v0.9.1 with concurrency ranging from 2 to 40.

• vLLM version: v0.11.0
• vLLM main: vllm-project/vllm@83f478b

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.