Enhancing AI Performance on Salvaged Data Center Hardware: A Deep Dive into Custom Quantization for “Landfill” GPUs
As artificial intelligence models continue to grow in complexity and size, leveraging existing hardware—especially older, repurposed GPUs—becomes increasingly vital. In a recent project, I explored how to optimize large-scale mixture-of-experts (MoE) models on salvaged datacenter GPUs, such as Tesla P100s, V100s, and even aging GTX 1080 Ti cards. These devices, often available at affordable prices, are typically overlooked due to limitations in mainstream quantization techniques, which frequently favor modern silicon architectures.
Addressing the Limitations of Conventional Quantization
Standard quantization methods, including int8 tensor core optimizations and fast fp16 paths, are predominantly designed for the latest GPU architectures like Turing and Ampere. These approaches often neglect pre-Turing cards, resulting in sub-optimal performance when deploying large models on older hardware. Recognizing this gap, I developed a tailored solution: pxq_llama, a fork of the popular ik_llama.cpp project, incorporating a specialized quantization family dubbed PXQ and optimized CUDA kernels suited for legacy GPUs.
Benchmarking Performance Gains
To validate the effectiveness of this approach, I conducted rigorous head-to-head comparisons:
- Hardware Used: Same GPU card, identical 5,800-token prompt, temperature=0, median of three runs.
- Models: 35-billion parameter MoE with 256 experts.
- Configurations: Upstream ik_llama.cpp versus my pxq_llama implementation, built with applicable performance flags and optimized quantization (IQ_K).
The results are striking:
| GPU Model | Prefill Speed | Decode Speed |
|————|————–|————–|
| Tesla P100 | +88% | +30% |
| Tesla V100 | +13% | +13% |
| GTX 1080 Ti | +25% | — |
Specifically, the P100 outperformed the upstream version by 88% during the prefill phase and 30% during decoding. The V100 saw a 13% boost across both phases, while the aging GTX 1080 Ti achieved 25% faster decode rates.
While the upstream implementation retains an advantage in one aspect—its 2-bit MMQ prefill tile is more mature on the 1080 Ti—this is expected in early iterations and is on my development roadmap for future enhancements.
Innovations in Quantization and Kernel Optimization
The core of pxq_llama revolves around a novel quantization scheme called PXQ, designed specifically for hardware constraints of older GPUs. Key features include:
- Quantization Strategy: Utilizes learned codebooks combined with per-row fp16 anchors amortized over 64-row panels, along with 4-bit sub-scale per 16-element block.
- Variants: PXQ2 (2.27 bits per weight), PXQ3 (3.27), and PXQ4 (4.27).
- Kernel Enhancements: Fused CUDA kernels that are meticulously tuned for Pascal and Volta architectures, enabling grouped MoE GEMM, K-split GEMV decoding, gate and GLU fusion, DeltaNet linear-attention fusion, and residual-add operations—all bit-exact and environment-gated.
Moreover, I introduced a flexible quantization mode within llama-quantize that dynamically mixes PXQ2/3/4 per expert tensor to utilize the full capacity of any GPU, with preset profiles for 16GB and 12GB VRAM configurations.
Additional Performance Improvements
A standout feature in this release is an optional int8 DP4A prefill tile tailored for Nvidia GTX 10-series cards. For example, it significantly increased decode throughput on the 1080 Ti from 251 tokens/sec to over 1,001 tokens/sec during cold-start prompts.
Practical Insights and Recommendations
Key observations emerged during benchmarking that benefit both end-users and developers:
- For pre-Turing cards, turning the fuse_flag_off (FA off) during prefill operations markedly improves speed—by 26-56%—without compromising accuracy. Conversely, FA on excels during the decode phase. Adjusting this setting based on workload can yield substantial gains.
- Replacing upstream’s gUDF-based inference with my custom kernels (without PXQ) results in a modest 2.7-3.3% speed improvement and identical output at temperature=0, confirming that performance gains primarily stem from the tailored format and kernels rather than code modifications or model pruning.
Reproducibility and Transparency
All benchmarking scripts, raw data, SHA verification gates, and detailed environment variables are available in the project repository. The docs/LEVERS.md file documents every tuning parameter, including dead-end options like Pascal’s DMMV mode, CUDA graph replay nuances, and the impact of different prefill kernels. This transparency ensures that others can replicate results or adapt them to their hardware.
The Broader Impact: Breathing New Life into Aging Hardware
The primary motivation behind this work is to democratize access to large AI models. With a budget of roughly $150, a Tesla P100 consumes about 250W and is capable of decoding a 35-billion parameter MoE model at a rate exceeding 58 tokens/sec—faster than many cloud-based API services. These older GPUs, often destined for e-waste, can continue contributing meaningfully to AI inference tasks with the right software optimizations.
Next Steps: Quantized Weight Compression
While this post focuses on engine optimization, a subsequent update will explore quantized weight reduction techniques, further lowering memory footprints without sacrificing performance.
Conclusion
By developing custom quantization formats and finely tuned CUDA kernels, it is possible to unlock significant performance enhancements on “landfill” GPUs. This approach not only extends the utility of aging hardware but also fosters a more sustainable and accessible AI ecosystem.
For those interested, the full project repository is available under the MIT license:
https://github.com/poisonxa16/pxq_llama
Feel free to reach out with questions about kernel tuning, quantization strategies, or deployment specifics. Let’s empower legacy hardware to enable next-generation AI applications.
