All news
AI

Huawei releases MACaQ, its internal quantizer for local inference

GitHub

SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch. - huawei-csl/MACaQ MACaQ can quantize a 122B-parameter model in 64 minutes. using just 32.5 GB of device memory.

Summary drawn from GitHub's own description of the story.

All tech news
Loading related stories