HIGGS (Hadamard Incoherence with Gaussian MSE-optimal GridS)

Chronicle

2025: Russia has made a breakthrough in optimizing large language models - now they do not need powerful servers

artificial intelligence Yandex The Research Laboratory, together with the HSE Massachusetts Institute of Technology MIT (MIT), the Austrian Institute of Science and Technology (ISTA), and the King Abdullah University of Science and Technology KAUST , developed a revolutionary method for rapidly compressing large language models without LLM loss of quality. Thanks to the new technology neuronets , expensive and powerful servers GPUs are no longer required to work with - just a regular or smartphone laptop. This became known on April 11, 2025.

The new quantization method is called HIGGS (Hadamard Incoherence with Gaussian MSE-optimal GridS). The technology allows you to compress neural networks without using additional data and without computationally complex parameter optimization, which is especially valuable when there is a lack of suitable data for further training the model.

𝓲

Фото: Freepik

Previously, to run the language model on a smartphone or laptop, it was required to quantize it on an expensive server, which took from several hours to several weeks. The new method allows you to perform this process directly on your phone or laptop in minutes.

HIGGS technology makes large language models more accessible not only to large companies, but also to small organizations, non-profit laboratories, individual developers and researchers. This opens up new opportunities for LLM use in a variety of areas, especially where resources are limited - such as education or social.

One of the key advantages of the development is the ability to compress even such giant models as DeepSeek-R1 by 671 billion parameters and Llama 4 Maverick by 400 billion parameters. Before the advent of HIGGS, these models were able to be quantized only by primitive methods, which led to a significant loss of quality.