Underdog releases 7.89GB quantized model Saluki 27B, with tool-calling self-tests outperforming the original
BIBIBI
AT A GLANCE
Underdog released 7.89GB's Saluki 27B, claiming that its tool-calling self-tests outperform the original.
Article
Underdog has Qwen3.8-27B outperformed 7.89GB: tool-calling self-tests surpass the original
Beating AI news,a16z-backed local AI assistant Underdog released Saluki 27B, based on Alibaba's Qwen3.8-27B model. The model uses approximately 2-bit quantization, reducing its size from approximately 54GB for the full-precision version to 7.89GB. The official claim is that 16GB of memory is sufficient to run it on a laptop, and it supports deployment through existing tools such as llama.cpp. The model weights are released under the Apache 2.0 license.
Saluki uses quantization results from the Institute of Science and Technology Austria's ISTA-DASLab. The laboratory had previously compressed Qwen3.8-27B to approximately 8.4GB. Building on this, Underdog further reduced the model size and focused on optimizing tool-calling performance.
In BFCL tool-calling questions selected from 120, Saluki answered 88 correctly, the ISTA quantized version answered 76 correctly, and the 54GB original answered 84 correctly. In 50 real GitHub issue-fixing tasks, Saluki solved 30, while the original solved 33.
The official claim is that, across 9 evaluations, Saluki retained an average of 96%of the original's performance, but the gaps varied substantially by task. Its mathematics competition and complex-reasoning scores reached only approximately 83%and 85%of the original's, respectively.
Original link https://m.theblockbeats.info/flash/370754
Key points
01
Underdog released Saluki Qwen3.8-27B made using Alibaba's 27B。
02
Saluki 27B using approximately 2-bit quantization, with a size of approximately 7.89GB; the full-precision version is approximately 54GB。
03
The official claim is that a laptop with 16GB of memory can run it, and that it supports deployment through tools such as llama.cpp.
04
The model weights are released under the Apache 2.0 license.
05
In 120 tool-calling questions, Saluki answered BFCL correctly, the 88 quantized version answered ISTA correctly, and the 76 original answered 54GB correctly. 84
06
In 50 real GitHub issue-fixing tasks, Saluki solved 30, while the original solved 33.
07
The official claim is that Saluki retained an average of the original's 9 performance across 96%evaluations; its mathematics competition and complex-reasoning scores were approximately 83%and 85%。
AI-assisted interpretation
The following is analysis, separate from reported facts. Verify important claims independently.
Quantization reduces the space occupied by a model while trying to preserve as much of its capability as possible. Saluki compresses the model to 7.89GB, making it easier to try on devices with less memory, but capabilities change differently across tasks.
Why it matters to readers
A smaller model may be better suited to local deployment; the self-reported results in the material show higher tool-calling performance than the original, but weaker code-fixing, mathematics competition, and complex-reasoning performance.
If you want to run a large model on a personal computer, model size and memory requirements are critical; however, you should not look only at tool calling, but also at performance on your specific tasks.
Risks and unknowns
Tool calling, 9 evaluations, and 16GB running requirements are based mainly on official statements; the material provides no independent retest results.
The material does not provide an official release page or model repository, so the claims that the weights are open and the Apache 2.0 license information still require verification.
Saluki's performance varies substantially across tasks, and the available data are insufficient to confirm that it is suitable for all uses.
Related Developments
Loading event timeline…
Related concepts
quantization
This term is not in the glossary yet. Browse related concepts in the glossary.
