独立 AI 资讯库
从自动抓取数据库实时读取，适合快速浏览、检索和后续扩展成完整资讯站。
I held off as long as I could, but finally gave in and set up strata. I am pretty blown away. 3.8 27b at Q6 kv8 was my driver before and that works super well, but Flash Next in strata IQ4 xs is really winning me over. Its not perfect, but it feels like models I was using professionally last year, w
Been working over the last few days getting DeepSeek V4.1 Flash 1017 (Unsloth UD-IQ1_S) running in Basalt (inference engine I'm developing). Specs: RTX 5090 (400W) + RTX 5060 Ti 16 GB (180W), 128 GB DDR5. Methodology: Single 243 GB file (weights + vision + drafter). Engram table on SSD. All experts
GitHub: https://github.com/alkove-ai/alkove (Apache-2.0, code and weights) Hi r/LocalLLaMA, A bit about me first: my day job is building LLM systems in production, a tool-calling assistant with its RAG, and two audio-heavy pipelines. The first turns a company's latest content into a two-host
Gguf's: https://huggingface.co/InternScience/Sage-A1.5-Q4_K_M-GGUF https://huggingface.co/InternScience/Sage-A1.5-Q8_0-GGUF "Sage‑A1.5 is a 35B Mixture‑of‑Experts agentic model based on Qwen3.6-35B-A3B from Shanghai AI Laboratory, built to advance long-horizon scientific problem solving throu
Raon-1 mini is a 30B-A4B Mixture-of-Experts (MoE) language model, designed and trained from scratch by KRAFTON AI. It demonstrates strong capabilities in Korean conversation and agentic tasks. As part of the Raon open model family, it is released under the Apache 2.0 license. https://huggingfa
Fits entirely in a 5090, no offloaded experts. Super-solid sub-2bpw! (Liquid cat demo video generated by the quantized model) From the Cat's mouth: Stable inference throughout, no reasoning loops, just smooth sailing at 200-300 t/s (and a lot of compacting since one 32 GB GPU still doesn't le
hey folks, its Ye, I work on an open source project called Kapitsa: its a fast Apple Silicon optimized inference engine built for Qwen and Nemotron models. this week two new releases landed in local inference: Inco shipped Splash 1.4.0 yesterday, and we shipped Kapitsa 0.2 today with a new spe
I’ve seen vastly more questions than answers about how people are using their local AI so I figured I’d make a post about my experience. It seems most people are looking for frontier-level coding expertise for huge projects with their setups, and that anything less is not enough. I disagree and don’
Insane numbers, so far only Qwen3.8-27B is supported but wow those are some good numbers. Will watch closely when this lands in llama.cpp/vLLM ### Qwen 3.8 27B Benchmark Results Method Speedup over Vanilla MTP 2.22× DSpark 2.57× DFlash 2 2.80× EAGLE-4 3.59× submitted by /u/T
https://huggingface.co/XHToken/Spark-X2.6-4B-GGUF Introduction A little over a month has passed since Spark-X2.5. In the rapidly evolving world of AI agents, that is enough time to rethink, refine, and move the frontier forward. Building on the Spark-X2.5-4B, we introduce Spark-X2.6-4B, the la
https://huggingface.co/bartowski/Agnes-AI_Agnes-3.0-Qwen-GGUF Strong agentic performance: agnes-3.0-qwen scores highest in its comparison set on τ³-Banking, and is within about one point of the best model on Terminal-Bench 2.1, while being a much smaller model than most of the flagship system
just noticed in my reasoning threads that seemingly its favorite word is now "hmm": Hmm!!! What is that 1? Hmm — it could be the has_value/nullable marker for the MQDescriptor parameter! For in MQDescriptor q (non-null), the NDK client writes writeInt32(1) before q.writeToParcel(). Hmm — OR it's the
At Gradium we just made our first open-source model release with Phonon, a collection of small Text-to-Speech models designed to run on device. The inference stack is fully open-source and targets 10+ different backends (CPUs, GPUs, NPUs), most of them in Rust. We hope that the community will
EDIT: Note that I built off the official vllm documentation and I just got done validating the system. This is now my control. That means this is a slow and stable config. Now that i have a baseline control I’m going to begin tweaking settings for performance runs. I’ll make posts as I find se
A few of you pointed out in my last post that comparing Flash-Next and Opus with different harnesses and effort levels wasn't fair. So here is a more controlled comparison. Same task for every run: a real feature from the LlamaStash roadmap (save llama-server's KV cache before an unload and restore
from Ornith: Ornith-1.5-122B Chirp Chirp! 🐦 We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, an
Long-horizon agents compact their history to continue within a finite context window, but a textual summary alone may not support every subsequent decision. We introduce REMORY, a neural memory network that supplements the summary with a bounded sequence of soft memory tokens. Given the histo
Speculative decoding has become a mainstream technique for accelerating large language model (LLM) inference by using a lightweight draft model to generate candidate tokens that are subsequently verified by the target model. However, agentic workloads introduce long and continuously evolving
Some time ago in this same place someone had suggested having your model create paintings through python (with programming) leaving it quite free. Doing so I had noticed how the model, after creating the first draft, made visual checks and quickly decided what he didn't like, adding corrections just
This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces. submitted by /u/feelspeaceman [link] [comments]
submitted by /u/Fcking_Chuck [link] [comments]
https://huggingface.co/migtissera/Tess-4.5-27B-GGUF from the author of Synthia I like this part: Less thinking. Same answers. Tess-4.5 reaches the same score with a quarter fewer tokens. Across all 150 BenchLocal scenarios it generated 25% fewer tokens than its base, reasoning included. It is
Hey LocalLLaMA, I released GameSummoner a bit over a week ago as an open source program and said "it works locally, but is a bit hard to use." (Post) Over the last few days I've cleaned up my prod host (https://gamesummoner.com) and made it much easier to run locally. It's now one line: git cl
So, I was always really fascinated with the idea of a smart/capable-enough model that worked at blazing fast speed. I assumed we needed some kind of ASIC-style dedicated hardware for a specific LLM to manage that, but then I saw DiffusionGemma could actually get some massive speeds on the right hard
Hey r/LocalLLaMA, Blockway here, a small team from Hong Kong. After our v1 post, the most common question was basically "ok, but why does this model exist?" Totally fair. Here's our honest answer. The problem Long agent sessions eat VRAM. On a normal dense model every layer keeps a KV cache, a
Ming-3.0-audio-flash is up on Hugging Face. It takes audio alongside text instructions and can return text or a spoken response. The model card lists transcription, spoken Q&A, long-recording understanding, and reasoning about speech, environmental sounds and music. A few details: 126B total
Hear clearly. Understand deeply. Act intelligently. Ming-3.0-audio-flash is a native audio-language model from inclusionAI. It accepts audio together with text instructions and supports speech recognition, spoken question answering, audio understanding, audio reasoning, and optional synthesize
I was working on exactly the same thing so hoping I can scrap my half-done job and adopt this. It seems to have all the features I was targetting. submitted by /u/DeltaSqueezer [link] [comments]
We’ve been testing which agent harness works best for small local models. The result: a single-tool IPython harness can match much heavier production harnesses on Terminal-Bench Lite, and much of the “randomness” in small-model agents comes from noise in the environment, not the model. Setup
Using u/lxe's https://github.com/lxe/dsx engine, I created DeepSeek-V4.1-Flash-1017-dsx-2.6bit for programming and agentic tasks. The model's weight has been uploaded. Config Experts KL ↓ Top-1 ↑ Perplexity ↓ FP8 / FP4 teacher FP4 0 100% 8.56 Lucebox ROCMFP 2.5 bpw 0.253 78.7% 9.78 Author's