独立 AI 资讯库
从自动抓取数据库实时读取，适合快速浏览、检索和后续扩展成完整资讯站。
https://huggingface.co/yandex/AliceAI-Foundation-80B-A3B-Base It's not a Qwen3 finetune, it's actually its own fully custom architecture. No Llama.cpp support yet sadly (Also note that this model is NOT post-trained like Qwen3.5/3.6) submitted by /u/Iwaku_Real [link] [comments]
I decided to do this experiment to see what would be the ability of an LLM model to, in a single shot, transfer the knowledge it may or may not have about that artist/brand/designer/animator to a completely new concept. I tested it on two fronts; first SVG, and then pages. SVG was easier and
I’ve been working on making small models more capable at agentic coding and work, because most people in the world don’t have the sort of hardware needed to run 3.8-27B, or even 35B-A3B or 9B dense, and I want to extend local agentic coding capability to less privileged users. This quant can b
submitted by /u/themixtergames [link] [comments]
Small, free finding. I use local Qwen models for typed decisions: a state plus a question with fixed allowed answers, and I read the probability of each answer from the logits of one forward pass instead of generating text. I used to build the prompt like you would for a human: situation first
Quote: DeepSeek is training a 2T-parameter model and plans to eventually build an 8T-parameter model. https://x.com/wallstengine/status/2101982843656388644 Current DeepSeek models: Flash parameter count of 552 billion Pro: 1.6T (trillion) total parameters with 49B (billion) activated weights per to
https://preview.redd.it/nulsv53o8vqh1.png?width=4500&format=png&auto=webp&s=74765dbd409f4c221640f9f6000a685f6fdbb242 Spent weekend benchmarking the Splash engine (by Incoai) and extending its architecture to native 8-bit on Apple Silicon (M5 Pro, 64 GB unified memory). Splash is a compiled C++
https://x.com/QwenDevs/status/2101917379785838660 submitted by /u/Bestlife73 [link] [comments]
Laya is an open-weight (Apache 2.0) "System 1" decision model from Convai Innovations, built by Nandakishor M as an open alternative to TypeSafe's closed Jev API. Instead of generating text, it takes a state (text, an email, a ticket, or JSON) plus typed questions (choice to pick a label, scor
ZCode is now open source, and the reported security issues have been addressed. Source code: https://github.com/zai-org/ZCode The repo includes its desktop app, web workspace, backend, Agent CLI, and runtime. Official announcement: In response to the ZCode product security issues reported by t
I know cloud subscription plans are not exactly the core focus of r/LocalLLaMA, so I want to be clear about why I’m posting this here. I’ve been building my own local text-based RPG/game harness and using external models as testing infrastructure: basically stress-testing the harness, finding errors
I wanted to share a quick update and performance video running the full Moonshot AI Kimi K3 (moonshotai/Kimi-K3) model across my 16x GB10 cluster. Getting a 2.8T parameter model running smoothly requires custom runtime patches and a solid network layout, but throughput and concurrency on c
TL;DR: Local agent loop, ~21 days, one RTX 3090. Task was pretty much "build a CUDA inference engine for optimized for yourself on this GPU arch." Got working kernels and benches, not a win over llama.cpp. ~12 human messages. Compaction ate ~83 hours. Old joke: you don’t criticize how well the bear
After seeing u/Nandakishor_ml’s post introducing Laya, I wanted to see how fast it could run in a standalone C++ implementation. Credit to u/Nandakishor_ml for the architecture, training and open-source release. My contribution is the inference implementation: laya.cpp, built on ggml with custom CUD
This weekend, I did a fun experiment to create something similar to Jev. I LoRA fine-tuned Qwen3.5 4B using a mix of publicly available datasets and synthetic data. For the synthetic data, I used DeepSeek V4.1 Flash, around 25M tokens. I trained the model for about 2 hours on a rented RTX 3090. So f
Another day, another Qwen Flash Next speedup submitted by /u/jacek2023 [link] [comments]
I forked llama.cpp to implement Declarative Attention (arXiv:2609.02737, Google DeepMind and KAIST AI). The model declares in its own output which context chunks it needs <focus magic\_chunks="N">), and the engine listens and restricts what the following tokens can attend to. No scorer, no training:
Meet Qwen-Image-2.1, the most balanced and cost-effective image generation model in the Qwen-Image series! Now open weights! 🎨 A unified model for both generation and editing, delivering top-tier quality in a lightweight package. Highlights: - Compact & exceptionally fast: A lightweight 7B ar
Overclocking the CMP 170HX 40GB I was able to get the memory bandwidth from 1,386.2 GB/s to 1,890.1 GB/s, that's a +36.4% increase. Qwen 3.8 27B token generation jumped from 110 T/S to 202 T/S, same config, nothing changed except the overclock. Just throwing this out there for whoever owns one of t
Hey everyone, After wrestling with hardware and engine issues for days, I finally got Qwen 3.8 Next running properly on my multi-GPU rig. Thought I’d share the setup journey, benchmarks, and thermal results for anyone trying something similar. Seeing all the ongoing memes on Reddit about multi
I have been anti-low quant. I can feel the difference, I swear. I am also anti-quantized KV cache. I have been burned there - I think the tradeoffs compound, especially at longer contexts. Due to this, I have been running Qwen 3.8 27B (UD-Q8_K_L) on 2x of the 3090s in TP. I swear by it. It is
I am trying to learn more about long-term memory and how people are handling that nowadays. Historically, I have used a mixture of paid cloud AI services (ChatGPT, Gemini, OpenRouter API) and my own local inference. However, at this point, I'm convinced that ChatGPT is deliberately throttling my us
I was somewhat disappointed with the performance of Qwen 3.8 Flash Next on my single RTX 5090 using llama.cpp. One issue is that llama.cpp still has no Expert Caching implemented for MoE models. There are various PRs and discussions (see here, for example), but nothing is merged yet. Then I stumbled
They claimed open-weight models are dangerous but the benchmarks say otherwise. Source submitted by /u/Intrepid_Travel_3274 [link] [comments]
Play it right in your browser, no download. Some of the GLB’s are messy, but overall I’ve enjoyed playing with it, and the last iteration made was the gun/weapon placement. Contributors are welcome, would like to see what can be made of this from local inference only. https://thecarsonford-do
https://github.com/W61k3r/LexiPanel for those interested in the panel Screenshot of heavy load. I did offload vision to a spare 2060 instead of local ram. The control panel was generated with qwen3.8-27b davidau q4k_m max mtp. Made the panel from scratch over a week, basically just building f
Took me a while since I'm on a family trip and have limited hardware, but here it is! Von: Open-source "System One" drop-in replacement for TypeSafe's JEV. https://github.com/wfzyx/von https://huggingface.co/wfzyx/von-1.0 It runs entirely on a CPU with 1–2 GB of memory (I haven't spent much time opt
I envy the NInfer, but I don't have a 5090 card, I have a 5060 TI :( I would like tonrun Qwen 3.8 Q3 at extreme speeds :( NInfer: https://github.com/Neroued/ninfer submitted by /u/FerLuisxd [link] [comments]
We're upgrading our server (Threadripper Pro 5955WX, 128GB 8-channel DDR4) from two RTX 3090 to four cards. Currently we're running Qwen 3.8 27b Q8 with vLLM for a few users. I'm wondering what we should do next: Keep running Qwen 3.8 27b Q8 and enjoy the performance boost and use a bit of spare VRA
Top 3 papers on HF Daily Paper are all unusually delightful and interesting reads for anyone on the leading edge of local LLMs, agent harness optimization, etc, felt like sharing. DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression https://huggingface.co/papers/2609.19969 Cross-layer KV