独立 AI 资讯库
从自动抓取数据库实时读取，适合快速浏览、检索和后续扩展成完整资讯站。
I know there are many posts about this, but I would explicitly like to know the harness names instead of having a general discussion. Let's see what the community actually uses! For others, please comment below. Feel free to say why you prefer yours. Thanks View Poll submitted by /u/DifficultP
Hello; I'm a semi-beginner at local AI. I've been experimenting with this tech for a while, and I still haven't found a proper harness that fits my models, hardware, and needs. My use case is pretty simple: web search, fetching, and browser use. Summarizing websites and having the agent explain stuf
From my testings, gemma 4 26ba4 and qwen 27b are doing great But curious if someone knows a better option submitted by /u/Whole_Alternative_18 [link] [comments]
u/ilintar 👍 submitted by /u/pmttyji [link] [comments]
iq2_xxs tensor level allocation recovered reasoning from 28.9 -> 69.5 at the same 3.3gb budget. https://huggingface.co/ByteOtter/gemma-4-E4B-it-CADA-IQ2_XXS I posted my Gemma 4 12B q3 result a couple days ago, where tensor level allocation gave me an +8.55% relative improvement over the categ
It freaking worked lol🔥 Deepseek-v4-flash-0731 @UnslothAI 's IXQ2/Q3 checkpoint on one single RTX4090 with just 64 GB of RAM at usable token rate without dspark. All kernels running on Blaze (my custom developed ML compiler + inference engine) - no llama.cpp or vllm in the picture. The setup k
It could be just me and my setup, but I just tried to get fable to adjust my Qwen 3.8 deployment script and (simple task, mostly knob turning).... and it outright refused. Censor box immediately kicks in. Not reading too much into it, but it did make me giggle. submitted by /u/NotumRobotics
https://preview.redd.it/xwbkbbj55ijh1.png?width=1554&format=png&auto=webp&s=abd354c8a6bef033d5fb8383c49e4a682ab22105 Just wait and see https://github.com/modelscope/ms-swift/commit/ab726e9d445a6520a70df2c831177d46adb1f589 submitted by /u/BazzyIm [link] [comments]
Every time I self-host a model I rebuild the same stuff: start the container, set up a route, check why it died overnight, remember to shut the GPU off before it burns money. So I'm building a panel that handles it. Start/stop models, OpenAI-compatible endpoint, health checks with auto-restart, basi
After a lot of tweaking, I have come to the conclusion that 27B lacks a reasoning mode that is between low and xhigh. The "medium" mode isn't actually medium, it erases the explicit instructions to the model. When medium is enabled, the model acts very differently - to me it looks like it regresses
Muse Glimmer Cookbook Clone it, run it on the GPU you already own, and ship a working agent — one that plans, calls tools, and self-corrects — in a single sitting, fully offline. Muse Glimmer is an open-weight model built for local agentic work on a single GPU. This cookbook is how you go fro
Megathread to help with the influx of duplicate / similar posts around the release of the Qwen 3.8 27B release. Quants Fine-Tunes & Abliterations Chat Templates Inference Server Support & Configuration Experiences, Benchmarks & Model Comparisons Official: https://huggingface.co/Qwen/Qwen3.8-27B h
So a bit of context, I am a cybersecurity senior analyst I am interested in LLMs for that field especially with MCPs to connect them to the tools or for writing scripts I started this field by doing assembly language reading for hacking games when I was a teenager then that became malware analysis
mlx-dspark is an MLX port of DeepSeek's DSpark speculative-decoding drafters (the DeepSpec release), plus z-lab's DFlash, with one lossless verify loop. v0.10.0 adds Qwen3.8-27B via RadixArk's drafter, the first SpecForge/SGLang-packaged head it loads. Numbers (M4 Pro 48 GB, medians of 3, gree
I'm always testing an image prompt with a picture of a historic place in my hometown – a small but well known 250,000 people town in Germany. I'll just ask the model, in which City this photo has been taken. With the 3.6 generation of both the 27B and the 35B A3B variants, the models sometimes got t
The demo video above was created with a single prompt using Qwen 3.8 27B Q8_K_XL. Prompt: create me a single page, html/js/css GeoCities, Laser Wave, 80's retro, minecraft clone named RetroCraft. Embed it in a single go binary that serves the HTML page. It took Opencode only a few minutes to
Someone made a heretic version of Qwen 3.8 27B, giving us a local Opus 4.6 tier model but without any refusals or safeguards! Fuck Dario submitted by /u/Temporary_Idea8880 [link] [comments]
I'm currently testing out Qwen3.8-27B using Unsloth's UD-Q4_K_XL running a freshly rebuilt llama.cpp. I have a 22GB RTX 2080TI on which I'm able to fit 100k context with q8_0 quantization, and using MTP with --spec-draft-n-max 4 I get about 40tk/s which is slightly less than Qwen3.6-27B but usable e
Not a transformer. It's a recurrent latent reasoning setup that keeps "thinking" in latent space before answering. Sits completely outside the published cost/accuracy frontier for ARC-AGI, and something this size runs on basically anything. Paper is from the Pathway team, dropped 4 days ago. I
Tested this prompt on the full version (BF16). Though this was a single prompt, I executed using vscode GH copilot extension on agent (allow all) mode and let it do its thing. So there were 54 model turns. Copilot kept iterating and reviewing the functionality (using playwright) without any us
Facing any issues? Chat Template is fine? Looping issue? Too much reasoning thing? How's MTP with this one? Any other issues faced by Qwen3.6-27B & Qwen3.5-27B during release time? If I missed any other items, please mention in your comments. AND Share comparison with Qwen3.6-27B. On Memory & t/
With your experiments, Qwen 3.8 27B most close which frontier model? And please specify which quantization you run. I will post to comments my tests and experience too. submitted by /u/BarberIcy366 [link] [comments]
submitted by /u/kevin_1994 [link] [comments]
submitted by /u/de4dee [link] [comments]
What's happening in China? Kimi K3-2.8T Qwen3.8-2.4T DeepSeek-V4-Pro-0813-1.6T GLM-5.3-743B They’re all less than a month old! submitted by /u/chibop1 [link] [comments]
So far, it seems like Qwen 3.8 might drop today as a 27B dense model. If that’s the case, no MoE offloading this time , I used to run Qwen 3.6 35B-A3B at around 70 tok/s on an RTX 3060, but offloading a dense model is a completely different story it can be 100× slower or Even slower submitted by
If you scroll down from the countdown at https://huggingface.co/Qwen/Qwen3.8-27B, you see a big model card with a bunch of sections: Highlights, Model Overview, Quickstart, Best Practices, Citation, etc! No benchmarks on this yet as far as I can tell. We'll still need to wait another 5.5 hours
So, I have been dreaming of getting 17 tokens per second using my RTX 3050 6GB version on a decent context window for Hermes needed above 60k. The hope is that has was a 22GB of DDR 4, hoping they can take some of those experts and give me room for context. What did I get 10 or less tokens per
Not directly LOCALLlama related but I thought it was interesting since Mistral and Z.ai are competitors, and more surprisingly they are pricing it (GLM-5.2) even cheaper than their current flagship model Mistral Medium 3.5. Does this suggest a pivot in Mistral's strategy? Are they going to abandon f
Processing img az99qopcg8jh1... So same as my prior post 1bit test... although this 1bit is a bit interesting you can read on unlsoth blog https://unsloth.ai/docs/models/qwen3.8 508 gigs being used I am using unsloth studio on the mac ultra 512. Im getting ~50pp and ~9.6 tgen drop to 5 at 50k