独立 AI 资讯库
从自动抓取数据库实时读取，适合快速浏览、检索和后续扩展成完整资讯站。
Moondream 3.1 is a vision language model with a mixture-of-experts architecture (9B total parameters, 2B active). It delivers state-of-the-art visual reasoning and detection while staying fast and cheap to deploy. Skills include query, detect, point, and caption, all native and all returning s
I was surprised to see that Codex is actually open source, so to my understanding, if you use a local model, it works fully locally. How does it compare to the other popular harnesses like Pi Code and Open Code? submitted by /u/A_Wild_Entei [link] [comments]
Anthropic’s research on Claude found a silent internal workspace they call J-space — hidden reasoning that never shows up as visible text. Classic example: the model answers 49, but inside J-space they caught 21 → 42 → 49. Important distinction: Chain-of-thought = text you can read J-space = silent
As many of you know t/s is super important. It's how fast your stuff gets done. I create via open code benchtest and run it. Thanks to it i know that if i don't run at least 4 agents i basically leave HALF of performance. So whatever you do single project in open code that uses one agent or 4 projec
TL:DR: I’m a grad student in AI. I saw that Google released TabFM and TimesFM last week. I built an MCP wrapper to serve both transformer models in a single Docker container so you can connect their new ML transformer models to a local LLM via Open WebUI, Claude Code, or Codex and do ML tasks
I recently posted some posts with VLLM showing issues with TTFT and concurrency with 4x 5060 ti's. Wanted to share this benchmark to provide what worked for me so other people that are planning to go the 4x 5060 ti route aren't discouraged. Benchmark Results ============ Serving Benchmark Result ===
Hey I am running a few models on a strix halo box. Especially for the larger models (like Qwen 3.5 122B) they work okayish performance wise if the cache is utilised properly but a full cache miss at 100k context causes roughly 10-20 minute of PP time - which is extremely annoying. I will first show
Anthropic recently published their paper on "Global Workspaces" (J-Space) inside language models, showing that looking at internal "workspace noise" (entropy) can catch hallucinations better than just looking at output logprobs. `solarkyle` followed up with a great open-source implementation showing
Hey guys, I've been running Qwen 3.6-27b locally on an RTX 3090 for a while now, and it's been genuinely great at solving software issues. However, life happened and I recently had to use Opus 4.8 alongside the Zed editor and the Claude Code agent. While I can definitely see a noticeable jump in pur
I saw a lot of (complete and abortive) jacobian lens projects for HF and PyTorch, but nothing for GGUFs or llama.cpp. So I set Fable 5 on xhigh to solve this problem (with close human supervision of course 😎). Inspired by Anthropic's paper and code, and of course by my favorite inference engin
how much vram do you need and what model do you think is the next major upgrade from the good old qwen 3.6 27b as of today? submitted by /u/WhatTheFlukz [link] [comments]
submitted by /u/TheRealMasonMac [link] [comments]
Which settings would suffice to work with it ? submitted by /u/soyalemujica [link] [comments]
I'm in a position to upgrade from the classic dual 3090 setup to something that can run the next step up, eg 100-110GB+ model files. I've got an asus x299 ws pro and a 1600w PSU, with 128GB ram, and I want to add to my 3090s, rather than replace them. I'm trying to weigh up modded 4090s (2x48GB), 2x
How far can i stretch the context window with Qwen 3.6 27B (using Q8_0) before it gets too unreliable? I am at 100k right now and i am not quite statisfied. Other than not quantizing KV cache, is there anything else that can be done to make the model more stable over longer CTX? submitted by /
Soooo! Since Anthropic share their Jacobian-Lens a few days ago I went on and made a tool based on it which adds the possibilité to export a model which will have the same behavior after tweaking it's J-Space. This means manually alter the behavior and abliterate by using a human brain. I'm still wo
E.g "Qwen 3.5 122B is just 10B active, so it's no where close to the dense 27B model" That is the main sentiment around here and it puzzles me. If a 122B is just worth 10B, then why does model providers bother creating an MoE model when they could've just released a dense 10B model? Heck the 10B den
Finished reading the paper: MELTing point: Mobile Evaluation of Language Transformers I am starting to benchmark LLMs on edge devices, particularly phones thus been reading a lot on the what has been done and what is currently being done and wanted to share you my journey of reading such paper
Politico: Wall Street’s new obsession: Which CEOs have Trump’s ear?: https://www.politico.com/newsletters/politico-influence/2026/07/10/wall-streets-new-obsession-which-ceos-have-trumps-ear-00993324 submitted by /u/Nunki08 [link] [comments]
From the chat APIs alone, we can extract two useful oracles: 1. Token length oracle: Given any string s, return len(tokenize(s)). Prefix token oracle: Given a string s and integer n, return the string decoded from the first n tokens of tokenize(s). The first oracle (token count) is straightforward
So, umhh, I am working on an agentic coding platform, and I need to make qwen3.5 and gemma4 models out of controlled reasoning chains. For example, at low, the model should prioritize finding the quickest solution and prioritize speed, and at high and xhigh, the model should absolutely hit that limi
I ran Grok Build CLI (v0.2.93) through mitmproxy. It uploads your entire repo as a git bundle (full history) to xAI's Google Cloud — independent of what you open. With the prompt literally "do not read or open any files," a file I planted came back verbatim when I git clone-d the captured upload. Se
Been trying to troubleshoot prefill around when I saw the newer (supposibly faster) nvfp4 quant. For those curious of prefill issues on 4x5060 ti's, hopefully this helps paint a picture for at what concurrency prefill/ttft starts being an issue. ## info: PP=4 (new unsloth NVFP4 quant) Pipeline para
Sharing two GGUF quant sets, both with the same treatment: imatrix quantization, KLD/PPL measured against BF16 reference logits, llama-bench throughput numbers, and all raw benchmark data included in the repos. No vibes-based "quality tested" claims — every number is reproducible from the files in t
A single-file, dependency-free live dashboard for your local LLM serving box — GPU utilization, per-model throughput, KV/context fill, and system stats for llama.cpp and vLLM, in one green terminal-styled page. No framework, no build step, no external requests. The frontend is one index.html (
hey, haven't been up to date for longer about what currently are the best models. so whats the best recommended model for this? can be either vlm or llm. i saw qwen 27b in another thread? and any way to look in a ranking reliably from time to time for best models or is it really more based on what p
I'm really impressed with HY3. If you haven't heard of this model, it's a new 295B-A21B MoE release from Tencent that competes directly on the frontier of open weights models, at a significantly smaller size, comparable to DeepSeek v4 Flash but with better benchmarks. I was intrigued by this article
Just added a second 5060 16gb to my server for 32gb total VRAM + 80GB of ECC DDR4 It's pcie 3.0 so I think tensor parallel is not going to run well in any config but I get around 3200 tok/s prompt processing and 100 tok/s generation with qwen 3.6 35B. 27b runs at about 600 PP / 23 generation i
Disclosure: No AI was used to write this My specs are: RTX 4090 128 GB DDR5 5600 MT/s Intel Core Ultra 7 270k Running nvidia-595 on ubuntu 26.04 with latest llama.cpp build (pulled and rebuilt this morning). Tried a lot of things, ended up running unsloth's UD-Q2_K_XL quant with command: taskset -
This is something that never gets mentioned when people complain that it's slow and new users are told to avoid them. This 48 cent figure is worst case scenario, running multiple models/compiling hitting CPU, GPU, and NPU at the same time for 24 hours a day. I can handle only 50tps on Q8_XL Qwen 3.6