I've been using Qwen3.6 35B A3B, and with reasoning turned on, I'd say 2/3 (give or take) of the tokens for a response are thinking tokens. Which at 70+ tps locally, that isn't that awful. I run an 80k context across 4-10 "agents" for my solo TTRPG, where Qwen is the GM, each NPC at a location, the director, and the narrator.
Each turn is about 45-60 seconds to generate all of the various responses. The GM and director have reasoning on, and the NPCs/Location/Narrator do not.
It's a fairly good "engine" for that. I'm not sure how a denser Qwen would do here regarding speed.
I like the tabletop RPG use case, and wanted to say: If your hardware likes it you should check out Gemma 4 for creative DMing use case. I found it to be much better at holding the plotlines and being creative on gaming turns. My experimental case was an audio-only Zork and Gemma 12B and even E4B were pretty good!
This sound very interesting, do you have any resource I could look at?
Me and my son did a very rudimentary (compared to yours) setup to play Paranoia, but this is at another level.
I'll go over my repo, and see if it is hiding any API keys and maybe make it public. The issue I have is it relies on a nuget package that also isn't live (its in my local nuget feed).
I'm not sure what all is needed to make that work for people.
I'm working on something similar. My biggest annoyance is that the overly-helpful LLM was making every die roll succeed. I ended up building some tooling around rolling dice. Also some tooling around character stats and inventory management, so those don't get lost in context compression.
It is on a single 3090, and that seems to be where it averages out. I'll get 85tps on turn 0, but then it settles down to low 70s within a few turns, but holds steady at that.
My issue currently is KV Cache, because I can't keep enough parallel caches running (4 is where I'm at), so TTFT (is that the initialism?) can be long when I have a particularly large scene (basically more than 2 NPCs).
But my harness does let me offload to any OpenAI compatible endpoint, I just prefer local cuz free.
Back in 2023 I started my own C# LLM library for doing tool calls and structured output, and over the years it has morphed bigger and bigger, and that is the backbone of almost all of my LLM-based projects.
I've never released it, but its easy to understand, and simple to add your own tools:
[AIDescription("Get current weather for a location")]
static string GetWeather(
[AIDescription("The city name")] string city,
[AIDescription("The country name")] string country,
[AIDescription("Temperature unit", ["C", "F"])] string unit = "C")
{
// make some API call to a weather API and return a string to the LLM
return $"The weather in {city}, {country} is 22°{unit} and sunny";
}
var chat = client.StartConversation("You are a helpful assistant with access to weather data.");
var response = await chat.SendAsync<string>("What's the weather in London?", GetWeather);
I'm sure plenty of better libraries exist for this now, but in 2023, I don't think any existed in the dotnet ecosystem. I've never released it though, because I've never "finished" it.
Not parent, but I use Goose for my non-handcrafted Qwen use cases, I’m also working on handcrafting as well. Goose was the only harness that didnt bloat context too much with system prompts (like openclaw) and I could get reasonable web search working with Qwen.
Just to play devil’s advocate: you can’t compare Qwen to a (proprietary/closed source) hosted model and deduce that Qwen is overthinking, as Qwen gives you the full reasoning/thinking trace while all the proprietary models now give you only a summary “to prevent distillation”, making it hard to properly compare apples to apples here.
People say Qwen overthinks because they analyzed the thinking traces, and Qwen finds the answer relatively quickly but then second guesses itself multiple times for another 20,000+ tokens. Regardless of what other models do, that's clearly overthinking.
This is most likely because the vast majority of the information the model absorbed during training was in Chinese. As a native Mandarin speaker, I frequently need to convert the prompt into English and output it in English in order to avoid that the model falls back into Chinese reasoning logic.
PS: Switching the thinking process from Chinese to English can also significantly circumvent certain self-censorship mechanisms built into the model.
You can use any message you want, but the model was tested to react reasonably well to the specific token sequence of "\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n</think>.\n\n" (from a Alibaba paper, struggling to find it now)
Since i have tools to prune context and run subagents, i just tell it to do either since both require summarization which is usually what it needs to avoid the long if...then chains
Can you actually have an agent call the Antigravity CLI by the TOS? Feels like the type of thing that would get your account flagged by doing it programmatically.
It wasn't long ago when calling Claude -p would trigger usage billing. I don't know if they backtracked that but even apps like T3 Code were warned that using Claude like that was not going to use the Claude sub.
I think that’s the really important implied question right now. Mostly everyone already has some form of device to play games on. What happens when they start to die off?
It’s frankly embarrassing at this point. I’ve got free access through buying a Pixel phone and it’s not even worth using as it’s a waste of my time. Here’s my experience so far using it for basic sysadmin Linux type stuff.
Gemini 3.1 Pro just feels a generation behind, from when models would miss easy things and make bad assumptions. Its not actively detrimental in bad way but the opportunity cost vs using something like Opus to be productive is large.
Gemini 3.5 Flash is the most annoying model I have ever used. It loves to respond in ALL CAPS like “LOOK AT THAT” for no apparent reason. I realize it’s a flash model but I will give it a basic list of tasks and the output will simply vomit “Now I will X” “Now I will Y” “Now I will Z” over and over again filling my screen with garbage. It’s also no smarter than 3.1 Pro and consumes just as many tokens as 3.1 Pro, it’s really pointless without a newer Pro model in place.
3.1 Pro: while it's coding performance is mediocre, a lot of coding work requires minimum actual thinking. I use it often for light refactoring, boilerplate generation, testcase skeleton generation, code review, language questions ("is there a better way to write this code block?"). I don't have a corporation behind me so costs matter. Considering that I'm a Pro subscriber, it's quite cost effective.
3.5 Flash: excellent model for general (re)search. I use it for everyday tasks with Thinking instead of Flash-Lite. It's a much better version of Google Search for general queries like gaming tips, cooking, day-to-day first aid, tax and investment questions, etc.
Google is clearly aiming for cost-benefit here and considering that it gets bundled with YouTube Plus and Google One at $20/month, it's a killer deal IMHO.
PS: I don't work for Google and don't even like Google very much. But this is a good product.
It was a decent deal ~6-8 months ago. I had been using 3.1 pro almost since release, but it really is feeling old. After using other models more in the past two months though...I really can't go back to 3.1 pro, as I just have to explain my reasoning so damn much to get it on the right path, where as opus or fable just "get it" from the context of the project much better.
Sonnet is roughly the same level as 3.1 pro for me.
This is a great new development, llama.cpp is the ffmpeg of the AI world. The vast majority of "Local AI" tools just take it and put a front end on, delivering little value. Bringing users close to the core project is a big win for everyone.
Props to the author for putting in what looks like ton of work trying to navigate this issue, shame they have to go to these lengths to even have their case considered.
I went to hell and back trying to get PIP/PBP monitors on my 57" g9 ultrawide to work with my M2 pro. ended up having to use a powered hdmi dongle, displaylink cable, and displayport, with 3 virtual monitors via betterdisplay. Allowing resolutions outside of macs limitations setting in BD is what did the trick. I don't envy OP. Having 5120x1440 @ lodpi was the worst, just ever so slightly too fuzzy but perfect UI size but eventually got a steady 10240x2880 @ 120hz with HDR. I literally laughed out loud when I read the title of the thread. Poor guy.
Ah but you see, the challenge is to get a 3-split PBP on an M2 pro on a monitor with a native res of 7680x2160, each one scaled down 33%, working at 120hz with HDR, all hidpi like so:
┌─┐┌────┐┌─┐
│ ││ ││ │
└─┘└────┘└─┘
It creates some wonky math and requires plenty of dock and cable shenanigans and unlocking resolutions above 8k via BD. It's the third "monitor" where it gets tricky with the M2 pro especially at these resolutions.
I hate spending any unnecessary clicks or keyboard shortcuts on getting whats out of my head into the computer. I used yabai before primarily, now using aerospace. Since the monitor is super ultra-wide (57 inches with a very high DPI) the native resolution makes everything ultra small to my eyes. It's the same height as my 34-inch Samsung G5s which are 1440 pixels tall natively, but since this one is 2160, it would have to be 1.5 times larger physically to look decent at native res especially on macOS. The only other option is to scale the UI 1.5x which is where all the problems begin.
I like the three-column separate monitor layout because I have hotkeys, primarily driven by my mouse but also usable keyboard-only where I can easily switch between monitors with `⌘+`` which moves my cursor between them. I can select whichever monitor I want and put my mouse to it, and I can switch to any workspace on any monitor quickly. I also have hotkeys that sync three workspace numbers across monitors, so switching between them switches all AeroSpace workspaces on all three monitors simultaneously. If I have five projects going, I'd have the terminal on the left, Linear and other communication tools on the right in accordion mode with AeroSpace, and I can use my mouse or keyboard exclusively to find exactly what I'm looking for almost as fast as I think of it. I spend zero time on window management or organization now so it makes it thoughtless to use.
If I'm just using the monitor's native resolution there's no real way to do portals — having two apps open as sticky and only switching a portion of the monitor space to a different app while keeping the other sticky. There are hacks you can do with AeroSpace, especially since AeroSpace doesn't use native macOS Spaces, but the three-monitor layout is a much more robust approach in my opinion just a bit of a nightmare to setup. Theres a million little mac annoyances you have to fix.
...And then there is the near-infinite trickle down of apps that rely on apps that rely on arcane configs and so on. This is truly the OS from hell. At least with Windows you know it's going to be garbage so when anything works on any level you are maximally impressed. But I have to spend my weekends isolating window shadow disabling functionality from yabai into it's own binary because I switched to aerospace which requires 'displays have separate spaces' to off, which just so happens to be exactly what yabai requires to be on, to remove window shadows, which is the only use I have left for it.
Just like the excel world championship I would find a macos ricing/window tiling competition equally enthralling. You read articles like the OP and at some point all you can do is laugh because lord (Cook) knows you've cried.
Thanks, it was a good portion of my weekend bashing my head against the keyboard trying to figure out what was going on and if there was a workaround I could use (there isn't that I've found).
The post reminded me how I investigated a similar issue having no idea. Using Claude or GPT to investigate this kind of hardware issue is fast and easy. It gives you next command to try and then next one and you end up with similar summary. I wouldn’t be surprised that author didn’t know anything about displays before this.
The jump from 10 to 20$ a year*(correction) is fine? I’m happy to pay for a quality service I use daily, and price increases are inevitable. Zero issues on my end.
20$/year. And family plan is 48$/year (max. 6 users), meaning 8$/user/year.
If you find 2 people to share the family account with its cheaper than single account.
I don't doubt it, but what were they all doing? The Metaverse had 10k employees on it for multiple years and seemed to almost be a standstill for long periods of time. What do these massive teams do all day?
Have meetings to figure out how to interact with the other 9990 employees. Then try and make the skeleton app left behind by the team of transient engineers who left after 18 months before moving on to their next gig work, before throwing it out and starting again from scratch.
Exactly. What Meta accomplished could have been done by a team of less than 40 mediocre engineers. It’s really just not even worth analyzing the failure. I am in complete awe when I think about how bad the execution of this whole thing was. It doesn’t even feel real.
Actually I would like see a post-mortem that showed where all the money actually went; they somehow spent ~85x of what RSI has raised for Star Citizen, and what they had to show for it was worse than some student projects I've seen.
Were they just piling up cash in the parking lot to set it on fire?
At least part of the funding went to research on hard science related to VR, such as tracking, lenses, CV, 3D mapping etc. And it paid off, IMO Meta has the best hardware and software foundation for delivering VR, and projects like Hyperscape (off-the-shelf, high-fidelity 3D mapping) are stunning.
Whether it was worth it is another question, but I would not be surprised is recycled to power a futuristic AI interface or something similar at some point.
Even within the XR industry, we had no clue where all that money went. During the metaverse debacle, the entire industry stagnated. Once metaverse failed, XR adjacent shops started to fail. There was no hardware or technique innovation shared with the rest of the industry, and at the time the technology was pretty well settled.
Since then we lost all the medium players and it's basically just Facebook, Valve, and Apple.
Big company syndrome has existed for a long time. It’s almost impossible to innovate or move fast with 8 levels of management and bloated codebases. That’s why startups exist.
reply