> This is one reason the open-model threat is not as large as it appears: frontier models don’t merely outperform older models; they are rapidly becoming cheaper to run at any given level of performance.
This really misses the point. The “threat” (to the profits of big AI companies) is that competition will drive prices down in a race to the bottom. It doesn’t really “help” (drive the profits for big AI companies) if that competition comes from open-ish models or from competing big AI companies or anywhere else.
Why are people writing critical infrastructure in [any language]?
In reality, the tools this article is referring to were written in javascript because they could be and it was the best tool for the job (according to the people who matter: the people who did the work).
If someone wants to rewrite them for speed and/or to chase the next shiny language, that's fine with me, but let's not kid ourselves that there was something wrong with using javascript in the first place.
Using the right abstractions (including the right number of abstractions) and the right data structures and algorithms is always going to trump the constants language choice can optimize.
> Using the right abstractions (including the right number of abstractions) and the right data structures and algorithms is always going to trump the constants language choice can optimize.
I was with you until this paragraph, which is straight up incorrect. Language choice absolutely matters if your goal is performance.
The most obvious counterexample is that the TypeScript team pretty much did a direct port from TS to Golang, without significantly rewriting their core algorithms or changing their data structures, and it sped up the TS compiler by 10X, which is an enormous performance improvement that would be silly to dismiss as just a "constant." This is because JS is a bad choice of language for implementing a fast compiler, primarily due to its poor support for multithreading.
How fast could the TS compiler have been made without changing languages? We don’t know, they didn’t do that. It sounds like the main increase was from parallelism. While not exactly ergonomic, you can accomplish that with worker threads in js, and they are a good fit for a tool like the typescript compiler. They chose to change languages and parallelize, which is fine, but a lot of that performance was on the table for javascript as well.
I still think that rewrite was ill-advised. Yes they got a 10x speedup, but now it's unclear whether the JS tooling ecosystem lives in Go or Rust or JS.
Given that we've now got a three way language schism, the story for sharing data between processes isn't really solved. Data can move nicely within Go, but not between Go and Rust or JS. Tools like ts-morph also got left behind with no replacement.
I think that in retrospect the power of the JS ecosystem was all about plugins. It's plugins that make ESLint such a powerful platform that it still can't be replaced.
From the perspective of making a efficient platform for running plugins, JS actually is the most theoretically efficient implementation language! That's why, for example, VSCode put a bunch of work into moving its core state from C++ to JS, a change which brought them big perf wins and helped them beat out Atom: https://code.visualstudio.com/blogs/2018/03/23/text-buffer-r...
What do you mean when you say the "story for sharing data between processes isn't really solved"? My understanding of this is mostly academic because I don't personally mix languages all that often, but aren't there many established ways of communication between languages. You have IPC, unix sockets, FFI, protobufs, and other serialising/deserialisation formats. Not to mention the classic (but probably slightly overkill) move of spinning up a local HTTP server and using REST endpoints.
Yes, all those ways exist. I was more saying that if copying between structures processes using serialization is all you need, the JS ecosystem already has that.
The thing is if you need to do big expensive copies on the input or output it forces you towards a batch processing model, which is the opposite of incremental computation.
It's not the M5 Ultra itself, but the M7s or M9s that will do the damage.
99% of people will use whatever AI is free. The sophisticated, heavy users that are willing and able to pay a lot of money the ones that will be interested in controlling their inference bills.
Today, the sweet spot where an M5 Ultra makes sense is tiny. But we might expect that to grow a lot.
Even if you could get a frontier model, you would not be able to run it on any Mac. So speculating on what M7 or M9 will achieve in 5 years (if we even still exist) seems pointless.
True, but do you need a 10,000 dollar machine to do so?
I would love to find an excuse to buy a 10,000 dollar machine! But I cant find one yet. My current cloud bill is in excess of 400 USD per month. Just can't achieve frontier model capabilities locally.
Allocation happens. You can leave it to the compiler, the runtime, or let the code control it.
The automatic solutions are usually pretty good and usually the right place to start. But if performance is a priority, you want options.
BTW, “per-frame arena” is part of a general pattern of a repeated interval of work doing significant allocation. This is really common in software of all kinds… servers that process requests (like web servers and database servers) and typical command line tools.
Way back in the aughties I saw an interesting analysis (on a now defunct blog) indicating that under typical usage C# implementations tend to outperform C++ implementations in long-running business applications. The supposed reason was that C#’s compacting GC keeps the cost of new allocations fairly constant. By contrast, in C++ under typical use every new allocation requires probing for a sufficiently large block of free memory in an increasingly fragmented heap.
I haven’t tried to replicate this for myself. And, even assuming for the sake of argument that it was definitely true back then, a lot can happen in 20 years. But still, it does speak to wanting options when performance really is critical.
Passkeys are somehow 100% seamless for me, except for two services that have somehow screwed up the implementation: Google, and Okta. In Google's case, they have made it so that if you _ever_ add a passkey to their password manager, it somehow refuses to work with a third-party password manager going forward. In Okta's case, the admin policy UI makes it next to impossible to have passkey-only login without their Fastpass app or an oldschool password login.
> How much will you love them when you're locked out of something you can't do without?
I certainly wouldn't be happy if this did happen, but it would be my stupidity (or the stupidity of those who implemented it on the service/platform where the problem occurred) which I would blame more than the fault of the tool. I do keep all passkeys in Bitwarden now so that's something at least.
Security benefit appears strongly overstated. Compared to using a password manager (including generated secure passwords for each account), there isn't a significant security benefit. passkeys require using a manager as well, so there isn't much point.
> ...but it would be my stupidity...
All people are stupid sometimes. A security flow that doesn't account for this very well isn't a very good security flow.
(Related: everyone tends to be unlucky eventually, and, unfortunately, everyone becomes incapacitated/dies eventually. Security flows need to account for these as well. Not to mention that in my experience, no software company continues to offer a quality service at a reasonable price forever. You're lucky to get 10 years. Having access to my accounts tied to a single piece of software is likely to become a big pain at some point.)
I think this is probably one of the reasons people have very different passkey experiences. Hardware passkeys are great and software ones appear to be less great.
the legit uses don't need to be surreptitious.
reply