I think the question of technical possibility answers differently from the question of organizational feasibility. Starlink is optimizing costs because they have a financial incentive to do so. The less each satellite costs, and costs to maintain, the more income they make. I fear incentives are not so direct in governmental orgs. Outside contractors performing most of the work have outsized influence, and incentive to inflate costs. Budgetary decision makers all want to see investment in their own districts.
I think Starlink and cubesats have definitely proven the approach can reduce costs.
The engineering does not seem as challenging to me as figuring out how to get such a policy approved and pushed through all the administrative and decision making layers without watering down and budgetary inflation.
Maybe if we launched 30 Starships carrying 30 rovers each. With only 30 units, the prices would likely be similar to current commercial satellites, which are already built on standardized platforms.
I'd go even more radically low cost, and kit the first lander out with wifi, and a half dozen each of every commercially available robot dog, humanoid, and wheeled or tracked robot of any interest. Rumble in the Mars Robodome. Bit of publicity for the existing manufacturers who may be willing to pay for the privilege, and who will undoubtedly learn valuable things about how their designs fail and can be improved. And you figure out in one stroke if there's a COTS platform you can buy for $30k that gets you 80% of what you need. If any of them function for any period of time, you use them to prepare the space for future landings. Rinse, repeat. After a few iterations, you have a half dozen manufacturers with commercially produced platforms with a few strategic modifications to which you can bolt your science instruments.
All I can think about now is a bunch of e-waste littering Mars. At some point, other visitors will show up to investigate Mars and come to the conclusion that the Earthlings are just so trashy their trash even started filling another planet.
Even before AI we were trending toward an economy with very few entry level jobs anywhere. AI is supercharging this, and I can easily now imagine a future where there's basically zero (good) jobs for anyone who isn't an expert.
So how do we get the next generation of senior professionals then?
This, I think, is a civilization-scale problem we will need to confront in the next 20-ish years.
That’s what happened in US manufacturing. It’s now 20-30 years down the line and the economically important US manufacturing sectors, like defense, are scrambling to hold onto institutional process and trade knowledge as the final remnants of the old industrial system retire. We all know how useful someone with a fresh comp sci degree and no experience is in a real dev environment is. Imagine the leads showing them the ropes graduated two years prior?
The skills to drive AI are much the same as they were for business analysts in the old days. Domain knowledge, insight into user requirements, knowledge of modern UI paradigms, ability to write detailed, consistent requirements docs/ design docs.
There is a deep bench in the industry of these senior-ish people. Many have left to become baristas, dive instructors or hobby farmers, scarred by the pain of building large complex software systems using human labour and absurd Agile rituals. But they could slot straight back in, it's like riding a bike.
AI development is the new waterfall. Its just that the lower level that the BA hands off to, which used to be roomfuls of devs, is now a superhuman who can implement those designs at lighting speed and come back hungrily for more.
But if by senior you mean someone who is just a junior with more experience, someone who's not really in touch with the business's needs and who just works off stuff fed to them by PMs or the like - then yeah, agreed.
Another thing is that orgs will want to keep around somebody who 1) can be held responsible when things go wrong 2) has a reasonable understanding of the system that's their domain end-to-end 3) can reliably diagnose failures and fix them, regardless of how the code was written.
That's not a role that can reasonably be filled by a junior and management armed with agents doesn't really fit either.
> But they could slot straight back in, it's like riding a bike.
Yeah, not quite. There's a ton to learn about how to control an LLM while it's writing code, and even more to learn about how to manage a set of agents.
9 months ago I was comparing it to running a dev team, but now it has changed and there are practices and processes that are unique to managing agents.
e.g. a team of software devs have the self-awareness to not take a single marginally-relevant point in a spec document and spend 20% of their team effort to build an entire subsystem to meet it without checking. The process and rituals that we used to use to make sure that a software dev team was making progress and would hit the project deadline are now largely useless, but we need new ones to make sure the agents are not heading off into unnecessary rabbit holes.
I'm not saying those ex-seniors would not be able to get back in the saddle. I'm just saying there's been more change in the last 9 months than there has in the last 30 years, so it may take a period of adjustment.
But I don't think the things that burned them out before will have changed. Dealing with non-tech executives was always the worst part of the job, and LLMs can't help with that, and are even making it worse ("ChatGPT says this should only take a couple a hours and you're using Typescript instead of Go! Why? Fix it!").
No worries, obviously the AI will cure aging so we will be able to run civilisation with the current experts forever.
...at least, until they accumulate enough stocks to retire on.
...however, for each one that retires and cannot be replaced due to the no-upcoming-juniors problem, evidently, the value of the companies they own goes down. So they won't be able to retire, or at least not all of them. So there's no problem!
Education needs to adapt. Currently CS graduates barely compete with Gemma-4 26b in programming while seniors are feeding tons of expertise at armies of frontier AI agents. The market now needs junior employees good at "using AI like an expert". Once employers see that junior armed with AI can replace senior at half the salary, we're back where we were 10 years ago.
Senior (now):
- AI build me an app that uses a graph DB that will deploy in the edge. Batch DB requests to keep costs low. Oh, use good cache + memoization to reduce hits. Write great tests, some logging and implement hot-reloading for a great dev-to-ops workflow that is also resilient. Man, before I had to do this all by hand, this is a godsend!
Junior (now):
- AI build app please.
Email from boss: Junior! Whatever it is you've deployed, it's making 1M reqs/min to the DB and ate up our quota 2h ago. Shut it down now!
Junior (future):
- AI1 design for cost. AI2 eval for performance. AI3 align for business goals. AI4 criticize AI1 and AI2 using top standards. AI5 set up a competing swarm. AI6 here's a budget, spend on real users and action on telemetry and their feedback.
> Junior (future): - AI1 design for cost. AI2 eval for performance. AI3 align for business goals. AI4 criticize AI1 and AI2 using top standards. AI5 set up a competing swarm. AI6 here's a budget, spend on real users and action on telemetry and their feedback.
Ugh, please no, why isn't these instructions in the system prompt of the harness if they are so valuable to the junior's work...?
I assume those are non-batched. I think the M series GPU can do 4X to 8X depending on model quant, which means if you can batch queries you'll get almost 4X to 8X performance.
I'm deeply skeptical of AI doomer stuff, but I've been struggling to exactly articulate why. I can think of it in my brain and can try to explain but so far none of my explanations have seemed fully coherent to me.
I studied biology in college. I actually did this because I was interested in AI, and already knew a ton of CS and programming. I was already coding in C and using Linux and writing fairly complex stuff. I figured studying the only actual example of a truly intelligent system (especially at the time, early 2000s) was the best way to understand intelligence.
Didn't end up going into AI as my day job, but I've kept up with it all and worked through the math and played with different types (genetic, neural nets, transformers/LLMs, CA / alife, etc.).
I think the root of my skepticism is in what I learned studying biology about how life, adaptation, learning, and evolution actually work in the real world.
I'll try to TL;DR it: I think the doom crowd (and also some of the AI singularity crowd) has a "mystical" view of intelligence. Intelligence is magic pixie dust, and intelligence beyond human is fully unknown and likely has magic superpowers.
The deeper underlying thing seems to be some kind of mathematical mysticism.
Real intelligence is ubiquitous in nature. Trees, grass, bugs, and even bacteria are intelligent. It's also domain specific and limited in capability to what its domain, available training data, and available compute allows.
It's not context-free limit-free magic. It's not special. It's an adaptation like any other.
Yes, the doom crowd thinking comes directly from the Rationalist community, which takes a very religious view of AGI and intelligence. That sort of merged with the Effective Altruism crowd, resulting in that whole idea that we have to reach AGI as fast as possible and contribute literally all our resources to it, because it’s the only rational way to save as much suffering and lives as possible. Sounds incredibly stupid but it’s how AI leaders justify working on something they believe will be a machine god that might doom humanity.
Both OpenAI and Anthropic are from that world, where Rationalist doesn’t mean you’re a rational human being, but instead that it is rational to believe that AGI will be created, thus that they should be the ones creating it. It’s a doomsday cult.
If you start looking at the links between prominent rationalists and effective altruists you see how that whole doom prophecy came about. It’s also how they got so much capital in the early days, SBF literally invested into Anthropic as part of his Effective Altruism effort foundation, and both Amodei siblings are deeply into that whole world
It's amazing that apparently the best way to save as much suffering and future lives as possible involves giving them obscene wealth and ignoring the very real problems we are dealing with today.
Once again, I've said it elsewhere here, I think we could have nipped this thing in the bud if when the first "effective altruist" started talking about the EV of saving billions of "future lives" vs dealing with today's real problems, someone looked at him, cocked their head to the side, and said "That's fucking ****".
All of those magical future people do not get to live if we bake the Earth today. The best way to take care of future people is to take care of people today. And if they were truly serious about taking care of future people, that's what they would be doing. But it's not, they're just collecting resources. I don't know how the fuck Sam Altman owning a Koenigsegg Regera today helps save 20 quintillion lives in 40,000AD. Deeply unserious people.
I think Altman is closer to a sociopath who is taking advantage of the situation, he’s the only one I cannot see as a true believer (and investors like Andreessen who are just here for the money). I believe he will just say anything to gain power and influence. But yeah, unfortunately mixing EA/rationalists + SF monoculture + infinite money + a media ecosystem that acts more as a PR department resulted in one of the worst cult-like phenomenon possible.
Yeah I read some of that stuff, like some of "the sequences," and I have never been impressed by it. It's very weirdly dogmatic and full of magical thinking.
I still can't believe anyone ever took Roko's Basilisk seriously, but people did... to the point of freaking out about it.
The groups themselves are a little cultish too, or a lot cultish depending on which one you're talking about. Some of them are full-on cults.
> I still can't believe anyone ever took Roko's Basilisk seriously, but people did... to the point of freaking out about it.
If you've spent much time in US Christian circles you've probably encountered people suffering from apocalyptic thinking. The Roko's Basilisk folks seem like the atheist version of the same crowd. It's hard to take them seriously when they're always seeing portents of the end times, and their thinking is clearly clouded by anxiety and fear and not based in reality.
Interestingly I think I am coming at it from exactly the opposite angle. Viruses and bacteria aren't intelligent at all (in my opinion), and don't need to be to wreak havoc. Whether or not LLMs are "intelligent" is one for the philosophers, but they are absolutely able to exploit electronic systems today if their starting state has been set with that as a "goal". Given that alarmingly few physical control systems are really air-gapped, and also that I can today set an agent off to find a voice, clone a voice, and make a phone call, I think the only protection we had today is that the most capable models have been trained not to, and nobody's kicked off a sufficiently malicious task yet.
Viruses are intelligent. Not much, but their replication rate gives them a very rapid rate of evolutionary learning. Evolution is intelligent, but in a very different way from neural networks.
Bacteria and up are full of gene regulatory networks that are absolutely intelligent and share some features with neural networks. Bacteria learn, cooperate, signal, react, seek goals, and that's in addition to evolutionary learning which is also fast for them given their reproduction rate.
Evolution in nature isn't just zeroth order either. Look into a topic known as "the evolution of evolvability." Evolution does meta-parameter search on itself, more or less, and invents adaptations to improve its ability to adapt. Sexual reproduction is the most significant and largest example but there are others. There's environmentally dependent tuning of mutation rates, genome self-editing that we still don't fully understand (this is also how our immune system works), lots of epigenetic stuff we are still learning about, and so on.
If it's alive it's intelligent. All life is constantly learning.
Interesting, before Yudkowski was patient zero for AI doom he was a nanotechnology "grey goo" doomer. That was even more obviously absurd to me. Grey goo already exists. It's called biology, and it's been trying to maximize its footprint on Earth for 4.5 billion years. The miraculous instant "foom" grey goo thing is magical and likely violates conservation laws.
What gets me is that I haven’t heard a reasonable explanation of exactly what the AI will do besides: nukes and biolabs we already have
& ofc AI is making climate change worse, some of the Amazon datacenters are using fossil fuels to generate a country’s worth of power, but that’s never part of their calculations
Viruses against humans, viruses against livestock as an attack against humanity's food chin, plant diseases, half of humanity feeds on three plants maize, wheat, rice. Attacks against infrastructure like water, internet, electricity.
Which one of these would it be? All of them in parallel? Or something we wouldn't think of. If I were to play chess against Magnus Carlsen, I wouldn't know with which piece he will checkmate me, but I know that I am going to lose.
The more you know about a subject the better you can prompt AI, steer it toward the correct path, and recognize when it hallucinates or strays. Current generation AI is an automated memory-enhancement and thinking-accelerator tool, not a substitute for understanding or something that eliminates the need to think. A "mech suit for your brain" is the best analogy I've heard.
This is why good programmers get better results when vibe coding than non-programmers or poor programmers.
They're pretty basic and hallucinate a lot. There are some hard limits to how good you can get on a model that fits on a phone.
Qwen3 and Gemma level models that run on mid-high end laptops and desktops can be pretty good. Not frontier grade, but shockingly competent for something that runs on a single PC. But the hardware you need to run those fast is at least $1000-$2000. Cheap hardware can run them, but slooooooow.
> Math has built a gated, inaccessible institution which - by design or not - served as a moat.
This has always been my problem with math going all the way back to college. It was obvious to me that the concepts were far less hard than the combination of notation and esoteric jargon with liberal use of symbols and weird letters made them seem. Math felt (and still does feel) "encrypted."
The impression math gave off is of an arcane discipline that uses its arcane-ness as a gatekeeping tactic, intentionally or not, and makes itself intentionally hard for newcomers to learn without being hand-held by members of the guild.
Of course I can say the same about a lot of computing, and I'm old enough to remember efforts to make computing more approachable like easier to learn languages and GUIs being mocked and scoffed at by "real programmers."
I think this is a pretty typical human group behavior.
I think this underlies a lot of AI hate from these communities today. AI makes it easy for outsiders to bash their way into the field with the help of an LLM. Yes, this often results in low-effort "slop," but if used correctly it can also help people climb the learning curve really fast. I've had great luck having an LLM make me some passable "slop" and then explain it and go around with me as we fix and refine it, explaining each step, and as it does so it feels like we are learning together. It's very powerful and I, as the student, can control exactly how the teacher presents the material.
I was able to do this to finally start grasping the math behind LLMs themselves: attention layers, tensors, etc.
A lot of people have a powerful visceral probably instinctive reaction to large numbers of migrants entering their area. "Build the wall!" I suspect this is brain stem stuff going back to evolving under conditions of scarcity where migrants meant less food.
> the concepts were far less hard than the combination of notation and esoteric jargon with liberal use of symbols and weird letters made them seem
I really don't get the hate for mathematical notation here, do you have specific examples? Notation is a necessary tool to express complex ideas compactly so that others can understand them. It allows you to carry out technical proofs that confirm things that are "obvious". Then it uses previously introduced concepts to build new concepts on top of them. This keeps mathematics interconnected, there's value in explaining a concept in terms of already known things rather than just vibing it.
> makes itself intentionally hard
It doesn't make itself intentionally hard, it is just hard. You can simplify some expositions and make concepts easier to understand intuitively but it won't necessarily give you the skills to actually work with them in mathematical practice (which is the point of mathematics as a discipline).
My theory is that people run into the inevitable difficulty of math and rather than look inwards and come to the conclusion they aren't as smart, or special as they once thought, they blame "the system". It can't be that they haven't tried hard enough or aren't smart enough, it's the gated community who hides behind esoteric jargon that stops them from being successful in their mathematical studies.
They believe this despite the fact that there are a million different textbooks, all explaining the same concepts in different ways, or different branches that specify the same structures with unique methods. It's really strange when I read threads like this. I'm not the greatest at math either, but that's because it's a hard thing to be good at.
To learn math is an exercise in humility. You must struggle hard, do hundreds of problems, spend years refining your understanding, redoing proofs, finding counter examples, just to be at a good undergraduate level. Much harder than pretty much all CS classes, except maybe Theory of Computation, which is pretty much math.
reply