> "massive upfront investment and large and complex" and therefore predict progress stopping ages ago?
Regulatory and economic barriers are probably the easiest to overcome. But they are an obstacle. All it takes is for public sentiment to turn a bit more hostile towards technology, and progress can stall indefinitely.
> Opening with the recursively improving AGI and then having a section of "areas of promise for step-function improvements" and not mentioning any chance of an AGI breakthrough?
The premise of the article is that the hardware that AGI (or really ASI) would depend on may itself reach diminishing returns. What if progress is severely hampered by the need for one or two more process improvements that we simply can’t eke out?
Even if the algorithms exist, the underlying compute and energy requirements might hit hard ceilings before we reach "recursive improvement."
> How many autoimmune diseases have been cured, ever? Where does this “Probably” come from — the burden of proof very much lies with that probably.
The point isn't that we're there now, or even close. It’s that we likely don’t need a step-function technological breakthrough to get there.
With incremental improvements in CAR-T therapies — particularly those targeting B cells — Lupus is probably a prime candidate for an autoimmune disease that could feasibly be functionally "cured" within the next decade or so (using extensions of existing technology, not new physics).
In fact, one of the strongest counterpoints to the article's thesis is molecular biology, which has a remarkable amount of momentum and a lot of room left to run.
> We might not be that far away from a plausible space elevator.
I haven't seen convincing arguments that current materials can get us there, at least not on Earth. But the moon seems a lot more plausible due to lower gravity and virtually no atmosphere.
But I'd be very happy to be wrong about this.
> Based on what we know today, there isn’t “a” plateau — there are many, and they give way to newer things.
True. But the point is that when a plateau is governed by physical limits (for example, transistor size), further progress depends on a step-function improvement — and there's no guarantee that such an improvement exists.
Steam and coal weren't limited by physics. Which is the same reason why I didn't mention lithium batteries in the article (surely we can move beyond lithium to other chemistries, so the ceiling on what lithium can deliver isn't relevant). But for fields bounded by fundamental constants or quantum effects, there may not necessarily be a successor.
Your strongest case is that CPU design might stop getting faster, but that doesn't automatically mean progress towards AGI must stop.
Smartphones sell 1.2 billion every year. Add in server, laptop, embedded chips, GPUs, TPUS - even if transistor density and process improvements stall soon, the amount of compute power on Earth is vast and increasing rapidly until 'soon' happens, and can still increase rapidly by churning out more compute with the best available process indefinitely after that. You haven't made a case that process improvements are necessary or that they are not going to happen, you've only said that they might be neccessary and might not happen. "All it takes is for public sentiment to turn a bit more hostile towards technology, and progress can stall indefinitely" - again, true in terms of process improvements, not true in general because we could potentially make progress towards AGI with existing compute power by changing how we organise it and what we do with it; at least, you haven't given a good reason why that couldn't happen.
> "In fact, one of the strongest counterpoints to the article's thesis is molecular biology, which has a remarkable amount of momentum and a lot of room left to run."
One of the strongest counterpoints is that human brains exist - there's definitely some way to get to human-equivalent intelligence, on Earth, within the laws of Physics, the energy and temperature constraints which exist here. Handwaving "what if AGI is impossible because of the laws of physics" needs you to make a case why the laws of physics are a blocker, not just state that there are some physical limits to some things sometimes.
Yes transistors are hard to make smaller but that is not the only option - we've developed stacked 3D transistors to pack more into a small volume, hardware acceleration through design for more and more algorithms (compression, video compression, encryption) to better use existing transistor budgets, we've developed better more cache-friendly and SIMD friendly algorithms, chips where part of them can power down to allow more power for other parts without hitting thermal limits. More than one S curve involved, not just one plateau.
The paradigms are in initial conditions. This end state of binary/arbitrary/symbol/stat units are largely irrelevant. They're mindless proxies. They bear no relationship to the initial conditions. They're imagination-emasculated, they show no feel for the reality of real information signaling, merely an acquiescence to leadership expedience in anything arbitrary (like tokens).
Try to see the binary as an impassable ceiling that turns us into craven, greedy, status junkie apes. Mediocrity gone wild. That's the binary, and it's seeped already into language, which is symbolic arbitrariness. We don't know how to confront this because we've never confronted it collectively. There was never a front page image on Time Magazine that stated: Are we arbitrary?
Yet we are, we're the poster child for the extinct and doesn't know it sentient.
Each stage of our steady drive to emasculate signaling in favor of adding value displays this openly. Each stage of taking action and expression and rendering them as symbol, then binary, then as counted token, into pretend intelligence showcases a lunatic drive to double down on folk science using logic as a beard in defiance of scientific reason.
Regulatory and economic barriers are probably the easiest to overcome. But they are an obstacle. All it takes is for public sentiment to turn a bit more hostile towards technology, and progress can stall indefinitely.
> Opening with the recursively improving AGI and then having a section of "areas of promise for step-function improvements" and not mentioning any chance of an AGI breakthrough?
The premise of the article is that the hardware that AGI (or really ASI) would depend on may itself reach diminishing returns. What if progress is severely hampered by the need for one or two more process improvements that we simply can’t eke out?
Even if the algorithms exist, the underlying compute and energy requirements might hit hard ceilings before we reach "recursive improvement."
> How many autoimmune diseases have been cured, ever? Where does this “Probably” come from — the burden of proof very much lies with that probably.
The point isn't that we're there now, or even close. It’s that we likely don’t need a step-function technological breakthrough to get there.
With incremental improvements in CAR-T therapies — particularly those targeting B cells — Lupus is probably a prime candidate for an autoimmune disease that could feasibly be functionally "cured" within the next decade or so (using extensions of existing technology, not new physics).
In fact, one of the strongest counterpoints to the article's thesis is molecular biology, which has a remarkable amount of momentum and a lot of room left to run.
> We might not be that far away from a plausible space elevator.
I haven't seen convincing arguments that current materials can get us there, at least not on Earth. But the moon seems a lot more plausible due to lower gravity and virtually no atmosphere.
But I'd be very happy to be wrong about this.
> Based on what we know today, there isn’t “a” plateau — there are many, and they give way to newer things.
True. But the point is that when a plateau is governed by physical limits (for example, transistor size), further progress depends on a step-function improvement — and there's no guarantee that such an improvement exists.
Steam and coal weren't limited by physics. Which is the same reason why I didn't mention lithium batteries in the article (surely we can move beyond lithium to other chemistries, so the ceiling on what lithium can deliver isn't relevant). But for fields bounded by fundamental constants or quantum effects, there may not necessarily be a successor.