I think it's worth observing that at the time of Nick Bostrom's 2014 book Superintelligence the success of NN algorithms had not yet broken out in the form of Google's DeepDream. Eliezer Yudkowsky claims this as a success i.e. "progress in GAI has occurred even faster than I have anticipated" but I think it goes to your point that making bets on technology is not reliable. Even after we narrowed it down to NN (or appear to have!), the impact of just scaling up was not appreciated until OpenAI's ChatGPT.
So what it will take to make AI general, what it will take to overcome data limitations, what it will take to make AI agentic, and what it will take to control it I think are all bets that I don't think we can make reliably.
Well, first and foremost, do you want to bet with me about this?
I think I have more knowledge (though of course not certainty), than it sounds like you think I do. I could give some arguments for the plausibility of my view, but I think the virtuous thing to do is stick my neck out and bet on beliefs I might not be able to articulate or persuade you of?
I predict that by 2030, unless regulation curtails AI development in the meantime, there will be major technological advances in felids outside of software, that were produced autonomously by an AI or groups of AIs without human assistance (except perhaps in setting the system running, via prompts or instructions that are less than 50 pages in length).
Admittedly, we have a lot more info than we did in 2023, and your view might be different now.
> To be valid, this argument would also need an estimate of the distance to AGI, and no one has ever provided a good one.
I have at least a partial answer to this question (which itself imports some assumptions that you might not buy, alas). I don't claim that the following is a knock down argument, but it is the point of departure that I use for modeling.
Almost all the tasks that workers at the AGI companies do, to develop AI, take them less than a month (in the sense that it's unusual if they're working on the same thing two months in a row). Therefore, I think "the AIs have a one-month time horizon at software and AI research tasks" marks more or less the point where the AI labs could remove humans from the loop of AI development, and and we get a software-only intelligence explosion happening at computer speed.
Extrapolating the METR chart, we'll hit that point around April of 2027.
Whether AGI will follow in the following year depends on both the dynamics of the software-only intelligence explosion (which in turn depends on your model of the current AI minds, and what exactly is happening as they appear to get more capable) and what exactly your standards for AGI are.
I'll post a longer summary of my forecast for the kind of advanced AI this will lead to sometime in the next week, and how this differs from the 2015-LessWrong projection of what an AGI looks like. But I expect the result of that process to be sufficient to autonomously produce innovations in biotech and physical engineering, just as we are currently seeing long standing "interesting" mathematical conjectures fall to the AIs.
It maybe that one-month time horizons are too short for the RSI loop to really take off. Perhaps one year time horizons are necessary before you can automate everything that the research and development arms of an AGI company does. In which case, humans are removed from the loop (again, extrapolating the METR chart) in July 2028.
There are obviously many qualms and criticisms one could have of this model. As noted, I don't consider this a knock-down argument. But I think this gives me a solid enough grasp that I think it's more likely than not that I'll see AIs inventing new technologies by 2030 or earlier.
Some selected ways that this model might not hold up and why I'm not compelled by them.
Perhaps this extrapolation is missing the point, because METR is measuring skill on tasks that humans know how to complete, and not truly inventive thinking of the sort that human geniuses do. To which I would point out that AI research as practiced today doesn't seem to depend on genius insights; it seems to mostly involve repeatedly trying the next 10 obvious things to try, empirically, and doubling down on the ones that seem to work.
Perhaps that methodology is not adequate to discover "true AGI". I think that's possible, but not particularly relevant, because it's very likely enough to develop Strategically Superhuman AI that can upset the wold's apple cart.
Perhaps the exponential time horizon trendline that METR is trying to measure will stall out. I can't know that it won't. But I'll observe that we've gone through 13 time horizon doublings since gpt-2, and there are only 3.5 left until we reach one month time horizons (and 7 until we reach one year time horizons). There's just not that much time left, measured in doublings, for the trend to stall out, before we get into the Intelligence Explosion.
Perhaps the AIs will be able to take over AI research, but that won't accelerate progress towards AIs that can develop new technologies, or towards strategically superhuman AI, for some reason, and it will take many years of AIs doing AI research and developing new AIs before they can outcompete humans at invention and seizing and maintaining power? To the extent that progress is driven by compute build-out this seems plausible, but it currently seems like progress is driven about 50% by expansions in compute and 50% by algorithmic progress.
Perhaps you think that progress will be bottlenecked on real-world data collection and experimentation? I also expect this in some domains, but I don't think this actually changes the analysis much.
In general, yes, I’ve previously bet about the rate of AI progress with Doomers who objected to my reasoning here (I won), and I’m happy to do it again.
On this specific bet, no, when I wrote this Alphafold had already fulfilled your condition by making advances in drug technology which have since won a Nobel Prize. And perhaps you don't consider that a "major advance", but if so then you'll need a definition precise enough that it outperforms "defer to the Nobel Committee".
Vague resolution criteria like "major advance" mostly come down to whether the judge is feeling the AGI vibes. We'll get more mileage out of whether it solves a specific problem called in advance, e.g. "By 20XX, will AI be autonomously operating a car factory without human assistance (except perhaps in setting the system running, via prompts or instructions that are less than 50 pages in length)".
I think it's worth observing that at the time of Nick Bostrom's 2014 book Superintelligence the success of NN algorithms had not yet broken out in the form of Google's DeepDream. Eliezer Yudkowsky claims this as a success i.e. "progress in GAI has occurred even faster than I have anticipated" but I think it goes to your point that making bets on technology is not reliable. Even after we narrowed it down to NN (or appear to have!), the impact of just scaling up was not appreciated until OpenAI's ChatGPT.
So what it will take to make AI general, what it will take to overcome data limitations, what it will take to make AI agentic, and what it will take to control it I think are all bets that I don't think we can make reliably.
Well, first and foremost, do you want to bet with me about this?
I think I have more knowledge (though of course not certainty), than it sounds like you think I do. I could give some arguments for the plausibility of my view, but I think the virtuous thing to do is stick my neck out and bet on beliefs I might not be able to articulate or persuade you of?
I predict that by 2030, unless regulation curtails AI development in the meantime, there will be major technological advances in felids outside of software, that were produced autonomously by an AI or groups of AIs without human assistance (except perhaps in setting the system running, via prompts or instructions that are less than 50 pages in length).
Admittedly, we have a lot more info than we did in 2023, and your view might be different now.
> To be valid, this argument would also need an estimate of the distance to AGI, and no one has ever provided a good one.
I have at least a partial answer to this question (which itself imports some assumptions that you might not buy, alas). I don't claim that the following is a knock down argument, but it is the point of departure that I use for modeling.
Almost all the tasks that workers at the AGI companies do, to develop AI, take them less than a month (in the sense that it's unusual if they're working on the same thing two months in a row). Therefore, I think "the AIs have a one-month time horizon at software and AI research tasks" marks more or less the point where the AI labs could remove humans from the loop of AI development, and and we get a software-only intelligence explosion happening at computer speed.
Extrapolating the METR chart, we'll hit that point around April of 2027.
Whether AGI will follow in the following year depends on both the dynamics of the software-only intelligence explosion (which in turn depends on your model of the current AI minds, and what exactly is happening as they appear to get more capable) and what exactly your standards for AGI are.
I'll post a longer summary of my forecast for the kind of advanced AI this will lead to sometime in the next week, and how this differs from the 2015-LessWrong projection of what an AGI looks like. But I expect the result of that process to be sufficient to autonomously produce innovations in biotech and physical engineering, just as we are currently seeing long standing "interesting" mathematical conjectures fall to the AIs.
It maybe that one-month time horizons are too short for the RSI loop to really take off. Perhaps one year time horizons are necessary before you can automate everything that the research and development arms of an AGI company does. In which case, humans are removed from the loop (again, extrapolating the METR chart) in July 2028.
There are obviously many qualms and criticisms one could have of this model. As noted, I don't consider this a knock-down argument. But I think this gives me a solid enough grasp that I think it's more likely than not that I'll see AIs inventing new technologies by 2030 or earlier.
Some selected ways that this model might not hold up and why I'm not compelled by them.
Perhaps this extrapolation is missing the point, because METR is measuring skill on tasks that humans know how to complete, and not truly inventive thinking of the sort that human geniuses do. To which I would point out that AI research as practiced today doesn't seem to depend on genius insights; it seems to mostly involve repeatedly trying the next 10 obvious things to try, empirically, and doubling down on the ones that seem to work.
Perhaps that methodology is not adequate to discover "true AGI". I think that's possible, but not particularly relevant, because it's very likely enough to develop Strategically Superhuman AI that can upset the wold's apple cart.
Perhaps the exponential time horizon trendline that METR is trying to measure will stall out. I can't know that it won't. But I'll observe that we've gone through 13 time horizon doublings since gpt-2, and there are only 3.5 left until we reach one month time horizons (and 7 until we reach one year time horizons). There's just not that much time left, measured in doublings, for the trend to stall out, before we get into the Intelligence Explosion.
Perhaps the AIs will be able to take over AI research, but that won't accelerate progress towards AIs that can develop new technologies, or towards strategically superhuman AI, for some reason, and it will take many years of AIs doing AI research and developing new AIs before they can outcompete humans at invention and seizing and maintaining power? To the extent that progress is driven by compute build-out this seems plausible, but it currently seems like progress is driven about 50% by expansions in compute and 50% by algorithmic progress.
Perhaps you think that progress will be bottlenecked on real-world data collection and experimentation? I also expect this in some domains, but I don't think this actually changes the analysis much.
In general, yes, I’ve previously bet about the rate of AI progress with Doomers who objected to my reasoning here (I won), and I’m happy to do it again.
On this specific bet, no, when I wrote this Alphafold had already fulfilled your condition by making advances in drug technology which have since won a Nobel Prize. And perhaps you don't consider that a "major advance", but if so then you'll need a definition precise enough that it outperforms "defer to the Nobel Committee".
Vague resolution criteria like "major advance" mostly come down to whether the judge is feeling the AGI vibes. We'll get more mileage out of whether it solves a specific problem called in advance, e.g. "By 20XX, will AI be autonomously operating a car factory without human assistance (except perhaps in setting the system running, via prompts or instructions that are less than 50 pages in length)".
DM me if you want to work out terms.