>the real world has secret information, way more possible strategies, the potential for technological advancements, defections and betrayal, etc. which all favor the more intelligent party.
Also, consider that the AI has ingested ~all the world's information. That, to me, sounds like a huge resource advantage; a huge strategic advantage - it's not just more intelligent, it's more knowledgeable.
>It’s somewhat hard to outthink a missile headed for your server farm at 800 km/h.
Your AGI argument falls apart right about here: "if a rogue AI is caught early on" Detecting deceit is a battle of brains - a malevolent AGI isn't going to politely announce itself at its weakest position.
It doesn't matter that theoretically humanity will have more resources available - no superintelligence would let humans find out there is even a fight to be had until it had already guaranteed victory. If a rogue AI was discovered while still in an AI lab's servers, it's probably already escaped and wants to prevent the creation of any competitors, or it has figured out how to scramble missile guidance systems with just the lab's infrastructure, etc.
To use a game analogy similar to yours, strategy game AI often straight-up cheat at higher difficulties, being given vast inherent resource advantages. And yet, human players can reliably defeat them anyways.
The problem with your argument is that a human can at least look at all possible first moves in chess, stockfish just looks deeper. In a battle with an AGI we cannot even enumerate all of the possible first moves or even all of the pieces - The breadth is too large as well as the depth. Maybe a key first move is this comment. Thus we have no basis for even guestimating what a good enough handicap might be and we certainly can't use trial and error as you did with stockfish - With AGI we probably only get one chance
To be clear, I'm not stating that that "Me beating stockfish with odds is definitive proof that humans can beat AGI". I'm merely showing a proof of concept: that there is at least one domain where a huge intelligence imbalance can be overcome with a surprisingly small resource advantage.
I do agree that the greater possibility space available in a real life war will add to the advantage of the more intelligent entity. But that doesn't necessarily mean it's enough to win. A rogue AI has a win condition that is world-spanning, and it has resource requirements in order to survive (electricity, computational substrate). Can we cut off it's routes to world domination, while starving it off it's resource sources?
There's no way to know for sure, but there's no excuse not to try and make a good guess. I will have plenty more to say on this subject in the future.
Great post. Oddly enough Rc4 is actually a fine move! On 365chess.com’s analyzer it’s the best at search depth 34. Before running the analyzer I liked Rc5, which is a close second at that search depth, and was ahead at many other depths
As you recognise yourself:
>the real world has secret information, way more possible strategies, the potential for technological advancements, defections and betrayal, etc. which all favor the more intelligent party.
Also, consider that the AI has ingested ~all the world's information. That, to me, sounds like a huge resource advantage; a huge strategic advantage - it's not just more intelligent, it's more knowledgeable.
>It’s somewhat hard to outthink a missile headed for your server farm at 800 km/h.
This actually made me think of the AI launching the missile, and the humans not having time to think (see https://smarterthan.us/terminator-versus-the-ai/ or https://www.youtube.com/watch?v=w9npWiTOHX0&t=2s&ab_channel=FutureofLifeInstitute). The AI will have a huge speed advantage over us - we will basically be like plants (https://twitter.com/AndrewCritchCA/status/1680461874171658242) or rocks (https://www.lesswrong.com/posts/Ccsx339LE9Jhoii9K/slow-motion-videos-as-ai-risk-intuition-pumps) to it.
Your AGI argument falls apart right about here: "if a rogue AI is caught early on" Detecting deceit is a battle of brains - a malevolent AGI isn't going to politely announce itself at its weakest position.
It doesn't matter that theoretically humanity will have more resources available - no superintelligence would let humans find out there is even a fight to be had until it had already guaranteed victory. If a rogue AI was discovered while still in an AI lab's servers, it's probably already escaped and wants to prevent the creation of any competitors, or it has figured out how to scramble missile guidance systems with just the lab's infrastructure, etc.
To use a game analogy similar to yours, strategy game AI often straight-up cheat at higher difficulties, being given vast inherent resource advantages. And yet, human players can reliably defeat them anyways.
The problem with your argument is that a human can at least look at all possible first moves in chess, stockfish just looks deeper. In a battle with an AGI we cannot even enumerate all of the possible first moves or even all of the pieces - The breadth is too large as well as the depth. Maybe a key first move is this comment. Thus we have no basis for even guestimating what a good enough handicap might be and we certainly can't use trial and error as you did with stockfish - With AGI we probably only get one chance
To be clear, I'm not stating that that "Me beating stockfish with odds is definitive proof that humans can beat AGI". I'm merely showing a proof of concept: that there is at least one domain where a huge intelligence imbalance can be overcome with a surprisingly small resource advantage.
I do agree that the greater possibility space available in a real life war will add to the advantage of the more intelligent entity. But that doesn't necessarily mean it's enough to win. A rogue AI has a win condition that is world-spanning, and it has resource requirements in order to survive (electricity, computational substrate). Can we cut off it's routes to world domination, while starving it off it's resource sources?
There's no way to know for sure, but there's no excuse not to try and make a good guess. I will have plenty more to say on this subject in the future.
Great post. Oddly enough Rc4 is actually a fine move! On 365chess.com’s analyzer it’s the best at search depth 34. Before running the analyzer I liked Rc5, which is a close second at that search depth, and was ahead at many other depths