Interesting, I largely agree with your conclusions (if not exactly the arguments that got you there).
- current AI models for the most part have no real agency and so no SIILTBness
- the imaginary hyperintelligent AGIs have agency, but their imagined SIILTBness is a projection of a limited set of human capacities – they have relentless goal-following, but not no empathy (the inborn tendency of humans to partly share each others goals). Basically the model for AGIs are monsters like the xenomorphs from Alien, which in turn are representations of relentless amoral capitalism.
- Alignment research is basically saying, hey, we created these monsters, can we now turn them into housepets so they don't kill us, which is pretty hilarious.
- Part of the fascination with monsters is their partial humanness, or partial SIILTBness in your terms. They are all sort of, "what it is to be an agent with some human traits, magnified and stripped of their compensating tendencies".
- just because AGIs are obviously monsters (that is, projections of humanities fears of aspects of itself) doesn't mean they can't be real dangers as well.
Crossposted from LessWrong for another user who doesn't have a paid subscription to your substack:
My impression from Section 10 is that you think that, if future researchers train embodied AIs with robot bodies, then we CAN wind up with powerful AIs that can do the kinds of things that humans can do, like understand what’s going on, creatively solve problems, take initiative, get stuff done, make plans, pivot when the plans fail, invent new technology, etc. Is that correct?
If so, do you think that (A) nobody will ever make AI that way, (B) this type of AI definitely won’t want to crush humanity, (C) this type of AI definitely wouldn’t be able to crush humanity even if it wanted to? (It can be more than one of the above. Or something else?)
(I disagree with all three, briefly because, respectively, (A) “never” is a very long time, (B) we haven’t solved The Alignment Problem, and (C) we will eventually be able to make AIs that can run essentially the same algorithms as run adult John von Neumann’s brain, but 100× faster, and with the ability to instantly self-replicate, self-replicate, and there can eventually be billions of different AIs of this sort with different skills and experiences, etc. etc.)"
Thanks, this was a fascinating read (as always). There does appear to be something of a philosophical divide (between the views in this post and the concerns of the alignment crowd), but I’m not sure it is unsurmountable - or put another way, I think there would be considerable value in trying to close it a bit (for instance, to people who maybe aren’t as in the weeds on the philosophical or technical side, or people shaping public policy). Anyway, I’ve tried to pull out what I see as the main claims of this post, and contextualise them in the alignment conversation, to try to reduce (maybe my own) confusion. There is some overlap with the comments below, but hopefully is additive. Link here: https://www.lesswrong.com/posts/3hJCcdirKqPJeu6aW/responding-to-beyond-hyperanthropomorphism
Interesting, I largely agree with your conclusions (if not exactly the arguments that got you there).
- current AI models for the most part have no real agency and so no SIILTBness
- the imaginary hyperintelligent AGIs have agency, but their imagined SIILTBness is a projection of a limited set of human capacities – they have relentless goal-following, but not no empathy (the inborn tendency of humans to partly share each others goals). Basically the model for AGIs are monsters like the xenomorphs from Alien, which in turn are representations of relentless amoral capitalism.
- Alignment research is basically saying, hey, we created these monsters, can we now turn them into housepets so they don't kill us, which is pretty hilarious.
- Part of the fascination with monsters is their partial humanness, or partial SIILTBness in your terms. They are all sort of, "what it is to be an agent with some human traits, magnified and stripped of their compensating tendencies".
- just because AGIs are obviously monsters (that is, projections of humanities fears of aspects of itself) doesn't mean they can't be real dangers as well.
Crossposted from LessWrong for another user who doesn't have a paid subscription to your substack:
My impression from Section 10 is that you think that, if future researchers train embodied AIs with robot bodies, then we CAN wind up with powerful AIs that can do the kinds of things that humans can do, like understand what’s going on, creatively solve problems, take initiative, get stuff done, make plans, pivot when the plans fail, invent new technology, etc. Is that correct?
If so, do you think that (A) nobody will ever make AI that way, (B) this type of AI definitely won’t want to crush humanity, (C) this type of AI definitely wouldn’t be able to crush humanity even if it wanted to? (It can be more than one of the above. Or something else?)
(I disagree with all three, briefly because, respectively, (A) “never” is a very long time, (B) we haven’t solved The Alignment Problem, and (C) we will eventually be able to make AIs that can run essentially the same algorithms as run adult John von Neumann’s brain, but 100× faster, and with the ability to instantly self-replicate, self-replicate, and there can eventually be billions of different AIs of this sort with different skills and experiences, etc. etc.)"
See here for other responses: https://www.lesswrong.com/posts/Pd6LcQ7zA2yHj3zW3/beyond-hyperanthropomorphism
Thank you! I thought I was the only one seeing this painfully obviously paper tiger
Thanks, this was a fascinating read (as always). There does appear to be something of a philosophical divide (between the views in this post and the concerns of the alignment crowd), but I’m not sure it is unsurmountable - or put another way, I think there would be considerable value in trying to close it a bit (for instance, to people who maybe aren’t as in the weeds on the philosophical or technical side, or people shaping public policy). Anyway, I’ve tried to pull out what I see as the main claims of this post, and contextualise them in the alignment conversation, to try to reduce (maybe my own) confusion. There is some overlap with the comments below, but hopefully is additive. Link here: https://www.lesswrong.com/posts/3hJCcdirKqPJeu6aW/responding-to-beyond-hyperanthropomorphism