Misaligned superintelligence: How do we keep it from deleting New Hampshire?
Venice.ai is one free stater’s superior alternative to Big AI, but using it won’t be enough. Here’s what else you can do.
As some of you may be aware, there is a menacing new term in the English vocabulary: “P(doom).” P(doom) is the projected likelihood that artificial intelligence will wipe out humanity or at least civilization. Etherium founder Vitalik Buterin’s P(doom) is 10% as of 2024, presumably 90% confident of a tolerable outcome. Big AI whistleblower Daniel Kokotajlo has a P(doom) of 70%. Mine has risen to 25%.
Kokotajlo claims his high P(doom) number stems from a lack of sufficient “alignment prioritization.” AI alignment is the extent to which a given intelligence aligns its actions with the general well being of humans. Terminator’s Skynet would probably have an alignment rating around 10%, Space Odyssey’s HAL 9000 around 80% and Star Treks’ Commander Data perhaps 99%. Kokotajlo says the companies most likely to achieve superintelligence are recklessly under-focused on alignment…and many AI experts believe him. The safety these top companies do focus on seems to be more about shielding snowflakes from having their feelings hurt than from having their civilization disemboweled.
