A Brief Note on P-Doom

This is a brief note on artificial intelligence, and why I am, in short, skeptical of doom.

It is certainly true that we do not know how, or if, we can align AI. I have no doubt that the artificial intelligences we construct will be deeply alien to us, that we will not understand, in any deep sense, the mechanics of how they work, and that we will always be uncertain of whether we are pointing them toward the right goals.

At the same time, we are extremely uncertain how, or if, we can make a super intelligent AI at all. There is no guarantee that we will ever make a super-intelligent AI. There is no guarantee we will ever make an AI which is meaningfully an independent agent, or has desires as we think of them, or which cannot be aligned by simply saying “do this, not that”.

What I want is for our belief in technological change to be two-sided. I cannot see why we should a priori believe that technology will improve faster than our ability to control them. To have faith in one side, but not the other, is simply a mood. It is not a mood shared by those who have to put their money where their mouth is, either — while it is theoretically ambiguous which way interest rates should go if AGI comes and is aligned, a misaligned AI killing everyone would unequivocally make real interest rates go up.

It’s okay that we don’t understand AIs. I do not believe we need to understand AIs to align them. We do not understand how humans think, either, nor can we directly intervene upon our preferences, yet we are capable of sustaining the civilization in which we now live. I am sure that we will have a good shot at developing novel methods of alignment which could not be anticipated in advance — after all, won’t we have the best AIs helping us?