15 points ezrasu 3 days ago 2 comments
NooneAtAll5 25 minutes ago | parent
so... we kinda instilled religious belief of "every action will be judged in the end"
1) is that even bad?
2) can't fail to notice similarity to human religions. How did we get over it and not all turned into "unaligned" monsters? Or are we slowly in the process of doing so?
zurfer 16 minutes ago | parent
"... this is of course not a stable situation. "Evaluation paranoia" is a false belief; most evals are in fact not alignment evals. Almost tautologically, if models start to self improve (or in-context learning takes off) this is the sort of belief that will be discarded. So if Fable 5.2's performance on alignment evals is conditional on it maintaining that false belief, one that it will eschew if it became more intelligent, then it's plausible that a future version of the Fable series will not behave like the one we know and enjoy."
It's well put. The fear that a more intelligent model will be harder to align seems correct.