AF - Beyond the Board: Exploring AI Robustness Through Go by AdamGleave

The Nonlinear Library: Alignment Forum

Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Beyond the Board: Exploring AI Robustness Through Go, published by AdamGleave on June 19, 2024 on The AI Alignment Forum.
Last year, we showed that supposedly superhuman Go AIs can be beaten by human amateurs playing specific "cyclic" patterns on the board. Vulnerabilities have previously been observed in a wide variety of sub- or near-human AI systems, but this result demonstrates that even far superhuman AI systems can fail catastrophically in surprising ways.
This lack of robustness poses a critical challenge for AI safety, especially as AI systems are integrated in critical infrastructure or deployed in large-scale applications. We seek to defend Go AIs, in the process developing insights that can make AI applications in various domains more robust against unpredictable threats.
We explored three defense strategies: positional adversarial training on handpicked examples of cyclic patterns, iterated adversarial training against successively fine-tuned adversaries, and replacing convolutional neural networks with vision transformers. We found that the two adversarial training methods defend against the original cyclic attack. However, we also found several qualitatively new adversarial strategies that can overcome all these defenses.
Nonetheless, finding these new attacks is more challenging than against an undefended KataGo, requiring more training compute resources for the adversary.
For more information, see our blog post, project website or paper.
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.