Artificial intelligence company Anthropic announced that artificial intelligence systems are increasingly able to carry out software development and research processes without human intervention, and if this trend continues, they can turn into self-improving systems.
The company published the blog post titled “When AI builds itself,” written by co-founder Jack Clark and Anthropic Institute President Marina Favaro.
The article stated that in the past, artificial intelligence development processes were largely carried out by humans, but in Anthropic, these tasks were increasingly transferred to artificial intelligence systems, and this increased the speed of development.
It is stated in the article that if this trend continues, artificial intelligence systems may reach the level of being able to design and develop their own successors, and that this situation is defined as “recursive self-improvement”.
“We are not there yet, and recursive self-improvement is not inevitable. But it could happen sooner than most institutions are prepared for,” the article said.
Risk of weakening human control
It was stated in the article that technological trends indicate that artificial intelligence systems will have much more advanced capabilities in the coming years, and that self-improving artificial intelligence can provide significant benefits in science, health and many other fields.
However, it was noted that “recursive self-improvement” could also increase the risk of people losing control over AI systems.
“If systems are fully capable of building their own successors, the ways we secure them, monitor them, and shape their behavior become even more important,” the article said.
It is stated in the article that it may be useful to consider slowing down or temporarily stopping advanced artificial intelligence development studies so that social institutions can adapt to technological advances, and that it would be meaningful to implement this simultaneously by more than one powerful laboratory and in different countries.
