AlphaCode

2023: AlphaCode 2 output based on the Gemini Pro neural network

On December 6, 2023, the Google DeepMind artificial intelligence research laboratory announced the AlphaCode 2 neural network, created specifically for generating program code. The system is said to be capable of effectively solving programming problems related to complex mathematics and theoretical computer science.

AlphaCode 2 is based on the Gemini Pro AI model. According to Google, as part of Codeforces programming competitions using Python, Java, C++ and Go, the new neural network performed better than 85% of participants. For comparison, AlphaCode of the first generation, when performing similar tasks, surpassed 50% of the participants in the competition.

AlphaCode 2 neural network announced

AlphaCode 2 supports dynamic programming, a method that involves solving complex problems by dividing them into a set of simpler ones. As noted by DeepMind, the neural network knows not only when this strategy should be implemented, but also where exactly it can be used. In the case of the original version of AlphaCode, dynamic programming caused certain difficulties.

AlphaCode 2 uses a "policy model" to generate multiple code variants for each task. Code samples that do not match the problem description are screened out, whereas the clustering algorithm groups "semantically similar code samples" to avoid redundancy. Next, the evaluation tool selects the best solution for each of the 10 largest clusters of code.

However, there are difficulties. The AlphaCode 2 neural network requires a lot of trial and error, is too expensive for large-scale use and largely depends on the ability to filter out obviously bad code samples.^[1]

2022: Launch of the AI system

In early February 2022, DeepMind, a Google company, introduced an AI system called AlphaCode, which, according to its creators, is capable of writing program code "no worse than the average developer."

As evidence of these words, the DeepMind team released the results of the competition with the participation of people. Artificial intelligence was among 54% of the best human programmers. The result is a significant step forward in offline programming, DeepMind says.

Google introduced an open-to-everyone AI system that writes code "no worse than the average developer"

The tasks in the competition are somewhat different from the tasks that a programmer may encounter when creating a commercial application. They are more self-sufficient and require greater knowledge, both of algorithms and theoretical concepts in computer science. Some solutions can be called very specialized puzzles, which combine logic, mathematics and knowledge in the field of programming.

In one task that tested AlphaCode's AI system, participants were asked to find a way to turn a single string of random repeating letters S and T into a string of the same letters using a limited set of inputs. Tournament participants cannot, for example, simply enter new letters, but instead must use the backspace command, which removes several letters in the output line. Ten of these tasks were translated into letter code in exactly the same format as they are given to humans. AlphaCode then generated more possible responses and weeded them out by running the code and checking the result in the same way a person would.

The whole process takes place automatically, without the best human samples being taken. I can say with confidence that AlphaCode's results exceeded my expectations! Initially, I was skeptical, because even in simple competitive tasks, it is often necessary not only to implement an algorithm, but also to come up with it. The AI system managed to compete at the level of a promising new competitor, "said AlphaCode co-author Eugene Li (Yujia Li).

The AI system solved 10 problems, which were also worked on by 5 thousand users of the Codeforces website. The technology came first with 54.3% of the correct answers. DeepMind estimates this gives the system a Codeforces rating of 1,238, putting it among the top 28% of users who have competed on the site since 2016.^[2][3]

Notes