Experiments show AI could help to audit smart contracts, but not yet

While artificial intelligence (AI) has already transformed a myriad of industries, from healthcare and automotive to marketing and finance, its potential is now being put to the test in one of the blockchain industry’s most crucial areas — smart contract security.

Numerous tests have shown great potential for AI-based blockchain audits, but this nascent tech still lacks some important qualities inherent to human professionals — intuition, nuanced judgment and subject expertise.

My own organization, OpenZeppelin, recently conducted a series of experiments highlighting the value of AI in detecting vulnerabilities. This was done using OpenAI’s latest GPT-4 model to identify security issues in Solidity smart contracts. The code being tested comes from the

It’s important to note that ChatGPT and GPT-4 are LLMs developed for natural language processing, human-like conversations and text generation rather than vulnerability detection. With enough examples of smart contract vulnerabilities, it’s possible for an LLM to acquire the knowledge and patterns necessary to recognize vulnerabilities.

If we want more targeted and reliable solutions for vulnerability detection, however, a machine learning model trained exclusively on high-quality vulnerability data sets would most likely produce superior results. Training data and models customized for specific objectives lead to faster improvements and more accurate results.

For example, the AI team at OpenZeppelin recently built a custom machine learning model to detect reentrancy attacks — a common form of exploit that can occur when smart contracts make external calls to other contracts. Early evaluation results show superior performance compared to industry-leading security tools, with a false positive rate below 1%.

Striking a balance of AI and human expertise

Experiments so far show that while current AI models can be a helpful tool to identify security vulnerabilities, it is unlikely to replace the human security professionals’ nuanced judgment and subject expertise. GPT-4 mainly draws on publicly available data up until 2021 and thus cannot identify complex or unique vulnerabilities beyond the scope of its training data. Given the rapid evolution of blockchain, it’s critical for developers to continue learning about the latest advancements and potential vulnerabilities within the industry.

Looking ahead, the future of smart contract security will likely involve collaboration between human expertise and constantly improving AI tools. The most effective defense against AI-armed cybercriminals will be using AI to identify the most common and well-known vulnerabilities while human experts keep up with the latest advances and update AI solutions accordingly. Beyond the cybersecurity realm, the combined efforts of AI and blockchain will have many more positive and groundbreaking solutions.

AI alone won’t replace humans. However, human auditors who learn to leverage AI tools will be much more effective than auditors turning a blind eye to this emerging technology.

This article is for general information purposes and is not intended to be and should not be taken as legal or investment advice. The views, thoughts and opinions expressed here are the author’s alone and do not necessarily reflect or represent the views and opinions of Cointelegraph.