Today we're excited to announce new AI-powered products that help teams write, understand and modernise their legacy codebases written in COBOL. This marks a significant step forward in our mission to eliminate menial software software engineering tasks. If you're unfamiliar, COBOL is the most common programming language for mainframes.
Modernised codebases are highly productive to work on, can be easily integrated with distributed systems and hosted on or off the mainframe. As a result, businesses are currently moving 37% of their application portfolio off mainframe, with those that have already done so reporting a 9-11% increase in profits after modernisation*.
Teaching LLMs to (legacy) code
New AI software engineering tools are rarely tested on the development of COBOL applications. That's why we're introducing COBOLEval, a new benchmark based on the popular HumanEval benchmark, and the first to measure the performance of large language models over a diverse range of COBOL programming tasks.
Of course we couldn't release a benchmark without also releasing a model… so we're announcing mAInframer-1, a new large language model which beats GPT-4 to top place on the COBOL coding leaderboard. The model is based on Meta's Code Llama, which is then further fine-tuned on millions of synthetically generated lines of COBOL. Today we're releasing bloop write, which packages a smaller version of this model as a coding copilot on the VS Code Marketplace. The extension, including the model, runs completely offline so your code never leaves your device.
A novel architecture for code translation
Making the current small pool of COBOL engineers more productive won't solve the talent shortage, so we're also committed to widening the long term modernisation paths available to organisations. Our first product to address this is bloop modernise, our new AI-powered COBOL to Java conversion pipeline that produces readable Java, unlike previous-generation code translation tools which produce unreadable code known as 'Jobol'. By being target-platform agnostic, whether that's Java on z/OS, the cloud or elsewhere, we can widen the modernisation paths available and help teams move in whichever direction best suits their needs.
Continuing investment in code understanding
And finally, we've updated our codebase understanding tool to better support mainframe languages, including COBOL. This makes it easier to ask questions in natural language and use code navigation when working on mainframe projects.
About bloop
bloop was founded in 2021 as a research lab focussed exclusively on AI applied to programming languages. We have decades of individual experience training machine learning models and building compilers. Our investors include Y-Combinator, Khosla Ventures, Sands Capital and LocalGlobe.
To learn more about how we can support you, please get in touch.
*Kyndryl 2023 State of Mainframe Modernization Research Report (link)