The generative AI boom has supercharged a new, transformative wave of AI developer tools. LLMs are becoming a core component of developer workflows, both through their integration into existing products and via the development of novel, AI-first products. Recently, I’ve been looking into one particular class of developer tools: AI coding tools. AI coding tools are exactly what they sound like: they augment traditional software development workflows by generating and understanding code.
The most widely adopted AI coding tool is GitHub Copilot, which has over 1 million paying users across 37K organizations [1]. Of course, GitHub is not the only company building in this space. There are plenty of other players, from titans like Amazon to newcomers like Anysphere. In general, these tools can be divided into 2 categories: plugins that you install into your IDE and new development environments that replace your IDE. For plugins, we have Copilot (GitHub), CodeWhisperer (Amazon), Cody (Sourcegraph), Mutable, and many more, while for environments, we have Replit and Cursor (Anysphere).
At the moment, most companies are building plugins. Compared to new environments, plugins are lighter weight and can be more easily integrated into existing workflows. While this makes sense in the short-term, environments are by far the better long-term approach. Emerging AI tools give developers unprecedented speed and accuracy, so as AI models improve at writing code, the way we build software will inevitably evolve as well. Imagine a world in which developers manage fleets of AI agents to build new software, rather than writing every line of code themselves. When that time comes, we’ll need an AI-first development environment with deep AI integrations, something a plugin can’t achieve.
As an aside, I did a deep dive into AI coding tools last month, and one of the features that captured my attention was codebase-wide context. Despite how useful this feature is, most products don’t actually support it. For example, Copilot doesn’t have this feature, although GitHub does seem to be working on it. So why is that the case?
Well, at first glance, this feature might seem like a simple information retrieval problem: you store the code files as embeddings, retrieve and rerank them based on their similarity to the query, and then feed the top results as context to your LLM call. In reality though, it’s a bit more complex. For one, code files are not independent. In a traditional information retrieval system, the documents are usually assumed to be unrelated to one another, but that obviously doesn’t hold for code. You’re almost always importing functions or definitions from other files, so any particular file might only make sense in the context of other files. In addition, code follows a structured syntax. If you embed a code file just like a normal text document, then you lose out on any context hidden behind its structure.
Note that these are just the theoretical challenges. Whatever architecture you do implement also has to scale to large codebases in terms of performance, cost, and viability, or else the feature is not really useful. The tl;dr is that this is a tough problem, but that’s what makes it interesting. Cody and Cursor both have an implementation of this that you can play around with, so if I’ve piqued your curiosity, I recommend giving those products a try.
On the other hand, if you’re interested in learning more about how AI coding tools work, there’s this insightful blog post by Parth Thakkar, who spent some time reverse engineering the code for GitHub Copilot. I recommend giving it a read!
Given the long-term vision of a new, AI development environment, I would say that Cursor has a slight lead over everyone else. I’m not as keen on Replit because I don’t see cloud IDEs replacing local IDEs, even with AI features. That being said, the space is still in its early stages, so there’s plenty of room for disruption. According to Gartner, less than 10% of enterprise businesses have adopted AI coding tools into their developer workflows [2].
I do believe, however, that GitHub is best positioned to win the overall market. Not only is Copilot already the most widely adopted AI coding tool, but GitHub also has a huge distribution advantage with its 100 million users [3]. In addition, it’s interesting to note that GitHub’s parent company, Microsoft, owns VSCode, so they could foreseeably pivot towards building their own AI IDE in the future. Regardless, the developer workflows of tomorrow will be drastically different from those of today, and I look forward to watching how everything plays out.
[1] https://www.zdnet.com/article/microsoft-has-over-a-million-paying-github-copilot-users-ceo-nadella/
[2] https://www.cnbc.com/2023/11/08/microsoft-launches-github-copilot-enterprise-to-help-with-private-code.html
[3] https://github.blog/2023-01-25-100-million-developers-and-counting/