Devin Has Exposed a Major Issue with Software Engineering

And it isn't that we're all going to lose our jobs

Cognition Labs recently released the first AI software engineer and it has caused developers to fear their job security. I've even seen posts on X about high schoolers changing their major for college just because of Devin. I understand the fear here, based on the general fear regarding AI taking jobs and Nvidia's CEO Jensen Huang claiming people shouldn't learn to code only a few weeks ago, but a lot of this is hype. As someone who works in machine learning and software engineering, I'll let you know software engineers’ jobs are safe for a little while longer.

Devin is the next logical step for AI assistance with programming after Copilot. It amazes me how worried developers are about Devin automating aspects of their job when the community has heavily enjoyed Copilot doing the same. Did we not expect Copilot and similar tools to get better?

The supposed most techno-literate group of society has become the most fear-based regarding AI advancement. There are certain tasks AI is good at and they'll replace those tasks within jobs first. This isn't exclusive to software engineering. There are aspects of all jobs that will be automated soon and many more that will take a longer time for AI to do so. By the time software engineers are fully replaced, there are many other jobs that will already have been replaced. Why software engineers are the group panicking most about AI taking jobs, I will never understand.

The release of Devin has uncovered two things about most software engineers that are way more interesting to me than the autonomous engineer:

1. Most engineers have no idea what their job actual consists of and why they’re paid well.

2. Most engineers seriously lack an understanding of machine learning.

I’ll get into both in more detail below.

Devin has been shown to complete ~14% of issues end-to-end without intervention. This is much higher than similar programming tools.

What Devin Can Do

Cognition Labs have fine-tuned Devin for programming tasks and given it access to common developer tools to accomplish them. Devin’s listed capabilities are:

• Read blogs and learn from them. LLMs have already shown they can comprehend text and images given to them. This is doing the same, but using those directions with developer tools. The novelty here is the integration with developer tools.

• Build and deploy apps end-to-end. I'm curious to see what this means in practice, but this shouldn't come as a surprise due to the above. If Devin can read directions and implement them, Devin can build an app if it has the directions to do so.

• Autonomously find and fix bugs. Copilot has proven a LLMs ability to find bugs in a codebase and propose possible fixes. Again, the novelty here is actually implementing the fix. This is really cool, but I'm curious to see how it plays out in reality and where its limitations are.

• Train and fine-tune its own AI models. Again, there are many directions online for how to do this. I highly doubt Devin is doing the data preprocessing and other difficult decisions for creating high-quality AI models. If it is, that's the real moneymaker for them.

• Devin can address issues in GitHub repos. This shouldn't come as a surprise since Devin can autonomously find and fix bugs. In this case, Devin will just have to read an issue reported in the repo and determine the course of action from there.

• Devin can contribute to mature production repositories. This follows from all of the above. Just like Copilot, Devin will be able to understand code and propose fixes and improvements. The difference is its access to developer tools and its ability to use them.

• Devin can do jobs on Upwork. I'd have to look into this a bit more, but it seems like this follows from the above. I'm guessing more complex Upwork tasks would be difficult for Devin to accomplish.

To be clear: I'm very excited for Devin. I think this will take a lot of the monotony out of the often tedious work software engineers are faced with. Copilot and similar tools were very good at it and Devin is even better. I don't mean for the above bulleted list to be overly negative, but these are almost all capabilities we already knew LLMs had, just applied really well to a use case that benefits us. This is the direction we want AI-assisted coding to go and it is a net positive for the software engineering community.

What I'm more concerned about regarding Devin is what is has exposed about the software engineering community. It seems to me a lot of the fear surrounding Devin’s announcement comes from a lack of basic understanding of what software engineers are paid to do.

What Software Engineers Do

I think this misconception comes from the people thinking software engineers are paid well to code. We are paid well to solve difficult problems. This is why we call ourselves engineers even though we aren't accredited in any way. Engineers solve problems using the tools at their disposal.

In a largely digital world, that tool is often code and that's why software engineers exist. We use code to solve problems. It's as simple as that. We're paid well because that skill is in demand and highly valuable to a lot of companies. Companies need people who can identify how to address user needs and design the systems to do so. This becomes an increasingly difficult problem to solve when factoring in integration with other systems, scalability, reliability, local laws and regulations, etc. The software engineering skill that's valuable is solving the problems—not writing the code.

This is why I have a very difficult time recommending any sort of boot camp or course sold by a Twitter influencer to anyone to learn how to code. Too many bootcamps and courses focus on "learn this language that's in demand" or just "learn how to code in general". These don't teach the skills that are valuable for software engineers. They're necessary to be a software engineer, but they’re not the core skill. The valuable skill is knowing how to approach and solve a problem.

This is why I recommend a college degree from a good institution to anyone who wants to get into software engineering. A proper computer science curriculum will teach you how to approach problem-solving using code as a tool. These curricula take four years and cost money because its difficult to learn the concepts needed for a career in computer science quickly. It's much more beneficial to take your time and learn things correctly. Even though you'll spend time and money learning, there are few things with a higher return on investment than a proper degree in computer science.

Yes, you can learn how to be a software engineer on your own or through a bootcamp or program but very few people will. Most people will learn how to code on their own and won't even know there's a difference between knowing how to code and having the skillset of a software engineer. This misunderstanding is exactly what Devin is exposing. There are many software engineers that I fear don't even fundamentally understand the value of their job.

As an aside, I personally think this also contributes to why there are so many people who don't enjoy being a software engineer. If you learn to code quick-and-easy, you won't realize software might not be for you. If you join a computer science program, you'll quickly realize if it isn't for you.

Inadequate educational tools also lead to disparity in software engineering talent. I've worked with many great developers and many good or okay developers. It's pretty easy to tell which is which and there's a large disparity between what those groups are capable of. This is expected if some developers have spent years devoted to understanding problem-solving with software and others have spent a few weeks learning Javascript.

A lot of “software engineering” educational tools are low-quality resources designed to make money for those selling it. It's much easier to sell "Spend 5 weeks learning to code and make six figures" than it is "Spend 4 years and thousands of dollars to make six figures". There are very few software-related courses outside of a college degree I would recommend because most teach you how to code—not how to engineer.

The bottom line here: Devin has shown the software development community that we need better resources for learning software development and and understanding of what makes the jobs valuable. This is a tough task because it's impossible to learn something you don't know if you don't even know you don't know it. Ya know?

Software Engineers Should Understand Machine Learning

Devin has also exposed how little most software engineers understand about machine learning. Solving problems using machine learning is a skill all software engineers will need to have in the future. If you're truly worried about AI taking your job as a software engineer, dig into how AI works and better understand it. Every software engineer with an understanding of machine learning isn't worried about Devin. Almost every software engineer without, is.

As described above, a software engineer’s job is to create systems that solve problems using code. Currently, we do this by giving a system rules that we've coded and data as input and it'll give us answers as output.

Machine learning will fundamentally change this. It will become the default way to solve complex problems. In machine learning, we give a system data and answers and it figures out the rules for itself. We then use those rules to generate answers for other inputs. As software engineers, this is a system we need to understand because it will soon be fundamental to our jobs.

This is a way AI will change the software engineering landscape. While most are afraid of Devin entirely replacing software engineers, those who understand machine learning know that that AI will change the roles humans play in their jobs before replacing them. Software engineers will spend less time on monotonous tasks and more time solving problems. Many of those problems will be solved with machine learning.

Software engineers don't just need to understand how to use AI tools to solve problems, they need to understand the machine learning fundamentals to build the AI tools that will solve problems. That is what the job of software engineers will become.

If you want to better understand machine learning, I've put together a road map to walk you through the high-quality free learning resources available in the ML community. Feedback is greatly appreciated so I can improve it.

One last thing: regarding Jensen Huang's comment about software not being worth learning since AI will take it over anyway, just remember that Nvidia's business narrative greatly aligns with this statement. Any time a statement like this is made, always understand where it's coming from.

Let me know if you agree with my statements. If not, let me know what you think in the comments. Thanks for reading!

---

You can support Society’s Backend for just $1/month. You’ll get access to the full Backend Biweekly update on ML learning resources and developments, my notes, and occasional paid posts. You can also support Society’s Backend for a one-time payment and get access to all paid resources forever at https://societysbackend.com/subscribe.