September 25, 2023
Over the past few months, I've been thinking a lot about what all the hype around AI tools like ChatGPT will mean for my day-to-day work as a software developer. While tools like Github CoPilot are impressive and have made me more productive, I've noticed that they still have a lot of limitations. The past weeks I tried to read up on some of the information out there, to try to understand more of it. While certainly not everything I write here will become true, this post is an exploration of what I learnt on the way and by all means a lot of it is just personal imagination of what could be. To start this post, I will share a quote from a book I've enjoyed:
The past was once the future and the future will become the past.
Richard Hamming, a renowned mathematician and computer scientist, wrote a great book that explores how we can look into the future of engineering fields without loosing track of all the fundamtenals that shaped those fields. He delves into the limitations of hardware and software applications, and even AI, while also tracing the origins of modern computing. The Art of Doing Science and Engineering explores a lot of the intersections between how the past can influence what will happen in the future. Surprisingly, the art of hand-looming and weaving laid a big foundation for computing and the large productivity gains that came alongside these innovations. James Essinger sheds light on this connection:
The astonishing truth is that the Jacquard loom enabled decorated fabric to be woven about twenty-four times more quickly than the drawloom. Whereas in the past even the most skilled weaver and draw-boy duo could only manage two rows or picks of woven fabric every minute, a skilled lone weaver using the Jacquard loom could manage to fit in an average of about forty-eight picks per minute of working time.
Jacquard's invention was a game-changer for the tapestry industry, catapulting skilled weavers to unprecedented levels of productivity and creativity. The mastery of looming underwent a paradigm shift with the introduction of the programmable loom, laying the groundwork for the computing revolution through punch cards.
Fast forward to today, and we find ourselves in a similar situation when talking about AI synthesised code or media. Much like weaving, software development involves meticulously weaving countless threads of code to form intricate patterns of functionality. Each line of code carries a purpose, and is carefully chosen to contribute to the overall design. This field demands a keen eye for detail, an understanding of logic and problem-solving, and a passion for creating digital experiences that transform the way we interact with technology.
However, similar to the people working with hand-looming, software developers now face the challenges of an ever-accelerating technological landscape. The rise of artificial intelligence (AI) has ushered in significant changes, particularly in the area of code generation. AI's ability to analyze vast amounts of data, identify patterns, and generate various forms of content, including code generation, outperforms human capabilities already.
Over the past few decades, software development has intricately woven its way into the fabric of modern society. From the beginnings of punch cards and mainframes to smartphones, social media, and cloud computing, the field has experienced a remarkable transformation, continually accelerating its evolution. Reflecting on the transformative power of the internet, Tim Berners-Lee, the inventor of the World Wide Web, once wrote in 1999:
It is a vision that provides us with new freedom, and allows us to grow faster than we ever could when we were fettered by the hierarchical classification systems into which we bound ourselves. It leaves the entirety of our previous ways of working as just one tool among many. It leaves our previous fears for the future as one set among many. And it brings the workings of society closer to the workings of our minds.
Back in 1999 no one could know how the internet would evolve, many people were afraid of it and many others very excited about it’s possibilities (the iconic information highway news come to mind). AI today triggers similar fears, because overall we cannot really yet imagine in what positive or negative ways AI will shape our future. The conversations around limitation and ethics of AI are getting louder as many people realise that it will be taking away jobs or open the doors for mass generated fake media. But as much fear there is around AI, I want to start to look into how it could actually improve a software developers day-to-day:
By automating repetitive tasks such as code generation, testing, and debugging, developers can save significant time and effort. Github Co-Pilot is a good example, as it can analyze specific patterns in a codebase and suggest simple auto-completions, which a developer would have spent typing anyway and is a huge time-saver in daily, simple programming tasks. A Github study found that developers who used GitHub Copilot coded up to 55% faster than those who didn’t. Another productivity boost will come from automatic bug fixing. Getafix by Facebook, which learns bug-fixing patterns and automatically offers fix suggestions is a good example for this. We can use machine learning to figure out specific patterns in our commit history to automatically suggest an appropriate fix. Through pattern analysis of large codebases (like Aroma by Facebook does) we can automate all the tasks that are based on redoing patterns which occur a lot in a large codebase. Another example is Githubs new code referencing tool, which shows you when synthesised code matches open source patterns.
Even the most experienced developers can make errors, especially when we work on complex projects or under tight deadlines. We get tired or bored, which can result in sloppy or faulty code. AI systems on the other hand can maintain a high level of accuracy and consistency, reducing the risk of errors and improving the overall quality of the code. For example, static code analysers like DeepCode leverage AI algorithms to identify potential issues and vulnerabilities in code, enhancing code quality and reducing the probability of errors in software solutions. Similarly, AI tools like Amazons code guru assist with the code review process, pointing out potential security issues. The example of accuracy and consistency goes back to the speed and detail in which AI tools can analyze large amounts of data. With the advance of programming language processing (PLP), which is similar to natural language processing (NLP), machine learning systems can learn specific patterns within a system and use the patterns to automate different tasks, including code generation, bug finding and security analysis.
By handing over mundane and repetitive tasks to AI, we as developers can devote more time and energy to exploring new ideas, experimenting with new technologies, and driving innovation. With AI streamlining workflows and suggesting common optimisations or code snippets, we can build stable proof of concepts a lot faster. This frees us up to be creative, bring our human perspectives to problem-solving, and focus on developing creative solutions:
“The PACMPL study also found that developers used GitHub Copilot to find creative solutions when they were unsure of how to move forward. These developers searched for next possible steps and relied on the generative AI coding tool to assist with unfamiliar syntax, look up the right API, or discover the correct algorithm.”
There are of course more applications of AI, I've found this paper titled Applications of AI in classical software engineering to have a great overview, if you want to dive deeper. The paper explores the whole software development cycle and potential AI applications from project planning to software maintenance. After reading a lot of these insight, I've found that AI is still far from replacing human developers, so let’s look at some of the limitations.
One thing to recognize is that AI does not replace human creativity and expertise, because we as developers bring a deep understanding of user experience, aesthetics, and branding to the software we build. Let's look at three key limitations that AI faces in the realm of software development.
One of the primary limitations of AI lies in its ability to grasp the various nuances of software development within specific contexts. While proficient in code analysis and bug detection, AI struggles to comprehend complex project requirements. In the research field this is called the context problem. Programming demands a profound understanding of problem domains, user requirements, and industry-specific constraints that AI systems lack. Despite excelling in pattern recognition, AI fails to capture the contextual details that humans effortlessly perceive. For instance, while an AI can replicate brush strokes and mix colors, it cannot comprehend the emotional, symbolic, or cultural significance behind a painting. Although AI can write code, identify bugs, and suggest performance optimisations, it cannot comprehend the unique needs of specific user groups, the business implications of features, or the strategic importance of certain features. This contextual limitation exists in many AI fields, way beyond just software development.
Software development projects often deal with incomplete or evolving requirements, making it a challenging domain for AI. AI models rely on historical data and patterns to make predictions and decisions. However, in software development, new challenges frequently arise, pushing the boundaries of what is known. We innovate, talk to our customers and adapt to the changing requirement. Novel scenarios that may have limited or no historical data, makes AI generated code less effective or useless.
Another area where AI encounters limitations in software development are ethical and value-based decisions. Developing software requires considering diverse stakeholder interests, legal frameworks, and ethical considerations. AI models, being data-driven, may unintentionally reinforce biases present in the training data, leading to unfair or discriminatory outcomes. Moreover, AI systems often function as black boxes, making it challenging to understand the decision-making process behind their predictions. One interesting example for this is how bad ChatGPT is in generating random numbers. It will actually prevalently generate 7 and 42, because of the occurrence of those numbers in the training data. Lack of transparency raises concerns regarding accountability, trust, and the potential for biased or unjust results. One of the main concerns in anything automated, be it AI generated code or self-driving cars, in the end also comes down to the security and trust in these automatic behaviours. At this point most of them cannot be operated safely without the oversight of a human.
Now that we learned a little bit about the possibility and limitations of AI in software development, it’s time for us to take a speculative journey into a day in the life of a future software developer:
Her day begins with a cup of coffee and a code review session. She takes a look at the open pull-requests and engages in a conversation about the suggestions made by an AI automation. She shares her insights on the AI's recommendations along with her own informed perspectives about the change. After the code reviews, she opens her code editor IDE, which is smoothly integrated with an AI assistant (think cursor.sh). As she starts working on a new feature, her AI partner is right there, analyzing the codebase and offering intelligent suggestions by following what she's working on. The AI's help goes beyond just generating code. It understands her coding style, preferences, and past decisions, enabling it to provide personalized suggestions. It helps her stick to best practices and coding standards, ensuring consistent code quality throughout the project.
After the code review, she changes her focus to implementing a new feature within the existing codebase. She refers to the project documentation and consults her AI partner to gain a more complete understanding of the feature requirements. The AI provides valuable insights from similar implementations and offers suggestions on the most effective ways to integrate the new functionality, potentially leveraging existing patterns in the codebase. She uses AI-powered tools to navigate through the codebase, leveraging intelligent code search capabilities. Her AI partner assists in locating relevant code snippets, identifying reusable functionalities, and understanding system dependencies. This speeds up the development process, saving time on manual exploration and allowing her to dive straight into implementing the feature. Additionally, her AI partner acts as a virtual pair programmer, capable of identifying potential issues, bugs, and vulnerabilities. It performs static code analysis on the fly, using AI algorithms to flag areas that require attention, such as potential performance bottlenecks, security vulnerabilities, or other design flaws. This enables her to promptly address these concerns, eliminating the need for separate code analysis pipelines that would run after the code was written.
Towards the end of the day, she encounters a performance issue or a persistent bug in the codebase. She collaborates with her AI partner to efficiently diagnose and resolve the problem. The AI analyzes the codebase, runtime logs, and relevant metrics to identify the root cause of the issue. Based on its analysis, her AI partner suggests potential solutions and identifies bottlenecks to eliminate the bug. It draws from a vast knowledge base, incorporating previous solutions to similar issues. She evaluates the recommendations, experiments with different approaches, and implements the necessary changes. Throughout the optimization process, her AI partner offers insights on resource consumption, response times, and system behavior, facilitating the fine-tuning of the code for optimal performance.
Before wrapping up the day, she reviews the day's work, ensuring that all changes are thoroughly documented and communicated to the team. The AI assists in drafting a summary of the day's accomplishments, which she shares with other team members. The team engages in discussions about any outstanding tasks and challenges, leveraging their collective knowledge and experience to work together and plan future tasks.
As AI continues to change the software development industry, the role of software developers is definitely undergoing a transformation. Traditionally, software developers have been responsible for writing, testing, and maintaining code, most of which has been done manually. However, with AI systems increasingly capable of automating these tasks, developers are finding their roles shifting from manual coding to supervising and working together with AI systems. Thomas Domhke, the CEO of Github says that sooner or later 80% of the code is going to be written by Copilot (or similar tools), freeing developers to focus on more creative work. While developers will still be the experts, they will spend more time evaluating synthesised code and working together with code generation tools to become more productive.
This shift in responsibilities also necessitates a change in the skill set required for software developers. In the age of AI, technical skills related to machine learning and data science are becoming increasingly important. As AI takes over more routine tasks, soft skills like critical thinking, creativity, and problem-solving are becoming more crucial. Developers are now expected to focus more on understanding business needs, designing effective solutions, and making strategic decisions. Yuval Noah Harari emphasizes the importance of adapting to change and developing general-purpose life skills in this changing century:
Many pedagogical experts argue that schools should switch to teaching “the four Cs”—critical thinking, communication, collaboration, and creativity. More broadly, they believe, schools should downplay technical skills and emphasize general-purpose life skills. Most important of all will be the ability to deal with change, learn new things, and preserve your mental balance in unfamiliar situations.
We as developers need to learn how AI algorithms work and how to work with them effectively.
An interesting influence can also be seen in the rise of citizen developers and no-code tools. Citizen developers are individuals without a traditional programming background, who use AI-powered low-code or no-code platforms to create software or functionality for their own needs. This trend highlights how AI, specifically large language models functioning as translators, are democratizing software development, allowing more people to create (simple) software while freeing up professional developers to focus on more complex tasks. This vision is also described by Richard Hamming in his 1997 book:
What is wanted in the long run, of course, is that the man with the problem does the actual writing of the code with no human interface, as we all too often have these days, between the person who knows the problem and the person who knows the programming language.
While expert developers are still needed, the barrier of entry is reduced and the rise of LLMs and code generation tools make it easier and faster for non-experienced developers to produce the needed output they would not have been able to create before.
In today's fast-paced world, learning in the field of software development comes with its unique set of challenges. As technology evolves rapidly, we must constantly acquire new skills and knowledge to stay relevant and competitive. Every software developer knows this feeling after a few years in the industry. Yuval Noah Harari writes:
Traditionally, life has been divided into two main parts: a period of learning followed by a period of working. Very soon this traditional model will become utterly obsolete, and the only way for humans to stay in the game will be to keep learning throughout their lives, and to reinvent themselves repeatedly. Many if not most humans may be unable to do so.
However, the abundance of information online can be overwhelming, especially if you're new to the game or haven't had time to catch up. In a world cluttered with information, the ability to see clearly what is important and what is not, is a really powerful skill. If you are able to navigate this information overload and extract relevant knowledge you will have a significant advantage:
In a world deluged by irrelevant information, clarity is power.”
Setting clear learning objectives, seeking mentorship, and maintaining a growth mindset are essential skills s to thrive in the growing ecosystem that is getting even more complex with AI.
In exploring the intersection between software development and AI, I've gained a fair bit of insight into the possibilities AI presents for the future of coding. Learning about code synthesis, automated testing, and other possible areas of AI application has shown me that AI will become deeply integrated into the developer workflow. Just as we rely on version control and project management tools today, AI-synthesised code and automatic code suggestions will likely become commonplace.
I believe we'll increasingly work with synthesised code snippets rather than writing everything by hand. Humans have always embraced advances that boost productivity, so automating routine coding tasks with AI seems inevitable. At the same time, extensive training and experience will still be required to become an expert programmer and to work productively (and safely) with AI tools.
Furthermore, AI does have the potential to make software development more accessible to non-traditional coders. With user-friendly low-code/no-code tools powered by large language models, creative people outside tech can build relatively complex apps. While their skills may not be the same as a developer, who trained over 10 years, empowering more people to create through code is a exciting prospect.
In terms of learning, I still find it difficult to try to keep up with the rapid pace of changes in tech and AI. Overall, I have to say I feel critical about a lot of the developments and bad applications of AI, but having played around with various tools, from code, to text to image generation, I'm also highly impressed with the productivity and creativity gains these things bring. Gaining a better understanding sparks a lot of the natural creative interest I have and I see a lot of potential in all of these tools if used in the right way.
Disclaimer: The images in this blog post have been created with Leonardo.AI and large parts of the text have been improved for clarity with GPT like tools. The research, connections and overall outline were my own making