In the 1995 book “Dynamics of Software Development”, Jim McCarthy then director of Visual C++ had a thumb rule that said “Don’t flip the bozo bit” - meaning you should not label a person with this moniker and dismiss the person as not worth listening to. His point is that everyone has some perspective worth contributing to, in the context of Software Development.
In today’s era of ChatGPT many developers may be turning the bozo bit on themselves, now that these large models are handing over “answers” in a platter based on the prompts given by user.
However, this scenario is fraught with a lot of risks, and good old horse sense will become all the more relevant.
Timeless principles of good software engineering methods will make or break the success of the solutions.
Large language models generate text, pixels, code etc. based on the statistical probability of tokens that would follow based on the patterns the model has learned from training corpus. The training corpus is typically self supervised learning by crawling piles of web data, journal papers, books, github code repositories etc.
Code generation is a typical activity that most developers would do, as this has potential to greatly improve productivity. However, there is no guarantee that the code generated is correct or efficient or even understandable.
Imagine that GPT gives you hundreds of lines of Python or SQL code, now that would require a fair amount of effort to understand even if it works. Most of us working in teams often prefer to recreate code versus reuse code built by another developer as the latter would take more effort to understand often!
So while using GPT to generate code, the need for code review and exception scenario evaluation becomes very important. The pitfall to avoid here is the allure of taking agile to the far extreme and rapidly integrate GPT code without careful cross checks.
The move towards microservices architecture may also be beneficial, as this tends to decompose complex code into smaller manageable components that can be understood better.
From Application architecture lens, the key decision may even involve using specific GPT models for certain use cases. Here again is the alluring option of putting the latest shiny object and fitting it into your use case - person with hammer syndrome - every problem looks like it has a solution in the hammer of GPT models.
First question to ask is - do we need AI or will rule based classic computing solve the problem?
Second, if we need AI then do we really need large GPT model or traditional Deep learning or even classic Machine learning?
Simplicity in architecture will lend itself more easily to explainability and interpretability. Many classic machine learning models like Regression, KNNs, SVMs etc. are easy to explain, and one can see which features of data actually influenced the prediction of these models.
Deep learning models for activities like computer vision, recommenders etc. can be more accurate than classic ML models at the expense of some explainability, and yet the neural net architecture of these are much smaller than large GPT models, and there is also control over the lineage of data used in training and ease of hosting in infrastructure within your control.
And finally, there are many scenarios where large GPT models will be the best answer for tricky problems like content summarization, versatile chatbots, human like virtual agents etc. Careful evaluation of data privacy, cost of using the models, hallucination and bias guard rails become primary activities by involving Quality assurance team at every stage of the SDLC from requirements to develeopment to testing and operations monitoring.
The new AI regulations like EU 2024 act will require compliance procedures for safety of large GPT models which will come with the costs for the same. So, if a simpler architecture can give similar results without the cost then that is an advantage for business.
In many ways the era of GPT does not negate the timeless principles of good software engineering, instead it amplifies their significance in many ways.
Modular design, careful planning, and rigorous testing will pave way for easier integration of AI models with the rest of the software stack. User centered design principles and simplicity of interfaces becomes crucial for users to trust and easily navigate, even when AI technologies power them. So all the common best practices of design thinking and user journey mapping takes center stage when architecting these modern systems.
Rapid advancements and changes in the Generative AI landscape will only accentuate the need for traditional software engineering best practices to meet the demands of users in our AI-driven world.