Recently Amazon CTO Werner Vogels gave a keynote talk at AWS re:Invent 2023 where he spoke about the topic of the frugal architect - this happens to be a passionate subject for me!
So I will attempt to distill his key lessons and elaborate few of my thoughts and commentary on the same.
As software systems get more complex and the advent of large foundation models, the cost of building and maintaining software can get prohibitively high if not planned properly.
Cost is also a good proxy for sustainability, as it is difficult to measure milligrams of CO2 emitted by functions and services.
Frugal architect should apply some key principles or laws to design, measure, and optimise modern software systems.
Cost should be treated as a non-functional requirement (NFR) just like security, performance, or scalability. Along with cost, sustainability must also be a first class NFR.
Pricing in today’s cloud era is pay-as-you-go, unlike the old days of upfront costs. So, the total cost of ownership is the factor to consider.
There are costs based on size of packets transferred, storage, and number of requests. Cost must be factored in every step of design as against the commonly seen afterthought.
If you want your system to last a long time then the cost should align to business value and its profit levers. Architecture must follow the money.
For example when building a retail website, bigger catalog means more visits, and more orders. This will then create a flywheel of economy of scale, lower costs, and an even bigger catalog, and even more orders.
One should also not let technical debt pile on like kitchen sink, else this can compound and get more costly down the road.
Architecture is basically about tradeoffs. Cost and sustainability must balance against functionality, performance, security, and even regulations.
Sometimes these factors may even reinforce each other. For example, as the stability of a system improves then the cost may also come down over time due to less effort in support.
Systems should support good observability of the metrics that matter. Modern cloud platforms allow you to track the cost per micro-service request as well as at the overall systems level. This can then be checked against the amount of requests served, conversions done, and orders submitted.
Over time the business value must increase and the costs go down - by means of re-architecting, moving to more efficient hardware, and other platform enhancements.
Evolvable architecture is the key, so you can observe and improve it over time.
Cost-aware architecture must have cost controls or what is called as “tunable architectures”. Let’s say the cost of a certain component goes too high then tech should provide knobs and dials that business can use to turn that off.
Frugal architect must also determine the absolutely critical functionality that has to be up and running predictably all the time. For example, in a retail site like Amazon the catalog and shopping cart cannot have any downtime.
At a time when developers were building mobile and embedded systems a decade ago, we used to be very sensitive to tight resource constraints. It is time to bring that mindset of tinkering at a small level again. Identify the dead code and digital waste lying in your system, and reduce the same. Something as simple of turning off the development systems at night can save lot of energy and costs. Other common sense actions could include seeing things like downloading large images over network and then downsizing it in display, and instead reduce size at source itself before transmission.
Use code profiler to monitor exactly where the function call time is being spent on. Unexpected surprises may reveal and these could be fixed. Builders should be curious and dive into the details of their resource utilisation.
Often times the cost of building software is dwarfed by the cost of maintenance. So it becomes very important to observe the operations at runtime continuously.
Energy efficient programming language like Rust may be considered over Python as the former is 50 times more efficient in addition to good security, strong typing, and memory safety.
Transformer architecture is the most revolutionary one we have seen recently. And although the large foundation models are on the scene, good old fashioned AI is also very much relevant today.
Many businesses want small, fast, inexpensive models on edge devices to do mission critical predictions and guidance.
Today we have mountain of unstructured data. The only way to make sense of this is to use the magnet of machine learning to find that needle in this haystack.
Lastly, tech landscape is also changing very fast so we should constantly learn and evaluate smarter ways of doing work.
For example AI based coding assistants are now part of many cloud based IDE to help developers improve productivity and be a helpful partner in the development endeavours.
Do more with less and do it better.