The Transformation Diaries Week 4: The Cloud Bill Nobody Wants to Talk About
Westbridge’s AWS bill keeps growing. But is that actually a problem? This week, Narayan follows the money and discovers that business growth, cloud efficiency and cloud waste can all exist at the same time. From 100+ NAT Gateways and 300+ VPC interface endpoints to unexplained spend and questionable architecture decisions, the real challenge isn’t simply reducing the bill. It’s being able to explain where the money is going, why it’s being spent, and whether it’s creating value.
Series: The Transformation Diaries - 52 Weeks Journey (10 articles)
- …
- 4The Transformation Diaries Week 4: The Cloud Bill Nobody Wants to Talk About This article
By Monday morning, Narayan had a much better understanding of Westbridge's cloud estate than he had a week earlier.
That wasn't necessarily comforting.
He and Raghu had spent the previous week connecting applications to infrastructure, infrastructure to owners, and owners to business units. They had found outdated diagrams, inconsistent tagging, unclear ownership and more than a few things that existed because, somewhere along the way, somebody had decided they were needed.
But one question had followed Narayan into the new week.
It had come from Daniel, Westbridge's CFO.
“If we're becoming more efficient, why isn't that showing up in the bill?”
This week, Narayan intended to find out.
The Bill Was Going Up. That Wasn't Necessarily the Problem.
The easiest conclusion would have been that Westbridge was spending too much on AWS.
Narayan wasn't ready to make it.
Westbridge itself was growing. New customers were coming onto its platforms. Existing products were processing more transactions. New services had been launched, more applications were moving to AWS, and teams were building capabilities that simply hadn't existed a year earlier.
All of those things consumed technology.
And technology cost money.
When Narayan sat down with Daniel, he started there.
“Before we decide whether the increase is a problem,” Narayan said, “we need to understand what's causing it.”
Daniel nodded. “I'm not asking you to make the bill smaller just so I can show a smaller number.”
That was useful to hear.
A company could grow its cloud spend by 15 percent while its customer base grew by 40 percent and actually be becoming more efficient.
The opposite was also possible. A flat cloud bill could look impressive until you discovered that the business had barely grown and half the infrastructure wasn't being used.
The number alone didn't tell the story.
The goal wasn't to make the AWS bill smaller at any cost. It was to understand whether the bill was growing for the right reasons.
One Bill. Several Stories.
Narayan and Raghu began breaking the bill apart.
The more they looked, the less useful the headline number became.
Some teams had genuinely reduced their infrastructure costs. Instances had been rightsized. Old resources had been removed. Commercial commitments had reduced the cost of predictable workloads.
Those savings were real.
But while one part of Westbridge was saving money, another part of the business was growing.
Narayan drew a deliberately simple example on the whiteboard.
Optimisation: -$500,000
Growth and new workloads: +$900,000
Net change: +$400,000
Daniel looked at it for a moment.
“So Finance sees the bill increase by four hundred thousand.”
“And Engineering can legitimately say they saved five hundred thousand,” Narayan replied.
Raghu leaned back in his chair.
“And everybody gets to leave the meeting convinced they're right.”
Narayan smiled.
Because they were.
The problem wasn't necessarily the numbers. It was that Westbridge didn't yet have enough visibility to tell the complete story behind them.
Some of the Numbers Didn't Make Sense
Then they started finding things that were harder to explain.
The first was networking.
Westbridge had accumulated more than 100 NAT Gateways across its AWS environment.
Narayan stared at the number for a moment.
In a typical hub-and-spoke model with centralised internet egress, the organisation could potentially operate with a much smaller footprint, perhaps three NAT Gateways spread across three AZs in the central egress VPC, depending on resilience, traffic and architecture requirements.
Westbridge had more than a hundred.
The cost was one problem. The architecture concerned him more.
Each NAT Gateway sitting in a decentralised VPC potentially represented another internet exit point. Instead of outbound traffic flowing through a smaller number of centrally governed and monitored egress points, Westbridge had internet egress distributed across dozens of VPCs.
That meant potentially 100+ exit points, majority of which might not be passing through Westbridge's central inspection and monitoring controls.
What had started as a cost question was beginning to look like a security and governance question as well.
Narayan turned to Raghu.
“Do we actually need these?”
Raghu looked through some of the network designs.
“Some, probably. A hundred? That's going to be a much harder argument.”
Then came the VPC interface endpoints.
There were more than 300 of them, providing private connectivity from individual VPCs to various AWS services.
Again, Narayan initially assumed there must be a reason. Perhaps the workloads were deliberately isolated from the internet. Maybe there was a compliance requirement that access to AWS services remain entirely private.
But neither appeared to be broadly true.
Many of the VPCs already had outbound connectivity, and Westbridge didn't have an organisation-wide compliance requirement that explained why hundreds of interface endpoints were necessary.
As Narayan moved between accounts, another pattern started appearing.
Different VPCs contained remarkably similar collections of endpoints.
He had seen this before.
Someone builds a good Infrastructure as Code module. It gets reviewed, approved and becomes the standard. The next team reuses it. Then the next. Eventually everyone becomes very efficient at deploying the same architecture without anyone going back to ask whether every component is still required.
Infrastructure as Code had solved the consistency problem. Unfortunately, it had also made it possible to consistently deploy things nobody had stopped to question.
There was nothing wrong with reusable modules. Narayan strongly believed in them.
But standardisation shouldn't replace understanding the requirement.
He opened a spreadsheet and did some rough calculations.
Between the NAT Gateways and VPC interface endpoints, the potential annual saving appeared to be more than $150,000 USD.
It wasn't a business case yet. They would need to understand traffic patterns, dependencies, availability requirements, security controls and the effort required to change the architecture before anyone started deleting things.
But the number was large enough to make a note of.
Networking potential $150K+ annual saving. Needs deeper assessment.
Narayan looked at the number for a few seconds.
Earlier in the week, they had been discussing how optimisation savings could disappear inside business growth, how Westbridge could save hundreds of thousands of dollars and still see its overall AWS bill increase by around $400,000.
He smiled.
Well, that's an interesting start.
He didn't call Daniel. There was nothing to announce yet.
Instead, he turned to Raghu.
“I think we've found some genuine savings potential.”
Raghu looked at the spreadsheet. “Looks like it.”
Narayan nodded.
“Getting there isn't going to be straightforward though.”
The technical changes were probably the easier part.
The harder conversation would be explaining why something that had been deployed for years, embedded in standard modules and repeated across dozens of accounts might no longer be something Westbridge actually needed.
Narayan remembered the distinction they had started making the previous week.
Need. Want. Inherited.
It seemed they were going to be using it again.
Finding $150,000 on a spreadsheet was easy.
Getting an organisation comfortable enough to change the architecture behind it was where the transformation actually started.
Then another service caught his attention.
CloudWatch.
The logging bill was surprisingly high. But the number deserved an explanation.
He added another note.
CloudWatch, investigate.
That one would eventually prove far more interesting than any of them realised that morning.
Sometimes the Boring Architecture Wins
Cost also started exposing architectural decisions that had nothing to do with idle infrastructure.
One workload caught Narayan's attention because of its Lambda usage.
There was nothing inherently wrong with Lambda. Westbridge used it successfully in plenty of places.
But this workload was different.
It ran predictably, on a schedule, performed a reasonably consistent amount of processing and didn't gain much from the event-driven scalability that made Lambda so useful elsewhere.
Narayan and Raghu worked through the usage.
Then they compared it with something considerably less exciting.
A small EC2 instance running a scheduled task.
The numbers were difficult to ignore.
Raghu looked at the comparison.
“Horses for courses.”
Narayan nodded.
There was a tendency in technology to assume that the newest architectural pattern must also be the best architectural pattern.
It wasn't.
Lambda wasn't wrong. EC2 wasn't old-fashioned. Containers weren't automatically better than virtual machines, just as Kubernetes wasn't automatically better than everything that came before it.
The right technology depended on the problem.
Modern architecture wasn't about choosing the most modern service. It was about choosing the service that made the most sense.
Sometimes that happened to be serverless.
Sometimes it was a container.
And sometimes the boring EC2 instance quietly won the argument.
Non-Production Was Surprisingly Good at Pretending to Be Production
Another pattern was beginning to appear.
Development and test environments were expensive.
Not individually. But across dozens of applications, the numbers accumulated quickly.
Some non-production environments ran twenty-four hours a day. Some had infrastructure sized surprisingly close to production. Others duplicated services and resilience patterns that made perfect sense for a customer-facing production workload but were harder to justify for a development environment nobody touched overnight.
Narayan raised it with one of the application teams.
“We need non-prod to look like production,” came the response.
It was a reasonable principle.
Narayan just wasn't sure it was always being interpreted reasonably.
“Does it need to look like production,” he asked, “or does it need to behave enough like production for us to test safely?”
There was a pause.
Narayan wrote that one down too.
It felt like a conversation for another week.
Who Does This $147,000 Belong To?
By Wednesday, the problem from the previous week had returned.
Only now it had a dollar sign attached to it.
Narayan was looking at a collection of resources that had consumed roughly $47,000 over the period they were analysing.
“What application is this?”
Raghu checked the tags.
One suggested an application.
Another pointed to a team that had been reorganised two years earlier.
The cost-centre tag was missing.
The application owner listed in an old document had moved to another part of Westbridge.
A few conversations later, they had three possible owners and no particularly confident answer.
Raghu stared at the screen.
“Apparently everything in AWS has an owner. Finding them is the optional feature.”
Narayan laughed, but the problem wasn't particularly funny.
Finance wanted better visibility into which parts of the organisation were consuming cloud services. Eventually, Westbridge might introduce stronger showback or chargeback so business units could understand the technology costs they were generating.
But there was a more basic problem to solve first.
You couldn't build a credible chargeback model on top of “we think this belongs to them.”
Before Westbridge could allocate cost, it needed to establish ownership.
The technical and financial problems were becoming increasingly difficult to separate.
Daniel Was Asking the Wrong Question. So Was Narayan.
By Thursday afternoon, Narayan had stopped asking why Westbridge's AWS bill wasn't going down.
It was becoming the wrong question.
Some of the increase was exactly what Westbridge should want to see. More customers. More transactions. New products. New workloads.
But mixed into that growth were very different numbers.
$147,000 of spend they couldn't confidently attribute.
100+ NAT Gateways they couldn't fully explain.
300+ VPC interface endpoints whose requirements were questionable.
And potentially $150,000+ a year in networking savings sitting underneath architecture that had simply become normal over time.
Daniel's original question came back to him.
“If we're becoming more efficient, why isn't that showing up in the bill?”
Narayan looked at Raghu.
“I think we've been looking at this backwards.”
“How so?”
“We've been asking how much we're spending.”
Raghu waited.
Narayan pointed at the numbers on the screen.
“We should be asking how much of it we can explain.”
That was the real difference.
A million dollars of additional cloud spend supporting a rapidly growing product might be perfectly reasonable.
A hundred and fifty thousand dollars spent on infrastructure nobody could explain was something else entirely.
The objective wasn't simply to spend less.
It was to spend deliberately.
By Friday, the Bill Hadn't Gone Down
By Friday afternoon, Westbridge's AWS bill hadn't moved.
Narayan considered that perfectly fine.
Nobody had rushed off to delete a hundred NAT Gateways. Nobody had removed three hundred VPC endpoints because a spreadsheet said they looked expensive. And Narayan certainly hadn't called Daniel to announce $150,000 of savings that hadn't actually been realised.
There was still architecture to understand, dependencies to validate and, perhaps hardest of all, people to convince.
Raghu looked at the growing list of findings.
“So... good week?”
Narayan looked at the NAT Gateways, the VPC endpoints, the unexplained $147,000, the CloudWatch question mark and the non-production numbers.
“Depends who you ask.”
He pointed at the potential networking saving.
“Finding the money was the easy part.”
Raghu smiled. “And the hard part?”
“Convincing everyone that what they want isn't necessarily what they need.”
Narayan closed the spreadsheet.
The bill hadn't gone down.
But for the first time, Westbridge was beginning to understand why it was going up and which parts of that growth it should actually be worried about.
A growing cloud bill wasn't the problem. An unexplained one was.
There was one other number Narayan couldn't stop looking at.
Non-production.
For something that everyone insisted was different from production, it was doing a surprisingly good job of costing like it.
Narayan smiled.
That could wait until Monday.
Next week: Week 5 Production Is Different… Apparently.
Series: The Transformation Diaries - 52 Weeks Journey (10 articles)
- …
- 4The Transformation Diaries Week 4: The Cloud Bill Nobody Wants to Talk About This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article