Why Running an LLM Locally Taught Me More About AWS Pricing Than the Billing Module Did

This Hacktoberfest, I am creating a small project for a friend in an open source AI competition, a tool that will effortlessly collect all of their browsing and AI chat history into one easy-to-reach unified storage, similar to Obsidian but automatically filled with information. The idea was to include open-source within your project, which made my model selection rather straightforward. I went with a local model powered by Ollama, as the concept designed around open-weight models didn't leave me any other alternative if I wish to keep my project simple and manageable for my friend with no need for him to deal with APIs and payments.

Yet, after a few days of studying the pricing of Cloud Practitioner's course I found myself experiencing the very situation that the course was describing. The pricing system of AWS requires you to pay for what you use, and it only becomes clear to you once you experience the alternative situation. Local deployment of Ollama implies almost nonexistent cost of using this model, rather the cost of my own computer in which I am using this technology.

That's why, as I read through the on-demand EC2 pricing or the pay-per-invocation pricing for Lambda, my initial response was not "this is too expensive", but "ahh, now I can see how much we have been paying all along, and now it is done by someone else instead of us". The shift in thinking mattered more than I thought it would. It is easy to see a per hour rate for EC2 as a new payment. In reality, it is a payment that has always existed in some form, be it power bills, hardware depreciation, and time spent managing the infrastructure. We think of local as free because the expenses are physical and sometimes not even known to a user and in cloud computing everything is very transparent and itemized.

It also explains why there is free tier like that in the case of a student. A student project that will not generate any traffic does not need to think about the cost. It deals with the use of many simultaneous requests, the guarantee of uptime that an individual computer can't give, therefore, an itemized AWS bill no longer is considered a burden, but now is the reason for being able to grow.

I performed the project on my local machine because the description stated so, not due to the fact that I gave the topic a lot of thoughts. However, creating something real with genuinely free inference while learning how AWS prices the alternative gave a tangible idea of the trade-off between the two types of charges.