12 min read

Is Local AI Worth It? I Ran a Small Business on a £300 GPU

I ran a made-up UK pet shop for a day on one small local AI and a second-hand graphics card, then compared its accuracy and running costs with Claude in the cloud.

Craig Moore

Founder, Apex Insights

An imagined cartoon of a fluffy blue puppy beside the words aPets Insights in blue and gold

AI-generated illustration by Craig Moore

In this article

Modern businesses have a lot of moving parts. Keeping track of how they fit together, and keeping the day-to-day running smooth, can take a great deal of work. What if much of that routine work could be automated, with an AI reading and sorting everything that comes in? And could we go one step further, running it all on our own hardware, with inexpensive kit and free software, and still have it work reliably, accurately and safely?

And why would anyone bother? Would it save money? Would it mean no more worrying about data protection? Would it be any more secure?

Few businesses have gone that far. Only 10% of UK firms with 10 or more staff that use AI say they use it extensively (ONS). So what holds owners back? Among small business owners not using AI, the top reasons are worries about security and privacy (44%), no clear use for it (39%), not knowing how (37%) and doubts about accuracy (36%) (IT Brief). Can we address all of these concerns, and show that it can be secure, that there are clear uses for it, that it is simple to set up, and that it is accurate?

To find out, I built a fictitious pet shop called aPets Insights, and yes, I am quite proud of that pun. I deliberately built it on an old PC. Its graphics card, the part that runs the AI, sells second-hand for about £300 (Koukan).

The plan is ambitious. One small AI reads and sorts the work for the whole shop, from the website and email to the bank. Nothing is sent to an outside AI company, and apart from the hardware and its electricity, not a penny goes on subscriptions or software. Then I marked it like a test, to see how it measures up on accuracy, cost, data protection and security.

The full build, on video

For anyone who wants the detail, I go through the full build behind aPets Insights, and the results, in a video on YouTube.

Video. The aPets Insights build and test results, on YouTube, on YouTube

The plan for aPets Insights, built on n8n

The piece that makes all of this work is n8n. It is a platform for building automated workflows, passing each piece of work from one step to the next, and it connects to more than 1,500 other apps and services (n8n). It can run on your own computer, and its licence lets a business use it for its own internal work without paying (n8n licence).

The n8n home page, showing a refund workflow as a row of connected stepsAn example workflow on the n8n home page, n8n.io

That matters, because the hardware is limited. aPets Insights runs on one old PC with a 10GB graphics card, which has room for a single small "local AI" model to handle every job. So the AI gets small, clear tasks, such as sorting an email or reading an invoice, and n8n connects everything else.

The AI reads and sorts each piece of work. The shop's rules, written in code, decide what happens next, and anything the AI is unsure about goes to a person. Here is what each part of the shop does.

  • Website, a working online shop where customers browse and order
  • Control panel, a staff dashboard showing the whole day as it happens, with the items that need a person picked out
  • Email, where the AI sorts every message into one of 9 types, from sales leads and complaints to spam
  • Social media, where the AI sorts comments, and an angry public complaint is marked as urgent
  • Live chat, where the AI looks up orders and stock, and hands over to staff when it cannot help
  • Bank, where a supplier is only paid when its invoice matches a purchase order the shop sent. This fraud check flags a wrong amount, and treats an invoice with no matching order as possible fraud
  • HR, where sick notes and leave requests are sorted and logged on the staff rota, and leave is only approved when there is cover
  • Deliveries, where a simulated logistics department, not the AI, takes each order through packed, dispatched and delivered

The aPets Insights shop website, showing pet products for saleThe aPets Insights shop website

The aPets Insights control panel, a timeline of the whole day's workThe aPets Insights control panel

That is the plan. Whether one small AI on one old PC can really handle all of it, and get it right, is what the test set out to find.

How accurate the AI was in the test

I gave the shop's AI 100 new tasks, from emails to live chat messages. As a test, 1 in 10 of them could not be resolved by any AI and had to go to a person, to see whether it would guess instead.

Of the 90 tasks it could handle, the AI got 87 fully right first time. It was unsure about the other 3, and passed them to a person rather than guess. So not one wrong answer went out, but a person still had to deal with 13 of the 100, about 1 in 8.

I then ran the same 100 tasks through three of Anthropic's paid Claude models, Haiku, Sonnet and Opus, all version 5.5. Each got all 90 right and passed the other 10 to a person. On this test, the biggest model did no better than the smallest.

Tasks fully right first time, out of 90 an AI could handle

This article

Own test of 100 made-up tasks. The other 10 needed a person by design, and every model passed them on. No model gave a wrong answer.

What it costs to run

The main costs are the hardware and its electricity. Used RTX 3080 cards sell for about £300 in the UK (Koukan). The card is rated at 320 watts (NVIDIA). Run flat out for 8 hours a day, 22 days a month, at the current price cap of 26.32p per kilowatt hour (Ofgem), it would cost about £15 a month, or about £180 a year. That is the card alone, at a household rate, so it is a rough guide rather than a bill.

The software costs nothing. n8n is free for a business's own internal use (n8n licence), and Docker Desktop, which runs it all on Windows, is free for businesses with fewer than 250 employees and less than $10 million in annual revenue (Docker). The AI model and the software that runs it are both under the Apache 2.0 licence, which allows business use (vLLM, Qwen).

So what would the same shop cost on paid cloud services instead?

The AI's bill is where the surprise lies. Running one simulated business day's work through Claude Haiku 5.5 instead cost about $0.025, at its published prices (Anthropic). Over the same 264 working days a year, that is about £5, at the Bank of England's rate of $1.3384 to the pound on 21 October 2026 (Bank of England). Sonnet 5.5 would cost about £96 a year, and Opus 5.5 about £207.

What the shop's AI costs to run for a year

This article

264 working days a year. Electricity is the card alone at full power, at the household price cap, and leaves out the card's own price. Cloud costs are one simulated day's measured use at published prices.

So on running costs alone, the card's electricity costs more than 30 times Haiku's bill, before the card itself is paid for. That fits what research has found. A University of Vaasa study of 18 AI models run locally found that "online services were more cost-effective for almost all tested models", and that the stronger reasons to run AI yourself are "privacy and price risk mitigation" (University of Vaasa).

Keeping customer data on your own computer

Privacy was the top worry among owners not using AI, at 44% (IT Brief). Customer data never leaves the PC, so there is no "restricted transfer" to an organisation outside the UK, which UK GDPR would otherwise allow only under adequacy rules, safeguards or an exception (ICO).

But the law still applies. Personal data must be kept secure (ICO), and the regulator's guidance, now under review, says using AI will "in the vast majority of cases" need a data protection impact assessment (ICO).

Security also falls to the business. Researchers found 175,000 self-hosted AI servers left open to the internet (The Hacker News), and 42% of micro businesses and 46% of small ones reported a cyber breach or attack in the past year (DSIT). Local AI removes one privacy risk, but the machine has to be kept locked down.

What this test can't tell us yet

This is one test, and it has limits. It used one made-up shop, one AI model and one graphics card, so a different business, model or card could give different results.

The tasks in the test are made up, each with its right answer written down, so the score is only as good as those answers. The shop's rules were also tuned while it was built, so a real inbox could be harder, and the cloud costs come from one simulated day.

In summary

So, back to the opening question. In the test, the AI got 87 of the 90 tasks it could handle fully right, and passed everything else to a person, without a single wrong answer. It did that on a graphics card of about £300 that costs about £15 a month to run.

To be fair, what we asked of it was ambitious. Some of the test was hard by design, and a member of staff might have got some of it wrong too. In all honesty, I expected it to do far worse.

It was not simple to set up, though. It took real work to build, and a person still has to check about 1 in 8 tasks. Nor does it save money. Claude Haiku did slightly better, for about £5 a year. The real reason to run it locally is control. The data stays on your own machine, and the bill will not change on you.

For my own business, my current approach is cloud AI. As long as you meet your UK GDPR duties, it has real advantages. There is no equipment to buy or look after, and it is generally fast. You also get the biggest models, and you are not limited to one. Each job can go to the model best suited to it, such as a decision model like Jev for sorting work. The provider also handles much of the security work.

There are downsides. Costs grow with every token and prices can change. Customer data leaves your building, so UK GDPR rules follow it. And if the internet or the provider goes down, the work stops.

This test was about whether running it all locally was possible. In practice, most businesses would probably be better off using cloud AI and paying for the tokens they use.

If you take anything away from this article, I want you to think about this. The best AI on a single gaming card does what the world's best AI did 6 to 12 months earlier (Epoch AI). Epoch AI does warn that real work may lag further, and it tested bigger cards than ours.

If that pace continues, AI as good as today's best could, in theory, run on a second-hand card like ours a year from now. Would we trust it with big decisions, where one mistake could do real damage? No. But I am sure most businesses reading this could name a handful of workflows that would benefit from cheap, fast decision-making.

Sources(21)

Show

Responsible use of AI

Every article here is researched, written and edited by me. I use AI tools to help me find sources, to structure the article, and check spelling and grammar. The ideas and final wording are my own. Where possible, I follow the ten commitments of the Responsible AI Use Campaign.

The views and opinions in these articles are my own. They do not represent any organisation I work for or am affiliated with.

See my public declaration

You stay anonymous on this site.

We count visits to help us improve it. No tracking or advertising cookies, and we never follow you to other sites. Visits are anonymous, apart from personal video links we send. Cookie Policy