Cheaper, faster machine learning is revolutionizing computer vision

Inference is the process of applying a trained AI model to new data—essentially, making predictions or drawing conclusions. Until recently, the focus was on the cost and complexity of training models, which required massive datasets and specialized hardware.

In 2017, running an image recognition model on a cloud GPU could cost about $1 per 1,000 inferences. At scale, this was expensive. With the evolution of GPU hardware, optimized software libraries, and model compression techniques (quantization, pruning, knowledge distillation), the cost has dropped significantly. Today, inference cost for a similar image model can be as low as $0.01 per 1,000 inferences. Improved scaling, cheaper compute, and more efficient model serving architectures are some of the factors that contributed to the reduction in costs.

Google’s TPUs (Tensor Processing Units) and NVIDIA’s Tensor Cores (such as the A100 and H100) accelerate the calculations typically used in deep learning models. As both these technologies support parallel processing of data, they enable cost-efficient handling of large datasets.

Focus is shifting to applied AI

The drop in inference cost is not just an efficiency win—it’s a paradigm shift. Enterprises and startups that once balked at the ongoing expense can now deploy AI-powered services at unprecedented scale and affordability. Democratization of AI has made the power of AI accessible not just to Big Tech, but to small businesses, researchers, and even hobbyists.

Applied AI, which involves machine learning that is built directly into products, workflows, and service, is already reshaping industries. The global AI market is projected to hit $407 billion by 2027, up from $86.9 billion in 2022. Much of this growth will be driven by applied, not theoretical, AI—real-world solutions that depend on inference at scale.

SAI Group’s visual AI: affordable, and reliable

SAI Group’s visual AI platform is a great example of applied AI being adopted by businesses. Deployed in store networks of 3 of the top 5 retail chains in the UK, SAI Group’s visual AI platform provides:

• Instant detection of aisle theft: Instantly detect when customers conceal items in their clothing or bags.
• Seamless Self-Checkout (SCO) integration: Compatible with a range of self-checkout systems for straightforward multi-vendor integration.
• Subtle but powerful: Delivers real-time prompts to the SCO monitor, enabling customers to correct errors on their own, without the involvement of the store staff.
• Instant non-payment detection: Instantly identify when customers avoid payments at the SCO.
• Empowered store staff: Keeps the store staff informed by sending notifications on the store’s hand-held terminals.
• Violence detection: Utilizes AI to detect incidents of violence within the store and sends alerts directly to the security operations centre so that resources can be mobilized appropriately.
• Legally admissible evidence: Automatically extracts and stores the video footage of incidents detected, thus enabling the store staff to report theft more easily and efficiently.

About SAI

As a leader in computer vision technology, SAI Group delivers cutting-edge, multi-modal AI solutions into retail environments. Using a unique platform approach, its technology uses existing camera systems to target losses, increase store safety, and underpin operational efficiencies.

All solutions are built from the ground up to ensure the highest levels of security and data protection, respecting the privacy expectations of the public and operating to stringent ethical standards while delivering substantial value to our clients. Globally, SAI monitors millions of transactions per day, protecting the revenues from tens of millions of product sales and hundreds of millions of customer interactions. Its models also accurately identify anti-social behaviour, aggression and violence, helping to de-escalate situations with real-time interfaces to security officers and operations centres.