Cloud Computing Field Training Project

Vision-Based Retail POS Automation

An AI-powered local object detection system extending Open Source POS (OSPOS) to solve unreadable, missing, and anti-counterfeiting changing barcodes.

Team Members

• Marchelino Mina

• Elarya Ehab

• Mariane Magdy

Supervisors

• Dr. Aya Ibrahim

• Dr. Maged Fathy

• Eng Aly Ahmed Sewelam

Presentation Agenda

Part 1: Core Foundation

01. Problem & Motivation: Barcode failure & retailer challenges

02. Project Objectives: Specific measurable targets

03. System Architecture: AWS, Docker, YOLO & OSPOS

04. Environment & Tools: Hardware & cloud stack

Part 2: Execution & Results

05. Implementation Flow: Pipeline setup to deployment

06. Live Demo & Workflow: End-to-end POS API integration

07. Results & Challenges: Accuracy & engineering solutions

08. Future Improvements: Scalability & Conclusion

Problem & Motivation

The Problem

Unreadable / Missing Barcodes: Damaged labels, domestic items without barcodes, or glossy reflective surfaces fail on standard scanners.

Anti-Counterfeiting Rotations: Manufacturers constantly change barcode formats, causing registration errors at cash registers.

Retail Bottleneck: Cashiers manually keying in SKU codes slows checkout lines and introduces human error.

The Motivation

Automated Checkout: Create an intelligent, computer-vision assistant that recognizes products by visual features regardless of barcodes.

Seamless POS Integration: Direct API injection straight into existing Open Source POS (OSPOS) basket.

Nationwide Scale: Build a low-cost, client-agnostic model adaptable to any physical retail store.

Project Objectives

1. Train Local YOLO ML

Train a local YOLO object detection model specifically on domestic products frequently lacking clear barcodes.

2. Extend OSPOS API

Build a custom PHP extension for OSPOS exposing endpoints for external apps to push items directly to cart.

3. Deploy Cloud Infrastructure

Host OSPOS & CVAT in isolated Docker containers on AWS EC2, secured via Nginx, SSH keys & Let's Encrypt SSL.

4. Evaluate Hardware Setup

Design an artificially lit under-counter camera setup to guarantee high visual accuracy (>90%).

System Architecture

AWS EC2 Cloud & Container Stack

Edge Camera Hardware: Under-counter camera captures product, fed into Dell G15 local YOLO wrapper service.

YOLO Inference Engine: Detects product visually, resolves corresponding barcode/SKU instantly.

RESTful API Bridge: Sends async HTTPS POST payload to the custom OSPOS API endpoint.

Cloud Container Deployment: Nginx Reverse Proxy handles HTTPS requests and routes traffic to OSPOS & CVAT Docker containers on AWS EC2.

Cloud Architecture Microservices Docker EC2

Environment & Technologies

Layer / Platform	Technology Used	Role & Technical Purpose
Training Workstation	Dell G15 (NVIDIA RTX 3050 Ti GPU)	Local deep learning acceleration for YOLO ML training
Cloud Server / OS	AWS EC2 / Ubuntu Linux	Cloud hosting environment with cryptographic SSH security
Containerization	Docker & Docker Compose	Isolated microservices for OSPOS app & CVAT instance
ML & Annotation	YOLOv8 & CVAT Tool	500+ annotated domestic product frames divided across team
Core POS & API Extension	OSPOS (Open Source POS) + Custom API	Exposes REST API endpoints to manipulate active cart
Networking & SSL	Nginx Reverse Proxy & Let's Encrypt	Domain routing, SSL/TLS certificate, secure HTTPS API access

Implementation Flow

01. Annotation

Dataset creation (500+ frames) annotated via CVAT on AWS EC2.

02. Training

YOLO ML model training on Dell G15 RTX 3050 Ti GPU workstation.

03. API Extension

Developed OSPOS extension providing "add item to basket" REST endpoint.

04. Hardware Rig

Built under-counter camera rig with dedicated artificial light.

05. Cloud Deploy

Configured Docker, Nginx reverse proxy, & Let's Encrypt SSL on AWS.

06. Testing

Verified sub-second visual item scanning & basket injection.

Live Demo & Operational Workflow

Step 1: Container Check — Docker containers (OSPOS & Nginx) active on AWS EC2.
Step 2: Item Presentation — Unbarcoded product placed under illuminated camera.
Step 3: YOLO Inference — Python wrapper detects item & fetches associated barcode ID.
Step 4: API Payload — Wrapper sends POST call to OSPOS backend endpoint.
Step 5: Basket Updated — Item automatically registers in cashier's OSPOS checkout.

Results & Performance Evaluation

>90%
Detection Accuracy Achieved

                    Controlled Lighting Impact: Installing under-counter artificial lighting eliminated glare, boosting accuracy from ~65% to over 90%.
Sub-Second Latency: Product recognition to OSPOS cart injection completed in under 800ms.
Dataset Efficiency: 500+ annotated frames across troublesome items proved sufficient for precise domestic product classification.

                

Challenges & Engineering Solutions

1. Variable Ambient Light

Problem: Store reflections caused false positives.

Solution: Custom under-counter enclosure with fixed artificial LED illumination.

2. Unexposed POS API

Problem: Native OSPOS lacked external cart modification endpoints.

Solution: Developed custom OSPOS PHP API extension for reliable external POST calls.

3. Cloud Security

Problem: Securing live retail endpoints over public internet.

Solution: Nginx reverse proxy, SSH cryptographic keys, and SSL encryption.

Future Roadmap & Improvements

1. Cloud-Hosted Inference

Transition from local GPU workstation to cloud GPU instances for client-agnostic processing nationwide.

2. Automated MLOps Pipeline

Implement continuous training pipelines to auto-label new inventory items uploaded by store managers.

3. Edge Hardware Optimization

Deploy lightweight TensorRT models on edge microprocessors (e.g. NVIDIA Jetson) for offline operation.

Thank You!

A sustainable computer-vision platform solving unreadable barcode challenges for modern retailers.

Cloud Computing Field Training

Questions & Answers

Image Sources

https://miro.medium.com/v2/resize:fit:2000/1*Z8kZQj7Rpjbk4hfaAZcxGQ.png
Source: medium.com
https://moschip.com/wp-content/uploads/2026/03/Building-a-vision-based-self-checkout-solution.webp
Source: moschip.com