Slot games analysys — large-scale data analytics with AI
Summary
This project is a data-intensive analytics platform built to absorb and make sense of very large volumes of multimodal data: image-derived artefacts, high-throughput text, and operational metadata. The system combines distributed ingestion and storage (cloud object storage, relational warehouse) with integrated AI—vision and language models—to extract structured signals that would be impractical to produce by hand. The architecture applies wherever unstructured media and text must be turned into queryable analytics at scale.
The data challenge
High-throughput business workloads generate massive, open-ended datasets:
- Volume — Long sessions produce thousands of image samples and large message corpora over time.
- Velocity — Data arrives continuously; analytics cannot assume inputs are complete before processing.
- Variety — Imagery, unstructured text, and metadata must be joined into a coherent picture for reporting and operations.
The core problem is not a single API call—it is operating a pipeline that stays correct under scale: storing artefacts durably, scheduling work across workers, applying AI selectively (budgeted model calls, sampling where full analysis is too expensive), and persisting results so analysts can query history without reprocessing raw sources for every question.
Solution approach
This platform treats the problem as big-data processing with AI in the loop:
- Ingestion at scale — Long-running jobs write high-volume artefacts to object storage and index rows in a relational database. Work is spread across containerised batch jobs and serverless orchestration so capture and analysis do not block each other.
- AI integration — After material is stored, dedicated processors run multimodal models (vision for visual content, language models for summarisation and structured extraction) with explicit budgets and confidence thresholds, so cost and provider limits remain under control at high data throughput.
- Downstream access — A web application exposes authenticated APIs over the warehouse so operators see timelines, detections, and derived summaries—turning raw volume into decision-ready views.
- Isolation of workloads — Other high-volume jobs (for example auxiliary web-derived pipelines) use separate containers and queues so failure or load in one line does not contaminate the core analytics path.
The design prioritises throughput, idempotent writes, and operational guardrails (retries, quotas, timeouts)—what you need when data volume and AI cost both grow together.
Technology stack (overview)
Outcomes
At a platform level (without claiming proprietary metrics):
- High-volume feeds are reduced to stored, queryable records and derived features suitable for analytics and operations.
- AI is applied where automation wins: classification and summarisation over volumes no team would label manually.
- The architecture scales horizontally across jobs and Lambdas while keeping a single source of truth in the database and object store.
