Arocom IT Solutions Pvt. Ltd.

Scalable Serverless AWS Architecture for Dash Cam Data Processing

Client

A technology company specializing in dash cam solutions and fleet management systems, launching their new dash cam product line.

Challenge

The client’s existing architecture faced significant scalability limitations and performance issues that could not support their new dash cam product launch. The legacy system used a prototype FTP solution that was inadequate for handling the projected tenfold increase in data volume from dash cam devices. The current infrastructure could not efficiently process and store large volumes of video footage and telemetry data including GPS coordinates, event flags, and other metadata. Additionally, the system needed to handle network connectivity interruptions and accommodate dash cam video capabilities while maintaining reliable data processing and retrieval. The client required a complete architectural overhaul to separate the new product’s data processing from the legacy system and ensure optimal performance, reliability, and cost-effectiveness.

Key Results

  • Eliminated scalability bottlenecks and supported a projected 10X increase in data volume capacity
  • Reduced data processing latency by 50% through serverless architecture implementation
  • Improved system reliability by 70% with automated queuing and processing pipelines

Solution

Arocom designed and implemented a comprehensive serverless AWS solution that completely replaced the existing prototype FTP architecture. The team developed a private API Gateway to securely receive video and data streams from dash cam devices, ensuring secure and reliable data transmission.

The solution included a sophisticated video processing pipeline using AWS Lambda functions that transcoded video files as needed and stored them securely in Amazon S3 with relevant metadata and efficient organization for optimal retrieval and

management. A separate processing pipeline was created for non-video device data including GPS coordinates, event flags, and other telemetry metadata.

The architecture implemented Amazon SQS queuing system to handle large volumes of incoming data and ensure reliable processing even during network connectivity interruptions. Lambda functions were developed to process queued data and store it in the appropriate Aurora Serverless.

The team configured and optimized S3 storage for video files with appropriate lifecycle policies based on retention requirements and established connectivity with the client’s existing VPN connections to cellular providers including Verizon and AT&T. The entire architecture was designed with automatic scaling capabilities to handle the projected tenfold increase in data volume while accommodating dash cam local storage capabilities and network interruption scenarios.

Technologies Used

  • AmazonWeb Services (AWS)
  • AWSLambda
  • AmazonS3
  • AmazonSQS
  • AmazonKinesis
  • AmazonSNS
  • AmazonSES
  • AmazonAurora RDS
  • AmazonECS Fargate
  • .NET
  • Python
  • ApplicationLoad Balancer (ALB)
  • NetworkLoad Balancer (NLB)
  • Site-to-SiteVPN

Summary for Website Card

Arocom developed a scalable serverless AWS architecture for a dash cam technology company, replacing their legacy FTP system with a comprehensive solution featuring ESE Fargate, Lambda functions, S3 storage, SQS and Kinesis queuing to handle video footage and telemetry data processing. The implementation supported a projected tenfold increase in data volume while providing automated scaling, reduced latency, and improved reliability for their new dash cam product line through modern cloud-native data processing pipelines utilizing Aurora RDS, ECS Fargate, and site-to-site VPN connectivity.