We have been working in-memory OLTP Technologies to achieve high concurreny. Our platform needs to support about 500 K Concurrent users doing small ( less than 64 KB) reads/writes at high frequency. We have noticed LATCH, LOCK and WRITE Log waits using disk based tables and we are able to avoid those issues using in-memory OLTP in Azure SQL and we are able to achieve the high concurrency.
I was hosting http://h1bsalary.online site from AWS on m3.xlarge Instance. Even though its reserved instance , the cost of hosting has crossed the threshold limit of hobby project. So I have decided to host the site from Home . In this blog post, I am sharing my experiences with hosting the website from Home instead of public cloud provider.
We typically get data feeds from our clients ( usually about ~ 5 - 20 GB) worth of data. We download these data files to our lab environment and use shell scripts to load the data into AURORA RDS . We wanted to avoid unnecessary data transfers and decided to setup data pipe line to automate the process and use S3 Buckets for file uploads from the clients.