What is a Data Lake? A Data Lake is a centralized repository for large amounts of structured and unstructured data to enable direct analytics. The above image illustrates multiple functions of a data lake, such as data ingestion, data indexing, data processing, machine learning, and data visualization. Data Lakes provides the ability to harness more … Continue reading Build a Data Lake using Lake Formation on AWS
Tag: Cloud
Online Data Migration from SQL Server to Cloud Spanner using Striim
This post will focus on the implementation of a continuous migration from SQL Server to Cloud Spanner using Striim. The explanation will include steps for configuring and initial load and a continuous data replication using Change Data Capture. this is enablement post and doesn't talk about a full scale database migration and its challenges. This … Continue reading Online Data Migration from SQL Server to Cloud Spanner using Striim
Custom Email Notification In Azure Data Factory
Problem How to send an email from Azure Data Factory pipeline with a custom message body using gmail as email provider? Azure Data Factory does not allow us to send emails. It is possible to send alerts (emails) in ADF but these do not provide any clarity\customization on the message subject and body. Recently, I … Continue reading Custom Email Notification In Azure Data Factory
How I hosted my portfolio website on Amazon S3?
Background I was looking to build my own portfolio website for some time now but the problem was that I am not a UI Developer and I always wanted a kick-ass website. Long story short, I got nowhere with it until recently, when I was working with a customer to help them move their e-commerce … Continue reading How I hosted my portfolio website on Amazon S3?
Build a CI/CD pipeline using AWS developer tools
Building and delivering software isn't easy. Everyone wants to have a consistent quality and to be able to deliver new functionality quickly. In this write-up, I will demonstrate Continuous Integration and Continuous Delivery for a Simple .NET application using AWS Developer tools. Assumptions - I am assuming that you have prior working knowledge of AWS … Continue reading Build a CI/CD pipeline using AWS developer tools
The 'Cloud' Workbook 