Data Engineering Resources

I recently joined a company called SNC Lavalin as a Data Engineer. And this field is completely new to me. So I did a little bit of homework so that I can start picking up things and actively contribute to the upcoming projects.

Blogs

I have always found blogs to be incredibly useful in building up knowledge in any new domain. Following is a list of blogs which I have found useful.

• A Beginner’s Guide to Data Engineering — Part I
• A Beginner’s Guide to Data Engineering — Part II
• A Beginner’s Guide to Data Engineering — Part III
• Building Analytics at 500px
• Scaling Airbnb’s Experimentation Platform
• Using Machine Learning to Predict Value of Homes On Airbnb
• Introduction to Apache Spark with Examples and Use Cases
• Best Practices Writing Production-Grade PySpark Jobs

MOOCS

• Coursera
  • Big Data Specialization UC San Diego
    • Introduction to Big Data
    • Big Data Modelling and Management Systems
    • Big Data Integration and Processing
    • Machine Learning with Big Data
    • Graph Analytics for Big Data
    • Big Data - Capstone Project
  • Functional Programming in Scala Specialization
    • Functional Programming Principles in Scala
    • Functional Program Design in Scala
    • Parallel Programming
    • Big Data Analysis with Scala and Spark
    • Functional Programming in Scala Capstone
  • Big Data Engineering
    • Big Data Essentials - Spark RDD, HDFS and Mapreduce
    • Big Data Analysis: Hive, Spark SQL, DataFrames and GraphFrames
    • Big Data Applications: Machine Learning at Scale
    • Big Data Applications - Real Time Streaming
    • Big Data Services - Capstone Project
• Edx
  • Data Science and Engineering with Spark

Github Resources

• Awesome Spark
• Learn By Examples