amazon-emr

⛳️ PASS: Amazon Web Services Certified (AWS Certified) Machine Learning Specialty (MLS-C01) by learning based on our Questions & Answers (Q&A) Practice Tests Exams.

Reference Architectures for Datalakes on AWS

Jupyter notebooks and AWS CloudFormation template to show how Hudi, Iceberg, and Delta Lake work

A command-line interface for packaging, deploying, and running your EMR Serverless Spark jobs

Project files for the post: Running PySpark Applications on Amazon EMR using Apache Airflow: Using the new Amazon Managed Workflows for Apache Airflow (MWAA) on AWS.

A VS Code Extension to make it easier to manage and develop Spark jobs on EMR

Project files for the post: Running PySpark Applications on Amazon EMR: Methods for Interacting with PySpark on Amazon Elastic MapReduce.

Bits of code I use during live demos

Run templatable playbooks of Hadoop/Spark/et al jobs on Amazon EMR

Amazon EMR Notebook to show how to read from and write to Delta tables with Amazon EMR

3NF-normalize Yelp data on S3 with Spark and load it into Redshift - automate the whole thing with Apache Airflow

⛳️ PASS: Amazon Web Services Certified (AWS Certified) Data Analytics Specialty (DAS-C01) by learning based on our Questions & Answers (Q&A) Practice Tests Exams.

This repo provides cross-account integration code samples using Amazon S3 Access points

Sample CI/CD pipeline for using GitHub Actions with Amazon EMR Serverless Spark.

Quant Research stack using EMR on EKS

Orchestrate an Amazon EMR on Amazon EKS Spark job with AWS Step Functions

📓 Repository/Tutorial for initiallizing Jupyter Notebook and Spark cluster on Amazon EMR

Configure Hadoop YARN CapacityScheduler on Amazon EMR on Amazon EC2 for multi-tenant heterogeneous workloads

Samples related to data engineering, e.g. spark, embulk, airflow, etc.

Root-cause analytics for sales and supply ops. Traces KPI moves to the segments behind them, separates rate from mix, forecasts demand. Runs on 1M real invoice lines via S3, PySpark, PostgreSQL and Redshift, with a Power BI report; Spark job ready for AWS Glue and EMR.