Building a Big Data environment with Spark integrated with R on Azure

on YouTube ↗42m

" summary

Shows how to build an Apache Spark Big Data environment on Azure by provisioning a VM and configuring Spark to work with the R language. Covers integrating R-based machine learning workflows with Spark so R can interact with large-scale data on Azure.

AI-written summary · about

" chapters (11)

1. 0:00Introduction
2. 2:36Big Data's three Vs
3. 7:30Cloud service models
4. 13:04Spark tools and ecosystem
5. 18:43Local environment setup
6. 26:16R integration with Spark
7. 32:01Connecting to remote Spark
8. 36:41Data processing results
9. 37:22AI research in Brazil
10. 38:22Job opportunities
11. 39:27Closing and Q&A

" related talks

• 42mScalable Data Science in Python and R on Apache SparkFelix Cheung · PyData Seattle 2017
• 49mBuilding Advance Analytics pipelines with Azure DatabricksLace Lofranco · NDC Sydney 2018
• 51mSpotlight on Emerging Data Science Worskspaces and Spark as Development LeversCarlo Appugliese · ApacheCon North America 2017
• 48mMachine Learning in Azure DatabricksTerry McCann · SQLBits 2020
• 30mData & AI platform components in AzureRahat Yasir · Big Data & AI Toronto 2021
• 34mMaking Data Scientists Productive in AzureValdas Maksimavičius · Big Data Conference Europe 2019
• 37mPracticing Data Science in the cloudDr. Robert Hoffmann · WeAreDevelopers AI Congress Vienna 2018
• 35mPySpark : Combining Machine Learning & Big DataAyon Roy · Data Con LA 2020