BIG DATA TOOLS - PART 2

Code Cours
2324-IÉSEG-MBD1S2-MIS-MBDCE07UE
Langue d'enseignement
English
Matières
MANAGEMENT OF INFORMATION SYSTEMS
Ce cours apparaît dans les formation(s) suivante(s)
Responsable(s)
S.HOORNAERT
Intervenant(s)
S.HOORNAERT
Niveau
-
Année de formation
Période

Présentation

Prérequis
- Participants should be familiar with the basic concepts of R (e.g., vectors, dataframes, functions, packages).
- Participants should be familiar with reading and writing SQL queries (e.g., select, group by, having).
- Participants should know the basic concepts of Business Analytics and Predictive Modeling.
Objectifs
At the end of the course, the student should be able to:
- understand the available technologies in the Big Data universe and use the correct technology for a given Big Data problem
- know the technologies for reading and writing Big Data (e.g., MapReduce, Hadoop, HDFS, Parquet)
- know Spark, its architecture and its APIs
- use Spark as a tool for descriptive and predictive analytics using Spark SQL, MLlib, Streaming, and GraphX
- solve and present an end-to-end solution to a Big Data problem in an intercultural team

These competencies and/or skills contribute to the following learning objectives
- 1.B Successfully collaborate within a intercultural team
- 3.A Breakdown complex organizational problems using the appropriate methodology
- 4.B Compose constructive personal feedback and guidance
- 4.C. Convey powerful messages using contemporary presentation techniques
- 5.B Construct expert knowledge from cutting-edge information
- 7.A Demonstrate an expertise on key concepts, techniques and trends in their professional field
Présentation
Every day, 2.5 quintillion bytes (=2.5*10^18 bytes) of data are created. Every minute, more than 4.2 million posts are liked and 300 hours of videos are uploaded. This generated (Big) data is characterized by its volume, variety, velocity, and veracity, and requires a specific approach for reading, writing, transforming, and modeling. This course introduces the problem of Big Data, the Big Data universe, reading and writing Big Data, and the skills to work with these data. It uses Spark as a core processing engine for running descriptive and predictive analyses on Big Data.

Modalités

Organisation
Type Amount of time Comment
Travail personnel
Group Project 28,00
Individual Project 6,00
Charge de travail personnel indicative 34,00
Présentiel
Cours magistral 16,00
Cours interactif 16,00
Overall student workload 100,00
Évaluation
The assessment will consist of:
- a group work where students will solve a Big Data case study end-to-end with group feedback
- a set of individual assignments to support learning Spark with individual feedback
- a written exam to test the knowledge of Big Data, the Big Data universe, and Spark.
Control type Duration Amount Weighting
Autres
Projet Collectif 28,00 0 35,00
Projet Individuel 6,00 1 15,00
Examen (final)
Examen écrit 4,00 1 50,00
TOTAL 100,00

Ressources

Bibliographie
Chambers, Bill, and Matei Zaharia. Spark: the definitive guide: big data processing made simple. " O'Reilly Media, Inc.", 2018. -
Ressources Internet