Module Details

Module Code: DATA8005
Title: Distributed Data Management
Long Title: Distributed Data Management
NFQ Level: Advanced
Valid From: Semester 1 - 2018/19 ( September 2018 )
Duration: 1 Semester
Credits: 5
Field of Study: 4816 - Data Format
Module Delivered in: 1 programme(s)
Module Description: Big data analytics turns big datasets into high-quality information, providing deeper insights enabling better decisions.
However, big data requires novel data storage and data process techniques.
In this module the learners will be survey the main NoSQL-based data models as an alternative to the traditional relational model.
The learner will also explore the ecosystem of a big data framework, with an special focus on the data storage and the application
of large-scale data analysis libraries.
 
Learning Outcomes
On successful completion of this module the learner will be able to:
# Learning Outcome Description
LO1 Appraise the challenges posed by big data and the new infrastructure, data models and processing techniques it demands.
LO2 Survey the main NoSQL-based data models, exploring the best-fit for different use-cases.
LO3 Query a range of NoSQL databases using a high-level programming language.
LO4 Explore the scalability, flexibility and reliability of a distributed data cluster supporting large data sets.
LO5 Implement an analytical solution over a large-scale dataset using MapReduce and Spark.
Dependencies
Module Recommendations

This is prior learning (or a practical skill) that is strongly recommended before enrolment in this module. You may enrol in this module if you have not acquired the recommended learning but you will have considerable difficulty in passing (i.e. achieving the learning outcomes of) the module. While the prior learning is expressed as named MTU module(s) it also allows for learning (in another module or modules) which is equivalent to the learning specified in the named module(s).
Incompatible Modules
These are modules which have learning outcomes that are too similar to the learning outcomes of this module. You may not earn additional credit for the same learning and therefore you may not enrol in this module if you have successfully completed any modules in the incompatible list.
No incompatible modules listed
Co-requisite Modules
No Co-requisite modules listed
Requirements

This is prior learning (or a practical skill) that is mandatory before enrolment in this module is allowed. You may not enrol on this module if you have not acquired the learning specified in this section.
No requirements listed
 
Indicative Content
The Big Data Revolution.
Data storage and data process: Historical evolution. New infrastructure, data models and processing techniques required to deal with big data. Main challenges: Capture, store, search, analyse and visualise the data.
Big Data Framework.
Dataset characterisation: Variety, velocity and volume. Data Framework ecosystem overview: Tools to ingest, store, analyse and manage data. Data integration: Extracting, transforming and loading relational and nonrelational-based data.
Data Storage.
Distributed File System. Data nodes vs. name nodes. Data replication and fault tolerance. Cluster manager: component and roles. Large files splitting and distribution algorithms.
NoSQL Data Models.
NoSQL databases arising to tackle problem RDBMS is not good at: Schema-less, high level data representation, scale-out distributed-based infrastructure. CAP theorem. Lost of transactional properties: ACID relational properties vs BASE for NoSQL. Wide range of data models: Pure key/value, colummn-based, document oriented and graph-based. Trade-off between their expressiveness and efficiency. Polyglot persistance: On combining different NoSQL data models for a fit for purpose multi-component system.
Data Processing: Large-scale Analytics.
Text, temporal and geospatial-based datasets. Execution plan: Cluster nodes collaboaration, parallel processing, job scheduling, network transferrence, key/value-based communication. Large-scale analytics libraries: MapReduce and Spark. On comparing their expressiveness and efficiency.
Module Content & Assessment
Assessment Breakdown%
Coursework100.00%

Assessments

Coursework
Assessment Type Practical/Skills Evaluation % of Total Mark 50
Timing Week 7 Learning Outcomes 1,2,3
Assessment Description
Implement some queries for a large dataset stored both in a document-oriented vs. graph-based database.
Assessment Type Practical/Skills Evaluation % of Total Mark 50
Timing Week 12 Learning Outcomes 1,4,5
Assessment Description
Implement a solution for analysing a large dataset using MapReduce and Spark.
No End of Module Formal Examination
Reassessment Requirement
Coursework Only
This module is reassessed solely on the basis of re-submitted coursework. There is no repeat written examination.

The University reserves the right to alter the nature and timings of assessment

 

Module Workload

Workload: Full Time
Workload Type Contact Type Workload Description Frequency Average Weekly Learner Workload Hours
Lecture Contact Lecture based on Indicative Content Every Week 1.00 1
Lab Contact Lab based on Indicative Content Every Week 3.00 3
Independent Learning Non Contact Student undertakes independent study and develops relevant programming skills Every Week 3.00 3
Total Hours 7.00
Total Weekly Learner Workload 7.00
Total Weekly Contact Hours 4.00
Workload: Part Time
Workload Type Contact Type Workload Description Frequency Average Weekly Learner Workload Hours
Lecture Contact Lecture based on Indicative Content Every Week 1.00 1
Lab Contact Lab based on Indicative Content Every Week 3.00 3
Independent Learning Non Contact Student undertakes independent study and develops relevant programming skills Every Week 3.00 3
Total Hours 7.00
Total Weekly Learner Workload 7.00
Total Weekly Contact Hours 4.00
 
Module Resources
Recommended Book Resources
  • Pramod J. Sadalage and Martin Fowler. (2013), NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence, Addison-Wesley, [ISBN: 9780321826626].
  • Ofer Mendelevitch, Casey Stella and Douglas Eadline. (2017), Practical Data Science with Hadoop and Spark : Designing and Building Effective Analytics at Scale, Pearson Education, [ISBN: 9780134024141].
Supplementary Book Resources
  • John Sharp et. al. (2013), Data Access for Highly-Scalable Solutions: Using SQL, NoSQL, and Polyglot Persistence, Microsoft patterns & practices, [ISBN: 9781621140306].
  • Kristina Chodorow. (2013), MongoDB: The Definitive Guide, O'Reilly Media, [ISBN: 9781449344689].
  • Srinath Perera and Thilina Gunarathne. (2013), Hadoop MapReduce Cookbook, Packt Publishing, [ISBN: 9781849517294].
This module does not have any article/paper resources
Other Resources
 
Module Delivered in
Programme Code Programme Semester Delivery
CR_SDAAN_8 Higher Diploma in Science in Data Science & Analytics 2 Mandatory