Parallel Processing (Theory)

Module Information

Module Semester:
Module Part:
Sub-Module Code:
Hours per Week:
Module ECTS Credits:
Available to ERASMUS Students:
Module Staff:

Module Objective

The course aims to teach students fundamental and advanced concepts of parallel computing. Topics include memory systems, different paradigms of programming parallel computers (OpenMP, MPI, CUDA) and scalability and efficiency of parallel algorithms and systems.

Module Study Targets

Upon completion of the course, students will be able to:

  • Describe a number of different models of parallel and distributed computing and the basic techniques for designing algorithms in these models
  • Design efficient parallel algorithms and applications
  • Analyze the requirements for programming parallel systems and critically evaluate the strengths and weaknesses of parallel programming models and how they can be used to facilitate the programming of concurrent systems
  • Evaluate the efficiency of a parallel processing system and evaluate the types of application for which parallel programming is useful
  • Write parallel programs for parallel systems, shared address space platforms, and heterogeneous platforms
  • Make use of the international bibliography for related research problems and results

Module Acquired Abilities

  • Working independently
  • Production of free, creative and inductive thinking
  • Team work
  • Production of new research ideas
  • Working in an international environment

Module Description

  • Introduction to Parallel Processing
  • Parallel Processing Architecture
  • Principles of Parallel Programming
  • Software and Hardware for Parallel Processing
  • Methods, techniques and networks
  • Parallel programming models and tools
  • Analysis and efficiency of programs
  • Parallel Program Design, Development, and implementation
  • Distributed-Memory Programming with MPI
  • Shared-Memory Programming with OpenMP
  • Advanced OpenMP programming
  • GPU programming
  • Practical problems

Module Student Evaluation

Written examination: 60%

Laboratory exercise: 40%

Optional: Project submission (20% of the final Lectures grade).


  • M. McCool, J. Reinders, A. Robison, "Structured Parallel Programming: Patterns for Efficient Computation", 1st Ed., Morgan Kaufmann, 2012
  • D. Serpanos, T. Wolf, "Architecture of Network Systems", Morgan-Kaufmann Publishers, 2011
  • M. J. Quinn, "Parallel Programming in C with MPI and OpenMP", 1st Ed., McGraw Hill, 2004
  • A. Grama, V. Kumar, A. Gupta, G. Karypis,"Introduction to Parallel Computing", Addison Wesley, 2003