Distributed Computing Laboratory DCL

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
education [2019/04/05 16:11]
mmonti
education [2019/06/17 15:45] (current)
zablotch
Line 3: Line 3:
 ====== Education ====== ====== Education ======
  
 +\\
  
 The lab is teaching the following courses: The lab is teaching the following courses:
 \\ \\
 +
 +  * [[education/​ca_2018|Concurrent Algorithms]] (theory & practice)
 +  * [[education/​da|Distributed Algorithms]] (theory & practice)
 \\ \\
-  * [[education/​ca_2018|Concurrent Algorithms]] +The lab taught in the past the following courses: 
-  * [[education/​da|Distributed Algorithms]]+
   * <​html><​a href="​http://​moodle.epfl.ch/​course/​view.php?​id=14044">​Information,​ Calcul et Communication</​a></​html>​   * <​html><​a href="​http://​moodle.epfl.ch/​course/​view.php?​id=14044">​Information,​ Calcul et Communication</​a></​html>​
   * <​html><​a href="​http://​cowww.epfl.ch/​proginfo/​wwwhiver/">​Introduction à la Programmation Orientée Objet</​a></​html>​   * <​html><​a href="​http://​cowww.epfl.ch/​proginfo/​wwwhiver/">​Introduction à la Programmation Orientée Objet</​a></​html>​
Line 23: Line 26:
 DCL offers master projects in the following areas: DCL offers master projects in the following areas:
  
-  * **Probabilistic Byzantine Resilience**:  ​Development of high-performance,​ Byzantine-resilient distributed systems with provable probabilistic guarantees. Two options are currently available, both building on previous work on probabilistic Byzantine broadcast: (i) a theoretical ​project, focused the correctness ​of probabilistic Byzantine-tolerant distributed algorithms; (ii) a practical project, focused ​on numerically evaluating of our theoretical results. Please contact [[matteo.monti@epfl.ch|Matteo Monti]] to get more information. +  * **[[cryptocurrencies|Cryptocurrencies]]**: We have several ​project ​openings as part of our ongoing research ​on designing new cryptocurrency systems. Please contact [[rachid.guerraoui@epfl.ch|ProfRachid Guerraoui]].
- +
-  * **Dynamically Distributed Spatial Indexing**: ​ a project here would consist in studying existing spatial index data structures and algorithms, e.g., simple grids, Quadtrees, R-Trees etc., and how they may be dynamically distributed for indexing a large number of moving objects; please contact [[mailto:​benoit.garbinato@unil.ch|Benoit Garbinato]] to get more information.+
  
 +  * **Probabilistic Byzantine Resilience**: ​ Development of high-performance,​ Byzantine-resilient distributed systems with provable probabilistic guarantees. Two options are currently available, both building on previous work on probabilistic Byzantine broadcast: (i) a theoretical project, focused the correctness of probabilistic Byzantine-tolerant distributed algorithms; (ii) a practical project, focused on numerically evaluating of our theoretical results. Please contact [[matteo.monti@epfl.ch|Matteo Monti]] to get more information.
  
-  * **Multicore computing**:​ a project here would consist for instance in designing and implementing efficient lock-based or lock-free shared objects; please contact [[https://​people.epfl.ch/​igor.zablotchi|Igor Zablotchi]] to get more information. 
  
-  * **Distributed computing using RDMA and/or NVRAM**: contact ​[[https://​people.epfl.ch/​igor.zablotchi|Igor Zablotchi]] for more information.+  * **Distributed computing using RDMA and/or NVRAM.** RDMA (Remote Direct Memory Access) allows accessing a remote machine'​s memory without interrupting its CPU. NVRAM is byte-addressable persistent (non-volatile) memory with access times on the same order of magnitude as traditional (volatile) RAM. These two recent technologies pose novel challenges and raise new opportunities in distributed system design and implementation. Contact ​[[https://​people.epfl.ch/​igor.zablotchi|Igor Zablotchi]] for more information.
  
-  * **[[Distributed ML|Distributed Machine Learning]]**+  * **[[Distributed ML|Distributed Machine Learning]]**: contact [[http://​people.epfl.ch/​georgios.damaskinos|Georgios Damaskinos]] for more information.
  
-  * **Distributed ​and Fault-tolerant algorithms**: projects here would consist in designing failure detection mechanisms suited for large-scale systemsreal-time systems, and systems with unreliable communication or partial synchronyThis task also involves implementingevaluating, and simulating ​the performance of the developed mechanisms ​to verify ​the achievable guarantees; please contact ​[[http://​people.epfl.ch/​david.kozhaya|David Kozhaya]] to get more information.+  * **Robust ​Distributed ​Machine Learning**: With the proliferation of big datasets and modelsMachine Learning is becoming distributed. Following the standard parameter server modelthe learning phase is taken by two categories of machines: parameter servers ​and workers. Any of these machines could behave arbitrarily (i.e., said Byzantine) affecting ​the model convergence in the learning phase. Our goal in this project is to build a system that is robust against Byzantine behavior of both parameter server and workers. Our first prototype, AggregaThor(https://​www.sysml.cc/​doc/​2019/​54.pdf),​ describes ​the first scalable robust Machine Learning framework. It fixed a severe vulnerability in TensorFlow and it showed how to make TensorFlow even faster, while robust. Contact ​[[https://​people.epfl.ch/​arsany.guirguis|Arsany Guirguis]] for more information.
  
-  * **Consistency in global-scale storage systems**: We offer several ​projects ​in the context of storage systemsranging from implementation ​of social applications ​(similar ​to [[http://retwis.redis.io/|Retwis]], or [[https://github.com/share/sharejs|ShareJS]]) to recommender systems, static content storage services (à la [[https://www.usenix.org/legacy/event/​osdi10/​tech/​full_papers/​Beaver.pdf|Facebook'​s Haystack]]), or experimenting with well-known cloud serving benchmarks (such as [[https://​github.com/​brianfrankcooper/​YCSB|YCSB]]);​ please contact [[http://​people.epfl.ch/​dragos-adrian.seredinschi|Adrian Seredinschi]] for further ​information.+  * **Stochastic gradient: (artificial) reduction of the ratio variance/​norm for adversarial distributed SGD**: One computationally-efficient and non-intrusive line of defense for adversarial distributed SGD (e.g. 1 parameter server distributing the gradient estimation to several, possibly adversarial workers) relies on the honest workers to send back gradient estimations with sufficiently low variance; assumption which is sometimes hard to satisfy ​in practice. One solution could be to (drastically) increase ​the batch-size at the workersbut doing so may as well defeat the very purpose ​of distributing the computation. \\ In this project, we propose two approaches that you can choose to explore ​(also you may propose a different approach) ​to (artificially) reduce the ratio variance/norm of the stochastic gradients, while keeping the benefits of the distributionThe first proposed approach, speculative,​ boils down to "​intelligent"​ coordinate selectionThe second makes use of some kind of "​momentum"​ at the workers. \\ [1] [[https://papers.nips.cc/paper/6617-machine-learning-with-adversaries-byzantine-tolerant-gradient-descent|"​Machine Learning with Adversaries:​ Byzantine Tolerant Gradient Descent" ​]]  \\ [2] [[https://arxiv.org/abs/1610.05492|"​Federated Learning: Strategies for Improving Communication Efficiency"​]] \\ Contact  ​[[https://​people.epfl.ch/​sebastien.rouault|Sébastien Rouault]] for more information.
  
-  * **Distributed database algorithms**:​ a project here would consist in implementing and evaluating protocols that are running in today'​s database systems, e.g., [[https://​en.wikipedia.org/​wiki/​Two-phase_commit_protocol|2PC]],​ and comparing them with those protocols that can  potentially be used in future database systems; please contact [[http://​people.epfl.ch/​jingjing.wang|Jingjing Wang]] to get more information. 
  
 +  * **Consistency in global-scale storage systems**: We offer several projects in the context of storage systems, ranging from implementation of social applications (similar to [[http://​retwis.redis.io/​|Retwis]],​ or [[https://​github.com/​share/​sharejs|ShareJS]]) to recommender systems, static content storage services (à la [[https://​www.usenix.org/​legacy/​event/​osdi10/​tech/​full_papers/​Beaver.pdf|Facebook'​s Haystack]]),​ or experimenting with well-known cloud serving benchmarks (such as [[https://​github.com/​brianfrankcooper/​YCSB|YCSB]]);​ please contact [[http://​people.epfl.ch/​dragos-adrian.seredinschi|Adi Seredinschi]] or [[https://​people.epfl.ch/​karolos.antoniadis|Karolos Antoniadis]] ​ for further information.