Machine Learning for Communication Systems and Networks Summer School Report

From 1st to 3rd September, I attended to the "Machine Learning (ML) for Communication Systems and Networks" summer school, which is hosted by CONNECT research centre in Trinity. The summer school focuses on research topics, ethics, and innovation perspectives on use cases of ML for communication systems and networks. It is really good opportunity to learn about communication systems use cases given I'm more familiar with ML after my PhD. One of the best things about this summer school for me is the mixture of speakers from different backgrounds such as from academia, startups, journalists, etc.

This post provides an overview and summary of the summer school, which might be of interests for those who have missed the summer school or who will be interested in participating the summer school in the future. An overview of the program is in the following figure.

Day-1

The first day consists of 4 lectures, and started from Irene Macaluso (CONNECT) to give a nice overview of different ML approaches such as unsupervised learning, supervised learning, and reinforcement learning, and some research topics related to applying these approaches in the context of communication systems and networks. An interesting takeaway as a person very new to the research area is that the real-world telecommunication data is difficult to gather and to be used for ML/deep learning (DL) approaches. As a result, many research has been done by using simulation data.

The second lecture is more about the topics related to ethics when we are conducting ML research in general. The speakers introduced the Artificial Intelligence and ML Ethics Toolkit v.01, which is a set of questions should be assessed when doing your ML research with data related to your research.

The third lecture is about traffic analysis using ML/DL from Paul Patras (Univ. of Edinburgh), which is much more related to the current topics I'm working on. It is interesting to see they have introduced so many DL models for traffic forecasting such as STN, CloudLSTM, etc. An interesting work is ZipNet-GAN, which trys to get fine-grained traffic measurements from coarse ones (e.g., measurements aggregated every 10 minutes). The problem of ML research for communication systems and networks is the lack of public datasests. In this regard, some of the research papers have used public datasets "A multi-source dataset of urban life in the city of Milan and the Province of Trentino" and some from network operators which cannot be disclosed.

Finally, the last lecture is from Dermot Casey, a venture leader at NDRC for helping start-ups to grow. Take one step back from talking about all the ML/DL hypes at the moment, the main messages from this talk is focusing on the problem and customers for innovation.

Day-2

The second day is started by Panayotis Mertikopoulos (CNFR) who introduced online learning and optimization approaches with their use cases in the context of wireless communication systems, ranging from channel selection and adaptive routing to rate minimization in a multi-user MIMO network.

In the second lecture, James Little from NCI and ThinkSmarter who has been successful in many commercialized AI products such as Intucell (acquired by Cisco) for self-optimizing networks, talked about AI/ML from the commercialization point of view. This reminds me Andrew Ng said "Enough Papers, Let's Build AI Now!".  James Little mentioned that everything about AI/ML is good, but we need to think about use cases that match customers' thoughts.

AI has initial appeal, but must match the customers way of thinking

Another interesting talk is from Michaela Blott from XILINX research, where her research is mainly focusing on making hardware (e.g., Field Programmable Gate Arrays (FPGAs)) more efficient for training and inference of recent DL models.

Finally, the last talk of the second day was from Claire O'Connor, which focuses on how to communicate your scientific research with non-technical audiences. For example, a good way of doing it is "go with a use case that targets the audience". Some good practices can be trying to answer following questions or summerizing your research in an easy way:

• What are you working on?
• What will be the outcome of your work?
• Boil your story down to a tweet

Day-3

The final day of the summer school consists of three research talks. The first one is from Marco Di Renzo (CNRS) who talked about Model-based or AI-based or hybrid approach for solving problems in networks and communications such as optimal resource allocation problem. Model-based denotes that we approximate the wireless network (WN) model, and apply optimization for optimal resource allocation. In contrast, AI-based denotes we use live measurement data from real system and use ML/DL to solve the problem. The main drawback of the first approach is that the difference between the WN model and the real systems, and that of the second approach is difficult to get real-system data. It is interesting to see transfer learning in this context where they use WN model to simulate and get enough data for training neural network model, and transferring the model with target domain which is the real system with a small amount of data to refine the model compared to training with only target domain data from scratch.

The second talk is from Yong Li (Tsinghua Univ.) with respect to learning users' mobility patterns with the collaboration between telecommunication companies such as China Telecom and tech companies such as Tencent. Despite the big data they got from these companies, it still has many challenges with respect to low quality such as

• separate data 
• low resolution (e.g., district or city level data)
• part of data (only one telco given we are interested in the whole population's mobility)

The talk covered how to resolve these research challenges. After projecting this low quality data to high-quality one, a lot of interesting research has been done with respect to the application of the data such as predicting the move of an individual or crowd.

The final talk of the summer school is given by Jakob Hoydis (Bell Labs) with respect to DL for pysical layer. He talked about the idea of end-to-end learning through autoencoders. By interpreting a communications system as an autoencoder, they develop a fundamental new way to think about communications system design as an end-to-end reconstruction task that seeks to jointly optimize transmitter and receiver components in a single process. He also raised an interesting question with respect to ML/DL in networks and communications: when will the ImageNet moment for networks and communications be arrived. ML/DL which has been explored recent years especially started by computer vision (CV) research, we usally say ImageNet moment has been arrived for CV in 2015. And recently in NLP, one of my previous labmate Sebastian Ruder (now Research Scientist at DeepMind) also has a popular article saying that NLP's ImageNet moment has arrived.

SO WHEN WILL BE THE IMAGENET MOMENT FOR NETWORKS AND COMMUNICATIONS? ONLY TIME WILL TELL！

Summary

ML/DL has been widely adopted in at least academic research in communication systems and networks with a wide range of topics but not limited to:

• traffic analysis, forcasting
• adaptive rounting, channel selection
• resource allocation
• user mobility analysis
• end-to-end communication system
• ...

Real-world data is difficult to obtain for ML/DL, or even got the data, the quality is low in terms of requirements. Research data usually comes from following sources:

• simulation (can get enough data to some extent, the difference gap from real data)
• real-data (direct collaboration with telco operators, cannot be disclosed)
• the combination of above

Some topics such as transfer learning, DL for enhancing data quality can help in this regard. 

Sooner or later, ImageNet moment for communication systems and networks will be arrived