Monday, July 8, 2024

 

9:00 AM – 11:30 AM

  • Title: Learning and Regression for Multidimensional Data Using Tensor Decompositions
    • Presenters:
      Waheed U. Bajwa (Rutgers University, USA)
      Anand D. Sarwate (Rutgers University, USA)
      Batoul A.Taki (Rutgers University, USA)

Abstract: Since many real-world measurements are inherently multidimensional—involving multiple dimensions or aspects— tensors, which extend vectors and matrices to higher dimensions, are crucial in handling such data. Recent research literature has significantly highlighted the application of tensors in signal processing, machine learning, and statistics. Unlike previous tutorials targeting signal processing audiences, this tutorial focuses on the applications of tensors in supervised and unsupervised learning, rather than on tensor approximation and factorization algorithms.

We begin this tutorial by providing motivational examples where tensor-structured learning and inference problems may be useful. These applications are intended to exemplify the advantages of imposing tensor decompositions in machine/reinforcement learning problems, as well as illustrate the regimes in which imposing tensor structures is favorable. We will then cover fundamental tensor-related concepts, including foundational tensor algebra and tensor decompositions. We will introduce a continuum of five tensor decompositions—CANDECOMP/PARAFAC (CP), Tensor Train, Tucker, Low Separation Rank (LSR), and Block Tensor Decomposition (BTD)—juxtaposed on a scale of representation power/generalizability for each decomposition. After these preliminaries, we will describe the general foundations for a learning theoretic framework of Generalized Linear Models (GLMs) and representation learning, through the lens of tensor-structured regularizers. These approaches not only reduce the number of learnable parameters compared to the vector setting but also retain the tensor’s spatial structure, which often contains valuable information necessary for effective estimation. We will demonstrate how these tensor-structured problems exemplify the flexibility of tensor decompositions in tuning model complexity to match the available data. To estimate the tensor-structured parameters in such problems, we will introduce numerical methods including alternating minimization / block coordinate descent, projection-based methods, and importance sketching. Lastly, we will cover new theoretical results for tensor-structured problems by describing sample complexity bounds that provide a concrete theoretical understanding of the benefits of incorporating tensors into GLM and representation learning problems. This analysis highlights the impact of tensor-structured regularizers on the problem’s sample complexity. We will compare these theoretical guarantees with sample complexity upper and lower bounds introduced in more recent tensor works on regression, GLMs, and dictionary learning. We will conclude the tutorial with a discussion on open problems for future research.

This tutorial is designed to be accessible to students and researchers with a background in linear algebra, probability theory, and optimization, and does not assume prior familiarity with tensor algebra. Researchers interested in applying multidimensional signal processing techniques to their problems will find this tutorial an excellent introduction to the fundamentals, with pointers to recent literature.

Bio:

Waheed U. Bajwa is a professor and graduate program director in the Department of Electrical and Computer Engineering at Rutgers University, where he is also a member of the Department of Statistics graduate faculty. His research spans statistical signal processing, high-dimensional statistics, machine learning, inverse problems, 2 and networked systems. Previously, he has held positions at Princeton University, Duke University, and various technology startups. Dr. Bajwa has received multiple awards, including the Army Research Office Young Investigator Award (2014), the NSF CAREER Award (2015), and the Rutgers Presidential Merit Award (2016). He has also been recognized as the Rutgers ECE Professor of the Year multiple times (2016, 2017, 2019) and received the Rutgers Warren I. Susman Award for Excellence in Teaching (2021) and the Rutgers Presidential Outstanding Faculty Scholar Award (2022). Additionally, his collaborative work earned the Gallo Award for Scientific Excellence from the Cancer Institute of New Jersey in 2017. He was also a co-author on papers that received Best Student Paper Awards at IEEE IVMSP 2016 and IEEE CAMSAP 2017, and a participant in the 2015 National Academy of Engineering Frontiers of Engineering Education Symposium.

 

Anand D. Sarwate received his Ph.D. in electrical engineering from the EECS Department at UC Berkeley. He is a currently an Associate Professor at Rutgers and was previously a Research Assistant Professor at TTI Chicago and a postdoc at the ITA Center at UCSD. His research interests include information theory, machine learning, signal processing, optimization, and privacy and security. Dr. Sarwate is a Distinguished Lecturer of the IEEE Information Theory Society for 2024–2025 and is on the Board of Governors of the IEEE Information Theory Society. Prior Tutorial Experience: He has given tutorials at WIFS 2014 and NIPS 2017 on Differential Privacy and Machine Learning (with Kamalika Chaudhuri) as well as a short course on Differential Privacy through the University of Maryland JPSM program.

 

Batoul A. Taki is a PhD Candidate in the Department of Electrical and Computer Engineering at Rutgers University New Brunswick. She received her B.S in Computer and Communications Engineering from the American University of Science and Technology and her M.S in Electrical and Computer Engineering from Rutgers. Batoul is a PreDoctoral Leadership Program Scholar at Rutgers. Her primary research interests involve theoretical machine learning for tensor-structured data, with applications to medical imaging and neuroimaging data. She is completing a dissertation under the direction of Prof. Bajwa and some of the material on tensor GLMs for this tutorial will draw on her work.

 

1:00 PM – 3:30 PM

  • Title: A Unified Framework for Distributed Adaptive Spatial Filtering
    • Presenter:
      Cem Ates Musluoglu (KU Leuven, Belgium)

Abstract: The emergence of Wireless Sensor Networks (WSNs) has underlined the need for distributed signal processing methods that reduce the communication and energy costs across the network by avoiding centralization of the data. In general, the aim is to find a filter, parameter or signal that is optimal in some sense for the data recorded by the WSN. In this tutorial, we focus on distributed data-driven spatial filtering in such WSNs. The goal is to estimate and track the spatial correlation across the network and exploit this to spatially combine the various (multichannel) sensor signals to generate a single output signal that is optimal in some sense (e.g., maximal SNR, minimal squared error, maximally correlated components, maximal variance, etc.), as commonly encountered in various WSN types, such as acoustic sensor networks, body- or neuro-sensor networks, and communication networks. However, most of the existing workhorse algorithmic frameworks for distributed signal processing such as consensus, diffusion, gossip, or alternating direction method of multipliers (ADMM), consider a per-node partitioning of the objective function of the optimization problem of interest. For spatial filtering problems, these methods are either inapplicable, or at least highly impractical due to the emergence of expensive nested iterations, which often require more communication bandwidth than a centralized approach. In this tutorial, we will first provide some initial insights into why distributed data-driven spatial filtering problems often do not fit well with these traditional distributed signal processing methods. Some dedicated algorithms that perform better will be presented next, however, it will be highlighted that these are typically highly tailored to specific problem statements, i.e., there exists no workhorse framework that is generally applicable for such distributed spatial filtering problems. A generic unifying algorithmic framework based on recent work will be presented next and will be the main topic of this tutorial. The framework has several existing distributed spatial filtering algorithms from the literature as a special case, while at the same time generalizing these to admit more general problem formulations. Both theoretical and practical aspects will be discussed during this part. The final part of the presentation will focus on various extensions of the framework, either to extend it to larger families of problems or to improve its overall performance.

Bio:

Cem Ates Musluoglu received the B.Sc. and M.Sc. degrees in electrical and electronic engineering from École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, in 2017 and 2019, respectively and the Ph.D. degree in electrical engineering from KU Leuven, Leuven, Belgium in 2023. He is currently a postdoctoral researcher at the STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics within the Electrical Engineering Department of KU Leuven. His research interests include distributed learning, spatial filtering, and adaptive learning within the broader fields of signal processing, machine learning, and optimization, with an emphasis on electroencephalography applications.

 

 

4:00 PM – 6:30 PM                        

  • Title: Do More with Less: Revolutionize Direction Finding with Sparse Arrays
    • Presenter:
      Yimin D. Zhang (Temple University, USA)

Abstract: Direction finding stands as an important area in array signal processing and plays fundamental and crucial roles across various fields, such as radar, sonar, wireless communications, speech separation, radio astronomy, and medical imaging. The evolving needs in these fields demand resolving more signals with higher spatial resolution and improved estimation accuracy. However, meeting these demands often leads to significant hardware costs, primarily due to the need for a high number of array sensors to achieve the required array aperture and degrees of freedom (DOFs). To address these challenges, sparse arrays offer an attractive solution, providing low-complexity alternatives with significantly fewer array sensors while maintaining and enhancing the direction-finding capabilities and performance compared to non-sparse counterparts.

This tutorial delves into recent advancements in direction-of-arrival (DOA) estimation utilizing sparse sensor arrays. It consists of the following three parts: (i) The first part begins by introducing the fundamental principles of sparse arrays and difference coarrays which augment array aperture and DOFs. Traditional DOA estimation methods and Cramer-Rao bound (CRB) analysis are then introduced. Optimal sparse array designs aimed at attaining a high number of consecutive lags with low mutual coupling will be discussed. (ii) The second part presents the intertwined progress in array design strategies, DOA estimation methods, and data recovery techniques that collectively increase the array aperture, the number of DOFs, and, subsequently, the number of detectable signals. (iii) The third part explores various research directions and real-world applications, including the consideration of distributed arrays, frequency diversity, multi-dimensional and multi-domain processing, signal coherency, and machine learning-based processing. These innovations harness recent developments in compressive sensing, matrix completion, convex optimization, tensor processing, and machine learning. The tutorial aims to furnish the audience with a comprehensive understanding and valuable insights into cost-effective and efficient solutions for sparse array design and processing.

Bio:

Dr. Yimin D. Zhang is an Associate Professor of the Department of Electrical and Computer Engineering, Temple University, Philadelphia, PA, USA. He received his B.Sc. degree from Xidian University, China, and his Ph.D. degree from the University of Tsukuba, Japan. His research interests lie in the areas of statistical signal and array processing, compressive sensing, convex optimization, machine learning, information theory, and time-frequency analysis, with applications to radar, wireless communications, satellite navigation, and radio astronomy.

Dr. Zhang is a Senior Area Editor for IEEE Transactions on Signal Processing and an Editor for Signal Processing. He serves for the IEEE Signal Processing Society (SPS) as a member of the Signal Processing Theory and Methods (SPTM) Technical Committee, a member of the Integrated Sensing and Communication (ISAC) Working Group, and previously as a member of the Sensor Array and Multichannel Signal Processing (SAM) Technical Committee. His publications on sparse array design and processing have received the 2018 and 2021 IEEE SPS Young Author Best Paper Awards, the 2019 IET Communications Premium Award, and the 2021 EURASIP Best Paper Award for Signal Processing. He also received the 2017 Harry Rowe Mimno Award from the IEEE Aerospace and Electronic System Society for his magazine article on dual-function radar-communications. He is a fellow of IEEE, a fellow of SPIE, and a Distinguished Lecturer of the IEEE SPS (for the term 2024-2025).

4:00 PM – 6:30 PM

  • Title: Learning Techniques for mmWave/THz Communications and Sensing
    • Presenters:
      Kumar Vijay Mishra (United States Army Research Laboratory)
      Ahmet M. Elbir (University of Luxembourg, Luxembourg)

Abstract: The design of future communications and sensing systems entails several tasks such as transmitter signal selection, precoder design, receive processing, channel estimation, tracking, or resource allocation to name a few. This often requires solving difficult nonconvex optimization problems that involve either a common performance metric as a cost function or inclusion of several constraints related to the employed sensing and communications regime. There is no standard methodology for solving such problems, and usually, the derivation of an efficient solution is a very challenging task. However, optimization-based approaches suffer from high computational complexity and their performance strongly relies on factors such as perfect channel knowledge, ideal information exchange between sensing and communication modules, specific constraints, and mobility. In this context, learning techniques provide robust performance at an upfront training cost. Moreover, full exploitation of similarities between sensing and communication channels when operating at mmWave/THz frequencies, either for communication-aided sensing or sensing-aided communication, would require extremely complicated models accounting for propagation differences, potential antenna differences, or calibrations to compensate for different locations of sensor and communication modules. Learning strategies can also become the preferred solution in this case, where the mathematical models are unknown and would potentially require excessive complexity. In this tutorial, the audience will learn about applying learning to various aspects of hybrid beamforming including channel estimation, antenna selection, wideband beamforming, and spatial modulation. In addition, we will examine these concepts in the context of joint radar-communications architectures.

Bio:

Kumar Vijay Mishra (S’08-M’15-SM’18) obtained a Ph.D. in electrical engineering and M.S. in mathematics from The University of Iowa in 2015, and M.S. in electrical engineering from Colorado State University in 2012, while working on NASA’s Global Precipitation Mission Ground Validation (GPM-GV) weather radars. He received his B. Tech. summa cum laude (Gold Medal, Honors) in electronics and communication engineering from the National Institute of Technology, Hamirpur (NITH), India in 2003. He is currently Senior Fellow at the United States Army Research Laboratory (ARL), Adelphi. He is the Distinguished Lecturer of the IEEE Communications Society (2023-2024), IEEE Aerospace and Electronic Systems Society (AESS) (2023-2024), IEEE Vehicular Technology Society (2023-2024), and IEEE Future Networks Initiative (2022). He is the recipient of several prestigious best paper awards and fellowships, including the IET Premium Best Paper Prize (2021) and U. S. National Academies Harry Diamond Distinguished Fellowship (2018-2021). He is the Chair (2023-present) of the Synthetic Apertures Technical Working Group of the IEEE Signal Processing Society (SPS) and Vice-Chair (2021-present) of the IEEE Synthetic Aperture Standards Committee, which is the first SPS standards committee. He is the Chair (2023-2026) of the International Union of Radio Science (URSI) Commission C. He is the lead co-editor of three books on radar: Signal Processing for Joint Radar-Communications (Wiley-IEEE Press, 2024), Next-Generation Cognitive Radar Systems (IET Press, 2024), and Advances in Weather Radar Volumes 1, 2 & 3 (IET Press, 2024). His research interests include radar systems, signal processing, remote sensing, and electromagnetics.

Ahmet M. Elbir (S’08-M’14-SM’20) received the B.S. degree with Honors from Firat University in 2009, and the Ph.D. degree from the Middle East Technical University (METU) in 2016, both in electrical engineering. He is currently a Research Fellow at University of Luxembourg. He is also a Research Associate at King Abdullah University of Science and Technology (KAUST) since 2023. In the past, Dr Elbir has held Visiting Postdoctoral Researcher positions at Koc University in 2020-2021, Carleton University in 2022-2023, and the University of Hertfordshire in 2021-2023. In 2016-2022, he worked as a Senior Researcher at Duzce University. His research interests include array signal processing for radar and communications, and deep learning for multi-antenna systems. He has served as an Associate Editor for IEEE Access, and a Lead Guest Editor for IEEE Journal of Selected Topics in Signal Processing, IEEE Signal Processing Magazine, IEEE Wireless Communications, IEEE Communications and Standards Magazine, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing and Elsevier Signal Processing. Dr. Elbir is the recipient of the 2016 METU Best Ph.D. thesis award for his doctoral studies, the 2022 IEEE Turkey Section Research Encouragement Award, and the IET Radar, Sonar & Navigation Best Paper Award in 2022.