Sequential Signal Mixing Aggregation for Message Passing Graph Neural Networks

Mitchell Keren Taraday [1]*, Almog David [1]*, Chaim Baskin [2]
[1] - Department of Computer Science, Technion
[2] - School of Electrical and Computer Engineering Ben-Gurion University of the Negev
NeurIPS 2024, the Thirty-Eighth Annual Conference on Neural Information Processing Systems
*Indicates Equal Contribution
Presentation Paper Code arXiv

Abstract

Message Passing Graph Neural Networks (MPGNNs) have emerged as the pre- ferred method for modeling complex interactions across diverse graph entities. While the theory of such models is well understood, their aggregation module has not received sufficient attention. Sum-based aggregators have solid theoretical foun- dations regarding their separation capabilities. However, practitioners often prefer using more complex aggregations and mixtures of diverse aggregations. In this work, we unveil a possible explanation for this gap. We claim that sum-based aggre- gators fail to "mix" features belonging to distinct neighbors, preventing them from succeeding at downstream tasks. To this end, we introduce Sequential Signal Mix- ing Aggregation (SSMA), a novel plug-and-play aggregation for MPGNNs. SSMA treats the neighbor features as 2D discrete signals and sequentially convolves them, inherently enhancing the ability to mix features attributed to distinct neighbors. By performing extensive experiments, we show that when combining SSMA with well-established MPGNN architectures, we achieve substantial performance gains across various benchmarks, achieving new state-of-the-art results in many settings.

Poster

BibTeX

@misc{taraday2024sequentialsignalmixingaggregation,
      title={Sequential Signal Mixing Aggregation for Message Passing Graph Neural Networks},
      author={Mitchell Keren Taraday and Almog David and Chaim Baskin},
      year={2024},
      eprint={2409.19414},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2409.19414},
}