Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Review Help
Search
Attention models in graphs: a survey
Lee J., Rossi R., Kim S., Ahmed N., Koh E. ACM Transactions on Knowledge Discovery from Data13 (6):1-25,2019.Type:Article
Date Reviewed: Sep 20 2021

Having spent some time on early attempts to bring together neural networks and symbol-oriented knowledge representation, I am intrigued by the more recent work on deep learning and knowledge graphs. While many of the approaches appear conceptually interesting, I have some reservations about their use in large, noisy real-world graphs. This survey looks at attention as a way to tame some of these issues. The authors review publications that use attention in graphs and machine learning. Their review is structured around three taxonomies: the type of attention, the task, and the problem setting.

The authors provide rigorous definitions of different types of graphs and graph attention. The latter identifies nodes that are of particular interest, and ranks them according to a relevance function. These could be nodes in the vicinity of a given node or distant nodes with similar properties to the current one. Multiple relevance functions can be used to shift the perspective of attention accordingly. The notion of attention has been used for a variety of contexts and tasks, including computer vision, natural language processing, and information retrieval, with node or link classification, link prediction, node ranking and alignment, graph classification, and generation of sequences from graphs as particular tasks.

While I had difficulties following the intricacies of all the approaches evaluated according to their taxonomies, I found the survey very valuable. It offers a systematic approach to comparing and analyzing attention in graph-based methods. In addition, the authors point out interesting venues for further work, especially concerning the suitability of such approaches for large-scale real-world problems.

Reviewer:  Franz Kurfess Review #: CR147358 (2111-0270)
Bookmark and Share
  Reviewer Selected
Featured Reviewer
 
 
Graph Theory (G.2.2 )
 
Would you recommend this review?
yes
no
Other reviews under "Graph Theory": Date
Graphs and algorithms
Gondran M., Minoux M. (ed), Vajda S., John Wiley & Sons, Inc., New York, NY, 1984. Type: Book (9789780471103745)
Jan 1 1985
On graph rewritings
Raoult J. Theoretical Computer Science 32(1-2): 1-24, 1984. Type: Article
Sep 1 1985
Non-partitionable point sets
Avis D. Information Processing Letters 19(3): 125-129, 1984. Type: Article
Jul 1 1985
more...

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright 1999-2024 ThinkLoud®
Terms of Use
| Privacy Policy