Adversarial Normalization: I Can visualize Everything (ICE)
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Choi, Hoyoung | - |
dc.contributor.author | Jin, Seungwan | - |
dc.contributor.author | Han, Kyungsik | - |
dc.date.accessioned | 2023-11-14T08:17:53Z | - |
dc.date.available | 2023-11-14T08:17:53Z | - |
dc.date.issued | 2023-06 | - |
dc.identifier.issn | 1063-6919 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/192191 | - |
dc.description.abstract | Vision transformers use [CLS] tokens to predict image classes. Their explainability visualization has been studied using relevant information from [CLS] tokens or focusing on attention scores during self-attention. Such visualization, however, is challenging because of the dependence of the structure of a vision transformer on skip connections and attention operators, the instability of non-linearities in the learning process, and the limited reflection of self-attention scores on relevance. We argue that the output vectors for each input patch token in a vision transformer retain the image information of each patch location, which can facilitate the prediction of an image class. In this paper, we propose ICE (Adversarial Normalization: I Can visualize Everything), a novel method that enables a model to directly predict a class for each patch in an image; thus, advancing the effective visualization of the explainability of a vision transformer. Our method distinguishes background from foreground regions by predicting background classes for patches that do not determine image classes. We used the DeiT-S model, the most representative model employed in studies, on the explainability visualization of vision transformers. On the ImageNet-Segmentation dataset, ICE outperformed all explainability visualization methods for four cases depending on the model size. We also conducted quantitative and qualitative analyses on the tasks of weakly-supervised object localization and unsupervised object discovery. On the CUB-200-2011 and PASCALVOC07/12 datasets, ICE achieved comparable performance to the state-of-the-art methods. We incorporated ICE into the encoder of DeiT-S and improved efficiency by 44.01% on the ImageNet dataset over that achieved by the original DeiT-S model. We showed performance on the accuracy and efficiency comparable to EViT, the state-of-the-art pruning model, demonstrating the effectiveness of ICE. The code is available at https://github.com/Hanyang-HCC-Lab/ICE. | - |
dc.format.extent | 10 | - |
dc.language | 영어 | - |
dc.language.iso | ENG | - |
dc.publisher | IEEE Computer Society | - |
dc.title | Adversarial Normalization: I Can visualize Everything (ICE) | - |
dc.type | Article | - |
dc.publisher.location | 미국 | - |
dc.identifier.doi | 10.1109/CVPR52729.2023.01166 | - |
dc.identifier.scopusid | 2-s2.0-85173910149 | - |
dc.identifier.wosid | 001062522104042 | - |
dc.identifier.bibliographicCitation | Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, v.2023-June, pp 12115 - 12124 | - |
dc.citation.title | Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition | - |
dc.citation.volume | 2023-June | - |
dc.citation.startPage | 12115 | - |
dc.citation.endPage | 12124 | - |
dc.type.docType | Proceedings Paper | - |
dc.description.isOpenAccess | N | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Artificial Intelligence | - |
dc.subject.keywordAuthor | Explainable computer vision | - |
dc.identifier.url | https://ieeexplore.ieee.org/document/10203641 | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
222, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Korea+82-2-2220-1365
COPYRIGHT © 2021 HANYANG UNIVERSITY.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.