Joint Human Pose Estimation and Instance Segmentation with PosePlusSeg
- Authors
- Ahmad, Niaz; Khan, Jawad; Kim, Jeremy Yuhyun; Lee, Youngmoon
- Issue Date
- Feb-2023
- Publisher
- ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE
- Keywords
- Computer Vision (CV); Machine Learning (ML); Intelligent Robotics (ROB); Humans And AI (HAI)
- Citation
- The Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22), v.36, no.1, pp 69 - 76
- Pages
- 8
- Indexed
- SCIE
- Journal Title
- The Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22)
- Volume
- 36
- Number
- 1
- Start Page
- 69
- End Page
- 76
- URI
- https://scholarworks.bwise.kr/erica/handle/2021.sw.erica/112953
- DOI
- 10.1609/aaai.v36i1.19880
- ISSN
- 2159-5399
2374-3468
- Abstract
- Despite the advances in multi-person pose estimation, state-of-the-art techniques only deliver the human pose structure. Yet, they do not leverage the keypoints of human pose to deliver whole-body shape information for human instance segmentation. This paper presents PosePlusSeg, a joint model designed for both human pose estimation and instance segmentation. For pose estimation, PosePlusSeg first takes a bottom-up approach to detect the soft and hard keypoints of individuals by producing a strong keypoint heat map, then improves the keypoint detection confidence score by producing a body heat map. For instance segmentation, PosePlusSeg generates a mask offset where keypoint is defined as a centroid for the pixels in the embedding space, enabling instance-level segmentation for the human class. Finally, we propose a new pose and instance segmentation algorithm that enables PosePlusSeg to determine the joint structure of the human pose and instance segmentation. Experiments using the COCO challenging dataset demonstrate that PosePlusSeg copes better with challenging scenarios, like occlusions, entangled limbs, and overlapped people. PosePlusSeg outperforms state-of-the-art detection-based approaches achieving a 0.728 mAP for human pose estimation and a 0.445 mAP for instance segmentation. Code has been made available at https://github.com/RaiseLab/PosePlusSeg.
- Files in This Item
-
Go to Link
- Appears in
Collections - COLLEGE OF ENGINEERING SCIENCES > DEPARTMENT OF ROBOT ENGINEERING > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/erica/handle/2021.sw.erica/112953)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.