Multitask Learning with Heterogeneous Tasks

Abstract

It is general that more computation resources are required to achieve better performance of CNNs. This is even more severe when we perform multiple tasks. Multitask learning (MTL) can address this problem by sharing architecture across tasks. Previous MTL studies obtained a feature from a shared backbone or module and performed multiple tasks. However, learning from heterogeneous tasks has not gained much attention. For example, input data has different properties, such as resolutions and object classes; thus, the aggregation of the features generated from heterogeneous tasks is complicated. To mitigate this problem, we propose a simple learning strategy that generalizes the shared representation using multiple datasets for heterogeneous tasks. We evaluate the strategy using four datasets and show that learning from different tasks jointly gives a comparable performance to learning individual tasks and outperforms the existing strategy for some tasks. © 2022 IEEE.