CMLsearch: Semantic visual search and simulation through segmented colour, material, and lighting in interior image

Abstract

In product search systems, user behaviour changes according to their intentions, requiring adaptations in system requirements and information modelling. When purchasing home decor products, users must consider their existing home setting (EHS) and the need to pair multiple elements, not just a single product. However, no existing home decor search systems assist with varied search intents (target-finding and decision-making scenarios), nor have they focused on research that helps pair various elements of a user's EHS. Therefore, we introduce CMLsearch: a semantic visual search system that segments Colour, Material, and Lighting (CML), and includes light correlated colour temperature (CCT) simulation. In a user study (N = 44), CMLsearch significantly improved user satisfaction and purchasing decisions compared with conventional systems. The semantic visual search reflected user intent, offering object-level control that supported more focused searches in target-finding scenarios and broader exploration in decision-making scenarios. The light CCT simulation further boosted confidence by allowing users to visualize the products under different lighting conditions.