66th ISI World Statistics Congress

66th ISI World Statistics Congress

Bridging hard and soft clustering via L0 regularization

Conference

66th ISI World Statistics Congress

Format: WSC 2027: IPS Abstract

Keywords: fuzzy clustering, regularization

Abstract

A widely known property of fuzzy-based algorithms is that they produce soft assignments of units to clusters through the so-called membership degrees taking values in the unit interval. The closer a membership degree is to one, the stronger the assignment of a unit to a cluster. Consequently, fuzzy clustering algorithms are more flexible than standard clustering tools that strictly assign a unit to only one cluster. Such a flexibility is extremely useful in real life situations, where units often share the features of more than one cluster. However, a side effect emerges because even units clearly belonging to a single cluster tend to have non-zero membership degrees to all clusters. To overcome such a limitation, L0 regularization may be introduced to promote sparsity in the membership degree matrix. This ensures that units clearly belonging to a single cluster have a membership degree strictly equal to one for that cluster, while still allowing for soft membership degrees of the units with ambiguous assignments. The talk offers an overview of fuzzy clustering with L0 regularization and its use for complex data structures.