Parametric Classification for Generalized Category Discovery: A Baseline Study

Parametric Classification for Generalized Category Discovery: A Baseline Study (ICCV 2023)
By Xin Wen*, Bingchen Zhao*, and Xiaojuan Qi.

Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples. Previous studies argued that parametric classifiers are prone to overfitting to seen categories, and endorsed using a non-parametric classifier formed with semi-supervised $k$-means.

However, in this study, we investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We demonstrate that two prediction biases exist: the classifier tends to predict seen classes more often, and produces an imbalanced distribution across seen and novel categories. Based on these findings, we propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers. We hope the investigation and proposed simple framework can serve as a strong baseline to facilitate future studies in this field.

Running

Dependencies

pip install -r requirements.txt

Config

Set paths to datasets and desired log directories in config.py

Datasets

We use fine-grained benchmarks in this paper, including:

• The Semantic Shift Benchmark (SSB) and Herbarium19

We also use generic object recognition datasets, including:

• CIFAR-10/100 and ImageNet-100/1K

Scripts

Train the model:

bash scripts/run_${DATASET_NAME}.sh

We found picking the model according to 'Old' class performance could lead to possible over-fitting, and since 'New' class labels on the held-out validation set should be assumed unavailable, we suggest not to perform model selection, and simply use the last-epoch model.

Results

Our results:

Source	Paper (3 runs)			Current Github (5 runs)
Dataset	All	Old	New	All	Old	New
CIFAR10	97.1±0.0	95.1±0.1	98.1±0.1	97.0±0.1	93.9±0.1	98.5±0.1
CIFAR100	80.1±0.9	81.2±0.4	77.8±2.0	79.8±0.6	81.1±0.5	77.4±2.5
ImageNet-100	83.0±1.2	93.1±0.2	77.9±1.9	83.6±1.4	92.4±0.1	79.1±2.2
ImageNet-1K	57.1±0.1	77.3±0.1	46.9±0.2	57.0±0.4	77.1±0.1	46.9±0.5
CUB	60.3±0.1	65.6±0.9	57.7±0.4	61.5±0.5	65.7±0.5	59.4±0.8
Stanford Cars	53.8±2.2	71.9±1.7	45.0±2.4	53.4±1.6	71.5±1.6	44.6±1.7
FGVC-Aircraft	54.2±1.9	59.1±1.2	51.8±2.3	54.3±0.7	59.4±0.4	51.7±1.2
Herbarium 19	44.0±0.4	58.0±0.4	36.4±0.8	44.2±0.2	57.6±0.6	37.0±0.4

Citing this work

If you find this repo useful for your research, please consider citing our paper:

@inproceedings{wen2023simgcd,
    author    = {Wen, Xin and Zhao, Bingchen and Qi, Xiaojuan},
    title     = {Parametric Classification for Generalized Category Discovery: A Baseline Study},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    year      = {2023},
    pages     = {16590-16600}
}

Acknowledgements

The codebase is largely built on this repo: https://github.com/sgvaze/generalized-category-discovery.

License

This project is licensed under the MIT License - see the LICENSE file for details.