Skip to content

Latest commit

 

History

History
103 lines (71 loc) · 3.59 KB

README.md

File metadata and controls

103 lines (71 loc) · 3.59 KB
Raw

OWTTT on CIFAR10-C/100-C

Ours method and the baseline method TEST (direct test without adaptation) on CIFAR-10-C/100-C under common corruptions or natural shifts. Our implementation is based on repo and therefore requires some similar preparation processes.

Requirements

To install requirements:

pip install -r requirements.txt

To download datasets:

export DATADIR=/data/cifar
mkdir -p ${DATADIR} && cd ${DATADIR}
wget -O CIFAR-10-C.tar https://zenodo.org/record/2535967/files/CIFAR-10-C.tar?download=1
tar -xvf CIFAR-10-C.tar
wget -O CIFAR-100-C.tar https://zenodo.org/record/3555552/files/CIFAR-100-C.tar?download=1
tar -xvf CIFAR-100-C.tar
wget -O tiny-imagenet-200.zip http://cs231n.stanford.edu/tiny-imagenet-200.zip
unzip tiny-imagenet-200.zip

Pre-trained Models

The checkpoints of pre-train Resnet-50 can be downloaded (214MB) using the following command:

mkdir -p results/cifar10_joint_resnet50 && cd results/cifar10_joint_resnet50
gdown https://drive.google.com/uc?id=1QWyI8UrXJ6_H9lBbrq52qXWpjdpq4PUn && cd ../..
mkdir -p results/cifar100_joint_resnet50 && cd results/cifar100_joint_resnet50
gdown https://drive.google.com/uc?id=1cau93HVjl4aWuZlrl7cJIMEKBxXXunR9 && cd ../..

These models are obtained by training on the clean CIFAR10/100 images using semi-supervised SimCLR.

Open-World Test-Time Training:

We present our method and the baseline method TEST (direct test without adaptation) on CIFAR10-C/100-C.

  • run OURS method or the baeline method TEST on CIFAR10-C under the OWTTT protocol.

    # OURS: 
bash scripts/ours_cifar10.sh "corruption_type" "strong_ood_type" 

# TEST: 
bash scripts/test_cifar10.sh "corruption_type" "strong_ood_type"

    Where "corruption_type" is the corruption type in CIFAR10-C, and "strong_ood_type" is the strong OOD type in [noise, MNIST, SVHN, Tiny, cifar100].

    For example, to run OURS or TEST on CIFAR10-C under the snow corruption with MNIST as strong OOD, we can use the following command:

    # OURS:
bash scripts/ours_cifar10.sh snow MNIST 

# TEST:
bash scripts/test_cifar10.sh snow MNIST

    The following results are yielded by the above scripts (%) under the snow corruption, and with MNIST as strong OOD:

    Method ACC_S ACC_N ACC_H
    TEST 66.36 91.56 76.95
    OURS 84.05 97.46 90.26

  • run OURS method or the baeline method TEST on CIFAR100-C under the OWTTT protocol.

    # OURS: 
bash scripts/ours_cifar100.sh "corruption_type" "strong_ood_type" 

# TEST: 
bash scripts/test_cifar100.sh "corruption_type" "strong_ood_type"

    Where "corruption_type" is the corruption type in CIFAR100-C, and "strong_ood_type" is the strong OOD type in [noise, MNIST, SVHN, Tiny, cifar10].

    For example, to run OURS or TEST on CIFAR100-C under the snow corruption with MNIST as strong OOD, we can use the following command:

    # OURS:
bash scripts/ours_cifar100.sh snow MNIST 

# TEST:
bash scripts/test_cifar100.sh snow MNIST

    The following results are yielded by the above scripts (%) under the snow corruption, and with MNIST as strong OOD:

    Method ACC_S ACC_N ACC_H
    TEST 29.2 53.27 37.72
    OURS 44.78 93.56 60.57

Acknowledgements

Our code is built upon the public code of the TTAC.