Ulozto captcha recognizing with 87.5% accuracy

Model was increased.

Test accuracy: 87.5%.
Production accuracy: 82%

There's a certain bias in data the model was trained, validated and tested on. Labels were done by humans so captchas humans weren't able to recognize are not in the dataset. Therefore model will always have some limitations.

Ulozto captcha recognizing with 87.4% accuracy

Test acc: 87.4%
Production acc: ~81%

Ulozto captcha recognizing with 85% accuracy

I changed model to ResNet-like architecture. Now it achieves 85% accuracy on test set.

Ulozto captcha recognizing

Model has 60% accuracy currently.

Ulozto captcha recognizing

Accuracy on test set: 60%.

Ulozto captchas are quite difficult even for humans. We had approximately 80% accuracy solving them ourselves. Note that the model has been trained and tested only on captchas we were able to recognize.

Generic text captcha recognizing

At this point model can predict generic captcha (generated by captcha package) with ~89% accuracy on test set. It was trained on generated model, where every captcha consists of 4 letters, where each of those letters is upper or lower case, and labels for those captchas are case-insensitive (therefore model has 32 classes).

Note: Accuracy is computed as follows - either you guess captcha entirely (1) or you make a mistake (0).

Generic captcha recognizing

At this point model can predict generic captcha (generated by captcha package) with ~98% accuracy. Captchas are currently generated only from numbers 0 to 9999.

Note: Accuracy is computed as follows - either you guess captcha entirely (1) or you make a mistake (0).