Master, I understand — the stones of other mountains can attack jade

Status:

1. I have been cleaning the data, improving the quality of the data set, and improved the accuracy index to more than 90%. After a lot of efforts, it is difficult to surpass

2. GPUIt’s not enough. It takes a day or two to verify an experiment
3. My goal is more than 95%

stop

In the case of bottlenecks, it is better to stop first, rethink the direction of improvement indicators, and understand each link in the model

inspiration

In the debugging of another ViT project involving OneCycleLR parameter tuning process, I suddenly got inspiration, whether the learning rate setting is not reasonable enough, in fact, there is still room for loss to decrease

Verification immediately, waiting for more than an hour, unexpectedly, the first Epochs has a surprise, which is 0.5% higher than the previous best model index. There should be room for further decline.

PS: To learn more about ViT, click on this link -Vision Transformer