SCAMP-5 Inference Accuracy

[HenryMaxixi]

Thank you for your impressive work. I encountered an issue when running inference on SCAMP-5 using the code. I used CamGes to save images to test_outputs_empty_folders, and after running test_scamp_outputs.py, I obtained an accuracy of 14.44. However, when I used scamp_outputs/cambridge_all_oct2023_r4 folder, I got a result of 84.4.

Could this be due to the fact that I haven’t been regularly reloading the weights on SCAMP-5? I reload once a time after getting results for one class using the SCAMP host.

This is the image I get after inferring one class, and it seems that the weights show significant differences. Additionally, the output ot differs greatly from the images provided in scamp_outputs folder.

[haleyso]

Hi Henry! Replied in email, but in case others are running into some issues too, I'll repeat the email here

I uploaded a newer version of the scamp code. (Sorry for any inconvenience this may have caused!)

But looking at your weights, it looks quite noisy. If it still looks like this after trying the new code, then I think it will be very important to finetune the linear layer since the noise characteristics of our two scamps will be very different. For reference, this is a screenshot I just took today of the scamp outputs. You can see the weight planes are much clearer. Regardless, I think it is important to finetune the linear layer since there are also affects from thermals. The longer your SCAMP is running for example, the noisier everything will be. I think in the future versions of SCAMP, it is much less noisy so perhaps you wouldn't have to finetune then, but for now, unfortunately you may have to.