Experimental setup for LDS computations reported on the plots

[NarineK]

Hi there :)

Thank you for the wonderful package on data attribution!
I have a question related to the experimental setup for the LDS metric calculations for the CIFAR2 + Resnet9 model. I've run the benchmark several times and I'm not able to reproduce the LDS score for the TracIn attribution method for CIFAR2 + Resnet9 model.

I've tried to use all the checkpoints but still the LDS score I obtain is significantly lower than the one reported on your charts. How many train and test samples did you use (the github configuration shows 5000 train and 500 test samples) ?. What parametrization was used for TracIn and the LDS computations for CIFAR2 dataset + Resnet9 and TracIn algorithm ?

How many training runs did you use per subset ?

Thank you!

[Xiang-Pan]

I can not reproduce the results from the cached huggingface repo. It seems the model_half can not generate the score used in paper.