(无标题)
LeJEPA: a novel pretraining paradigm free of the (many) heuristics we relied on (stop-grad, teacher, ...)
- 60+ arch., up to 2B params
- 10+ datasets
- in-domain training (>DINOv3)
- corr(train loss, test perf)=95%
Paper: https://t.co/NpfB9G1pOP
Code: https://t.co/BsK5wmNEHc https://t.co/8iNoiuHCnr