Knowledge Distillation Logit Distillation

Logit Distillation Logit distillation is a form of transfer learning in which a smaller or simpler student model learns to imitate a larger and more capable teacher model by training directly on the teacher's… excerpt