#pretrained-models2 articles

2 articles

AINeutralarXiv – CS AI · 5d ago7/105

🧠

Robust Fine-Tuning from Non-Robust Pretrained Models: Mitigating Suboptimal Transfer With Epsilon-Scheduling

Researchers identified that fine-tuning non-robust pretrained AI models with robust objectives can lead to poor performance, termed 'suboptimal transfer.' They propose Epsilon-Scheduling, a novel training technique that adjusts perturbation strength during training to improve both task adaptation and adversarial robustness.

AIBullisharXiv – CS AI · 5d ago6/102

🧠

Inner Loop Inference for Pretrained Transformers: Unlocking Latent Capabilities Without Training

Researchers propose a new inference technique called "inner loop inference" that improves pretrained transformer models' performance by repeatedly applying selected layers during inference without additional training. The method yields consistent but modest accuracy improvements across benchmarks by allowing more refinement of internal representations.