Keywords
Fine-Tuning; Zero-Order Optimization; Deep Neural Network Adaptation; Fine-Tuning Generalizability; Fine-Tuning Guide; Curvature-Aware Optimization
Abstract
Fine-tuning is the process of teaching and specializing a pre-trained neural network on a downstream task. Fine-tuning is a rapidly growing topic in artificial intelligence domains; however, many fine-tuning endeavors are highly specialized without a coherent framework connecting them. This work presents a unified perspective on fine-tuning methods and performance metrics. Our perspective organizes the methods in terms of how they are applied to fine-tuning. This framework showcases methods that (i) update effective subspaces of the pre-trained model, (ii) change the adaptation optimization procedure, and (iii) alter the representations of the embedded input. Additionally, we present unconventional metrics such as generalizability, robustness, loss landscapes, and intruder dimensions that highlight additional perspectives relevant to model construction. By unifying both fine-tuning methods and evaluation techniques through a common lens, this work serves as a manual for designing modern networks. As a result, this work intends to inform users of both empirical and theoretical developments in fine-tuning deep neural networks.
Thesis Completion Year
2026
Thesis Completion Semester
Spring
Thesis Chair
Dutta, Aritra
College
College of Engineering and Computer Science
Department
Department of Computer Science and Department of Mathematics
Thesis Discipline
Computer Science, Artificial Intelligence, Mathematics
Language
English
Access Status
Open Access
Length of Campus Access
None
Campus Location
Orlando (Main) Campus
Notes
UCF Double Major; Computer Science from CECS and Mathematics from COS
STARS Citation
McGee, Cristian S., "Beyond Full Fine-Tuning: The New Playbook For Adapting Deep Neural Networks" (2026). Honors Undergraduate Theses. 475.
https://stars.library.ucf.edu/hut2024/475
Included in
Artificial Intelligence and Robotics Commons, Other Applied Mathematics Commons, Theory and Algorithms Commons
Accessibility Statement
This item was created or digitized prior to April 24, 2027, or is a reproduction of legacy media created before that date. It is preserved in its original, unmodified state specifically for research, reference, or historical recordkeeping. In accordance with the ADA Title II Final Rule, the University Libraries provides accessible versions of archival materials upon request. To request an accommodation for this item, please submit an accessibility request form.
Rights Statement
