Upgrade Transformers and support microsoft/phi-4 and meta-llama/Llama-3.*

[Mingxue-Xu]

Hi, I updated the transformers to 4.48.0 and added the following supported models. Unfortunately, due to the GPU resources, I could only finetune meta-llama/Llama-3.2-1B and facebook/opt-125m (and it ran through successfully). The rest of the models, including the previously supported models like microsoft/phi-2 and microsoft/Phi-3-mini-4k-instruct, were successfully run through run_slicegpt.py and run_lm_eval.py.

Newly supported models:

• microsoft/phi-4
• meta-llama/Llama-3.2-1B (-Instruct)
• meta-llama/Llama-3.2-3B (-Instruct)
• meta-llama/Llama-3.1-8B (-Instruct)

Can anyone please test if all the above models can be run through experiments/run_finetuning.py? Thanks!

[Mingxue-Xu]

@microsoft-github-policy-service agree