Skip to content

Supporting downloading Hugging Face models by hf_repo_id from models.yaml when weights are missing #193

@Vismayak

Description

@Vismayak

Is your feature request related to a problem? Please describe.

vec-inf currently expects model weights to already exist in the shared cache (/model-weights/<model_name>). If weights are missing, launches fail and users must request an admin download. There is a PR linked to Issue #166 that adds logic to download weights from Hugging Face when given an HF model id (e.g. via --hf-model). What’s missing is a way to store that HF id in the model config, so the system can fall back to downloading automatically when the cache doesn’t have the weights.

Describe the solution you'd like

Add an optional field in models.yaml for each model, e.g. hf_id: / (or reuse the PR’s hf_model name). When model_weights_path does not exist for a requested model_name, vec-inf should read hf_id from models.yaml and reuse the PR #166/167 logic to set the model source to that HF id so the weights are downloaded/cached automatically.

Additional context

This will use similar logic to the #167 to save model weights

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions