You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Jul 4, 2025. It is now read-only.
Repository navigation
This repository was archived by the owner on Jul 4, 2025. It is now read-only.
What is the data structure of ~/.cortexcpp/models?
Where do the following go
model yaml
model binaries, especially of multiple binaries (model_1_of_5.bin)
versions of the same model, e.g. llama3.1, llama3.2
presets (if any, can also defer for later discussion)
Is our preference on a more flat folder structure? rather than super nested?
Previously, we had many bugs resulting from expecting folder & file names to be a certain way.
e.g. we expected unique_model_id (used by backend) to be the same as the model folder name
Something to be aware of in this iteration 🙏
Model Downloading
What happens when model download fails halfway (e.g. internet disconnected)?
How do we detect models? e.g. if users "import models locally" would it still work
How do we version models?
If we update our model.yaml, or remote model binary in the HF branch, will download still work?
Or will "redownloading/updating" fail due to "model exists"
Letting users do cortex models update is currently out of scope right?
Model importing
Can users import existing models?
Is it hard copy or symlink
Model YAML
We are auto populating the model.yaml if user downloads a new GGUF file (not from our HF repo)?
What happens when user deletes the YAML accidentally?
What happens when YAML is there but Binary is deleted? (should we spec little unit tests like this)
After downloading model (from cortexso or other HF repository), cortex generates tinyllama.yaml file which is used for model management.
Q 2. Is our preference on a more flat folder structure? rather than super nested?
Can you give an example of the flat folder structure?
Q 3. Can you elaborate more about the issue? Do we have any ticket to track that issue yet?
Model importing
Q 1. Can users import existing models?
We don't support it yet. TBD when we will support it.
Q 2. Is it hard copy or symlink
What happens when model download fails halfway (e.g. internet disconnected)?
We don't support resume failed/pause download.
If model download is failed, its <model_id>.yaml (inside <data_folder>/models) won't be created and won't display in our model list.
How do we detect models? e.g. if users "import models locally" would it still work
Scan for <model_id>.yaml file inside <data_folder>/models.
How do we version models?
Currently, we using a field version inside yaml file to store the model's version. Please not that we don't have logic to support model update at the moment. For now, Version is just for display purpose.
3.1. If we update our model.yaml, or remote model binary in the HF branch, will download still work? Or will "redownloading/updating" fail due to "model exists"
It will display model exists.
Letting users do cortex models update is currently out of scope right?
Activity
freelerobot commentedon Sep 4, 2024
Data Folder Questions
Where do the following go
Is our preference on a more flat folder structure? rather than super nested?
Previously, we had many bugs resulting from expecting folder & file names to be a certain way.
e.g. we expected
unique_model_id(used by backend) to be the same as themodel folder nameSomething to be aware of in this iteration 🙏
Model Downloading
cortex models updateis currently out of scope right?Model importing
Model YAML
vansangpfiev commentedon Sep 5, 2024
Data Folder Questions
Q 1. What is the data structure of ~/.cortexcpp/models?
After downloading model (from
cortexsoor other HF repository), cortex generatestinyllama.yamlfile which is used for model management.Q 2. Is our preference on a more flat folder structure? rather than super nested?
Can you give an example of the flat folder structure?
Q 3. Can you elaborate more about the issue? Do we have any ticket to track that issue yet?
Model importing
Q 1. Can users import existing models?
Q 2. Is it hard copy or symlink
cc: @0xSage
nguyenhoangthuan99 commentedon Sep 5, 2024
Model Yaml
cortexso:<model_id>/model.ymland<model_id>.yaml, this 2 files is the same.cortex-cpp models recover, to check and resolve yml error.No such file or directory errorwhen load models.https://cortex.so/docs/model-yaml/is up to date.namchuai commentedon Sep 5, 2024
Model Downloading
<model_id>.yaml(inside <data_folder>/models) won't be created and won't display in our model list.<model_id>.yamlfile inside <data_folder>/models.versioninside yaml file to store the model's version. Please not that we don't have logic to support model update at the moment. For now,Versionis just for display purpose.3.1. If we update our model.yaml, or remote model binary in the HF branch, will download still work? Or will "redownloading/updating" fail due to "model exists"