Cortex.cpp: Data Structures

[namchuai]

Overview

• What is Cortex's data structure? (i.e. files that are persisted to disk)
• This should be different from Cortex Platform

An unopinionated (hopefully) to store cortex's files and folders.

Couple of requirements:

• Allow 3rd apps to be config where to store files with ease
• Easy for cortex to grab the configurations
• Easy for remove without interfere with user's data (user's own models, threads, files, etc.)
• Updating cortex should not damage user's data

Discussion

• Should we have a /cortex folder (will there be namespace collisions?)
• Should folder be hidden? (e.g. .cortex
• Where are we storing the config files?

Related

• feat: Respect XDG Base Directory Specification on Linux jan#3310
• feat: update data folder path #1121
• feat: Read application config .cortexrc from file #1031

[vansangpfiev]

I think we can separate application binary and data:

• Execution file: contains only cortex binary
  Windows: C:\Users\<username>\AppData\Local\cortex
  MacOS: /usr/local/bin/cortex
  Ubuntu: /usr/bin/cortex
• Application data:
  Windows: C:\Users\<username>\.cortex
  MacOS: ~/.cortex
  Ubuntu: ~/.cortex

We have different folders to store different application data builds:

~/.cortex
  |___ models
  |___ logs
  |___ engines
          |___ cortex.llamacpp

~/.cortex-beta

~/.cortex-nightly

We also have an application config file which allow user to change the data path and other parameters with the default values mentioned above.
https://github.com/janhq/cortex/issues/1031
cc: @dan-homebrew @namchuai @hiento09

[hiento09]

I put my comment here https://github.com/janhq/cortex/issues/1039#issuecomment-2327901258 cc @dan-homebrew @namchuai @vansangpfiev .
I understand that each channel (stable, beta, nightly) is considered a separate app, and we allow users to install them in parallel. Therefore, the config files should also be kept separate.

(Ex for windows:

• Stable: C:\Users<username>\AppData\Local\cortex\cortex-cpp.exe
• Beta: C:\Users<username>\AppData\Local\cortex-beta\cortex-cpp-beta.exe
• Nighty: C:\Users<username>\AppData\Local\cortex-nightly\cortex-cpp-nightly.exe
  )

[freelerobot]

@vansangpfiev - this is well thought through.

Questions:

1. Does this mean we will also have the following:

~/.cortex-platform
~/.jan

cc @louis-jan @marknguyen1302 for context when thinking through Cortex Platform data folders

So for a user who uses Jan, the most top level application, altogether do they have:

~/.jan
~/.cortex-platform
~/.cortex

2. For naming, I think @dan-homebrew was keen to establish cortexcpp, which is clearer naming
   So is it instead: ~/.cortexcpp

3. Can we break down the following folders more

• models: where is model binaries, yamls (Discussion: Cortex.cpp Model and model.yaml #1090 )
• engines: Discussion: Cortex.cpp Engine Dependencies Architecture (e.g. CUDA Toolkit) #1046
• logs: Discussion: Cortex.cpp Logs #1084

[dan-homebrew]

@0xSage Let's scope this discussion to cortex:

• cortex-platform and jan are handled separately (see discussion with @louis-jan @marknguyen1302)

A few points:

• I think we should use cortex.cpp to refer to Cortex (easier to SEO)
• The folder can then be cortex.cpp (reduces chance for edge case where user already has a /cortex)

Additionally, should this folder be hidden? (e.g. ~/.cortex.cpp or ~/cortex.cpp)?

[vansangpfiev]

I agree, we can use cortex.cpp instead of cortex. And I think the data folder should be hidden ~/.cortex.cpp (like .ssh)
cc: @dan-homebrew @hiento09

[freelerobot]

Should we put configuration file .cortexrc inside the data folder .cortex.cpp, i.e related data together, making it easier for users to find and manage their configuration files.

It's weird if cortex config is in ~, but engine configs are in .cortex.cpp

[dan-homebrew]

I think it matches how .vimrc, etc work - all in root

…

On Wed, 4 Sep 2024 at 2:00 PM, Nicole Zhu ***@***.***> wrote: Should we put configuration file .cortexrc inside the data folder .cortex.cpp, i.e related data together, making it easier for users to find and manage their configuration files. It's weird if cortex config is in ~, but engine configs are in .cortex.cpp — Reply to this email directly, view it on GitHub <#1040 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AYDVXFTRXAMA3ZILZJBLI2DZU2OXVAVCNFSM6AAAAABNHSW42WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRXHE4DKNRTHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[hiento09]

Should we put configuration file .cortexrc inside the data folder .cortex.cpp, i.e related data together, making it easier for users to find and manage their configuration files.

It's weird if cortex config is in ~, but engine configs are in .cortex.cpp

The purpose of the .cortexrc file is to customize configurations such as the data folder, port, listen address, etc. For example, the .cortex.cpp folder is the app data folder specified in the .cortexrc file, and its default value is ~/.cortex.cpp.
Ref https://github.com/janhq/cortex/issues/1031

[freelerobot]

Nit: naming a data folder: ~/.cortex.cpp might be confusing for users. Some users might expect a .cpp file extension to represent a C++ source file, not a folder.

And should we be consistent across package names & folder names, e.g.

Option 1
brew install cortexcpp
~/.cortexcpp

Option 2
brew install cortex-cpp
~/.cortex-cpp

Sorry to nitpick, but this is one of those things where changing in the future is a pain.

[hiento09]

Yep, I completely agree that the app/package name and the app data folder should be consistent with each other to avoid confusing the user. Now we just need to finalize the name.

[hiento09]

@0xSage Let's scope this discussion to cortex:

• cortex-platform and jan are handled separately (see discussion with @louis-jan @marknguyen1302)

A few points:

• I think we should use cortex.cpp to refer to Cortex (easier to SEO)
• The folder can then be cortex.cpp (reduces chance for edge case where user already has a /cortex)

Additionally, should this folder be hidden? (e.g. ~/.cortex.cpp or ~/cortex.cpp)?

@dan-homebrew , the naming of the folder and the binary file is up to us. However, I searched and found that no one uses a dot in their package name or app name. You can refer to it here: https://github.com/Homebrew/homebrew-core/tree/master/Formula/c. I think we should consider @0xSage 's options 1 and 2 above. Including a character that violates naming rules for package names or app names on other platforms (brew, npm) could cause us significant difficulties later on.

[freelerobot]

Cool, I suggest just cortexcpp then?

From a product perspective:

1. Official name CortexCPP
2. package names cortexcpp
3. folder names cortexcpp

[irfanpena]

Is the name finalized? Should I also change from Cortex to CortexCPP in docs and also Readme?

[namchuai]

If I have one vote then I will vote for option 1

[dan-homebrew]

@namchuai @vansangpfiev @nguyenhoangthuan99 @hiento09 Did we make a decision on whether it should be a hidden folder, or a visible folder?

Referring to my post in #1092: Folder name should be ~/.cortexcpp

Cortex Model Folder Visibility

Should it be a visible folder instead? i.e. ~/cortexcpp

Application data:

• Windows: C:\Users<username>\cortexcpp
• MacOS: ~/cortexcpp
• Ubuntu: ~/cortexcpp

We will have different folders for nightly, i.e:

• ~/cortexcpp-beta
• ~/cortexcpp-nightly

Config File

• ~/.cortexrc

Execution file (Cortex Binary)

• contains only cortex binary
• Windows: C:\Users<username>\AppData\Local\cortex
• MacOS: /usr/local/bin/cortex
• Ubuntu: /usr/bin/cortex

[dan-homebrew]

I've also updated the naming structure of Cortex files that the Installer + Uninstaller will track:

#1039 (comment)

Installer:

• Windows
  • Format: .exe
  • Binary location:
    • Stable: C:\Users\<username>\AppData\Local\cortexcpp\cortex.exe
    • Beta: C:\Users\<username>\AppData\Local\cortexcpp-beta\cortex-beta.exe
    • Nighty: C:\Users\<username>\AppData\Local\cortexcpp-nightly\cortex-nightly.exe
  • Configuration file:
    • Stable: C:\Users\<username>\.cortexrc
    • Beta: C:\Users\<username>\.cortexrc-beta
    • Nighty: C:\Users\<username>\.cortexrc-nightly
  • Data folder (contains engines, models, logs)
    • Stable: C:\Users\<username>\.cortexcpp
    • Beta: C:\Users\<username>\.cortexcpp-beta
    • Nighty: C:\Users\<username>\.cortexcpp-nightly
• Linux:
  • Format: .deb
  • Binary location:
    • Stable: /usr/bin/cortexcpp
    • Beta: /usr/bin/cortexcpp-beta
    • Nighty: /usr/bin/cortexcpp-nightly
  • Configuration file:
    • Stable: /home/<username>/.cortexrc
    • Beta: /home/<username>/.cortexrc-beta
    • Nighty: /home/<username>/.cortexrc-nightly
  • Data folder (contains engines, models, logs)
    • Stable: /home/<username>/.cortexcpp
    • Beta: /home/<username>/.cortexcpp-beta
    • Nighty: /home/<username>/.cortexcpp-nightly
• Macos:
  • Format: .pkg
  • Binary location:
    • Stable: /usr/local/bin/cortexcpp
    • Beta: /usr/local/bin/cortexcpp-beta
    • Nighty: /usr/local/bin/cortexcpp-nightly
  • Configuration file:
    • Stable: /Users/<username>/.cortexrc
    • Beta: /Users/<username>/.cortexrc-beta
    • Nighty: /Users/<username>/.cortexrc-nightly
  • Data folder (contains engines, models, logs)
    • Stable: /Users/<username>/.cortexcpp
    • Beta: /Users/<username>/.cortexcpp-beta
    • Nighty: /Users/<username>/.cortexcpp-nightly

[namchuai]

Hmm, so, is the cortex data folder be visible or not? .cortexcpp or cortexcpp. Let's confirm on this.

I think:

• configuration file .cortexrc<env> should be hidden.
• data folder cortexcpp<env> should be visible.

@vansangpfiev @dan-homebrew @0xSage @nguyenhoangthuan99

[namchuai]

Conclusion:

configuration file .cortexrc<env> should be hidden.
data folder cortexcpp<env> should be visible.

[dan-homebrew]

Thanks @namchuai

[freelerobot]

Context for the decision, user want to see where their model binaries are stored and be able to easily manage it, portability

[irfanpena]

@dan-homebrew since the naming is finalized, Should I also change from Cortex to CortexCPP in docs and also Readme?

[freelerobot]

@irfanpena for docs naming, when we refer to cortex, we should say cortex.cpp or Cortex.cpp if at beginning of sentence 🙏