bug: DeepSeek-Coder-V2-Lite-Instruct won't load

[rotilho]

• I have searched the existing issues

Current behavior

• Downloads DeepSeek-Coder-V2-Lite-Instruct
• DeepSeek don't load

Minimum reproduction step

• Download DeepSeek-Coder-V2-Lite-Instruct
• Load it

Expected behavior

Model loads and run fine

Screenshots / Logs

2024-06-23T16:00:17.821Z [CORTEX]::Error: llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'deepseek2'
llama_load_model_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model '/var/home/rotilho/jan/models/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf'

2024-06-23T16:00:17.821Z [CORTEX]::Debug: {"timestamp":1719158417,"level":"ERROR","function":"LoadModel","line":167,"message":"llama.cpp unable to load model","model":"/var/home/rotilho/jan/models/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf"}
20240623 16:00:17.821756 UTC 326224 ERROR Error loading the model - llama_engine.cc:385

2024-06-23T16:00:17.822Z [CORTEX]::Debug: Load model success with response {}
2024-06-23T16:00:17.823Z [CORTEX]::Debug: Validate model state with response 409
2024-06-23T16:00:17.823Z [CORTEX]::Debug: Validate model state failed with response "Conflict"
2024-06-23T16:00:17.823Z [CORTEX]::Error: Validate model status failed

Jan version

0.5

In which operating systems have you tested?

• macOS
• Windows
• Linux

Environment details

• Fedora 40
• AMD Ryzen 7040
• 96gb

[Van-QA]

@rotilho Can you give us the full app.log?

Not reproducible on our Linux using Jan nightly build, would you like to try it as well?

[rotilho]

Sure thing! I'll try in the latest build later today or this weekend.

jan.log

[rotilho]

No success with nightly build

jan-linux-amd64-0.5.1-497-AppImage: It didn't load
jan-linux-amd64-0.5.1-495-AppImage: same error
jan-linux-amd64-0.5.1-492-AppImage: same error

[vansangpfiev]

Hi @rotilho, could you please delete the extensions folder in the path /var/home/rotilho/jan/extensions then start Jan and try again?
Seems like there is a bug in installation so our extensions are not updated.
From the jan.log:

20240627 17:37:34.529677 UTC 51983 INFO  cortex.llamacpp version: 0.1.11 - main.cc:78

Expected: something like

20240628 00:13:45.487000 UTC 2748 INFO  cortex.llamacpp version: 0.1.17-19.06.24 - main.cc:78

cc: @Van-QA

[rotilho]

Deleting the extensions folder worked. Thanks guys!