Vicuna Install


Get Linux on Windows

The official instructions for vicuna are here.   We had some trouble on Windows so the first thing we did was get Linux onto our Windows machine.  

In a Power Window type:

   wsl --install

   After this we had to reboot the computer and then it will ask for a Linux user name and password.
   After this the terminal icon will have an Ubuntu option for starting terminals.
   Start a linux terminal and continue with everything below in linux.

Get Vicuna

git clone
cd FastChat
pip3 install --upgrade pip 
pip3 install -e .

Get Weights

mkdir vicuna.weights
Use browser to go to

Click download next to each file.  Put these in directory vicuna.weights
(note really saved to Windows directory and then moved from /mnt/c/..)

Run Vicuna

cd FastChat
python3 -m fastchat.serve.cli --model-name ~/vicuna.weights --device cpu


This got it running on our CPU.   We would see 75% CPU usage and 61 GB of memory when it is running.

It is slow.   Maybe 1/5th talking speed.   But it does run!
It is amazing to have such impressive AI on our own machine! 


Popular posts from this blog

Amazingly easy to run LLMs with Ollama

Chatbot Arena and Leaderboard