Vicuna Install

 

Get Linux on Windows

The official instructions for vicuna are here.   We had some trouble on Windows so the first thing we did was get Linux onto our Windows machine.  

In a Power Window type:

   wsl --install

   After this we had to reboot the computer and then it will ask for a Linux user name and password.
   After this the terminal icon will have an Ubuntu option for starting terminals.
   Start a linux terminal and continue with everything below in linux.

Get Vicuna

git clone https://github.com/lm-sys/FastChat.git
cd FastChat
pip3 install --upgrade pip 
pip3 install -e .

Get Weights

cd
mkdir vicuna.weights
Use browser to go to https://huggingface.co/eachadea/vicuna-13b/resolve/main/

Click download next to each file.  Put these in directory vicuna.weights
(note really saved to Windows directory and then moved from /mnt/c/..)

Run Vicuna

cd
cd FastChat
python3 -m fastchat.serve.cli --model-name ~/vicuna.weights --device cpu

Results

This got it running on our CPU.   We would see 75% CPU usage and 61 GB of memory when it is running.

It is slow.   Maybe 1/5th talking speed.   But it does run!
It is amazing to have such impressive AI on our own machine! 







Comments

Popular posts from this blog

Chatbot Arena and Leaderboard

Amazingly easy to run LLMs with Ollama

Orca