Open-source large language models like Llama 2 can be used by anyone with basic coding knowledge to build on or modify or just use a copy of the model. Building a chatbot using such a model is a relatively inexpensive and quick affair.
Here’s how you can do it:
First off, you will need access to a cloud platform to use the LLM. Self-hosting a model requires a more powerful computer with access to a costly, high-end Nvidia GPU to perform computations. Hosting the model on a cloud platform like Google Cloud, Replicate or RunPod is more feasible option. There are a bunch of others like AWS which offer support for Llama 2 models. (The pricing structure will depend upon the service you choose).
Secondly, you need to download the Llama 2 model to your local machine from the Hugging Face repository or visit the Meta AI website request for the license and submit the form. Once Meta has approved your request, you will receive a pre-signed URL in your email. You can clone the Llama 2 repository from there. If you are downloading them from Hugging Face, you should opt for the GGML version rather than the full version. (GGML is a machine learning library meant to handle large models efficiently using normal hardware. It uses a slightly approximated version of parameters compared to the full version which leads to a little less accuracy but it offers faster inference on CPUs and uses less space and compute)
The Llama 2 model is available in three different sizes: 7-billion parameters, 13-billion parameters and 30-billion parameters. You can use the nifty 2-billion parameter one.
(For top technology news of the day, subscribe to our tech newsletter Today’s Cache)
Next, you need to create an API or Application Programming Interface key by signing into the platform you chose using your GitHub credentials. APIs for large language models help them to interact with the application. One could just get access to a LLM API and connect it with say their Google Sheets directly for analysis. The Llama 2 API is then tested in a Google Colab notebook, which is free to use and essential for coders training models on the cloud.
Finally, to be able to host the code for the model, you also need a docker container which packages the code.
