Note
This project is designed for local environments. Do not run it in production.
- Docker
- For the default model, set the memory limit to 6GB or more.
- If a larger model is used, or if there are other active Docker containers in the environment, increase the memory limit further to take into account their memory requirements.
- Copy
.env
and update the values, if necessary:cp .env.example .env
- Start the services:
docker compose up
- Pull the SEA-LION model with Ollama:
docker compose exec ollama ollama pull aisingapore/llama3-8b-cpt-sea-lionv2.1-instruct
- Navigate to http://localhost:8000 to access the chatbot.
- The default model is llama3-8b-cpt-sea-lionv2.1-instruct:q4_k_m.
- If you would like to test the other models, choose the model in https://ollama.com/aisingapore/llama3-8b-cpt-sea-lionv2.1-instruct.
- Check that there is sufficient disk storage and memory. For example, llama3-8b-cpt-sea-lionv2.1-instruct:q8_0 requires 8.5GB of disk storage and 9.3GB of available memory in Docker.
- Pull the model with Ollama.
docker compose exec ollama ollama pull aisingapore/llama3-8b-cpt-sea-lionv2.1-instruct:q8_0
- Update the model name in
.env
.LLM_MODEL=aisingapore/llama3-8b-cpt-sea-lionv2.1-instruct:q8_0
- Please feel free to fork this repo and customise it.
- Upgrade the Chainlit version in the
Dockerfile
to support newer features. - Examples:
- OAuth
- Data Persistence
- Integrations with LangChain or other inference servers
- Kudos to the AI Singapore Team for their good work!