Local RAG: running Qdrant or Chroma on your own mini PC
This guide helps you assemble a local RAG pipeline on a mini PC using the two tools from our channel posts: EmbeddingGemma 2 for embeddings and PDF Desk for page-level PDF search. It is suitable for macOS/Linux systems; Windows commands are in the README of the project. Because the original posts do not give detailed setup commands for Qdrant or Chroma, the steps below focus on the published components, and we mention vector stores as a possible extension.
Требования и подготовка
- Mini PC capable of running a 740M-parameter model locally.
- Python environment for PDF Desk, a Python application.
- macOS or Linux. If you are on Windows, read the README on GitHub.
- Hugging Face access to download EmbeddingGemma 2.
- PDF Desk repository from GitHub.
- One PDF document for testing — the app supports one active document at a time.
- Optional: a vector database like Qdrant or Chroma for a custom pipeline beyond one PDF.
Before you start, use this checklist to confirm you have everything:
- [ ] Download EmbeddingGemma 2 from Hugging Face
- [ ] Get PDF Desk source from GitHub
- [ ] Install Python and dependencies
- [ ] Prepare a single PDF file
- [ ] Check your mini PC memory for the 740M-parameter model
If you are new to running AI locally, read our beginner guide: How to run AI models locally on your PC: a beginner's guide. For hardware selection, see What Is NPU and How to Choose a Mini PC for Local AI.
Пошаговая инструкция
1. **Download EmbeddingGemma 2 from Hugging Face.**
The model is open and available. Find the EmbeddingGemma 2 page and download the weights.
*Expected result:* model files are stored locally on your mini PC.
2. **Get PDF Desk from GitHub.**
Clone or download the repository.
*Expected result:* you have the project folder with the Python/Streamlit files.
3. **Launch PDF Desk.**
Run the application on macOS or Linux. If you use Windows, follow the commands in the README.
*Expected result:* a Streamlit interface opens in your browser.
4. **Upload one active PDF.**
PDF Desk works with one active document. Select your test PDF.
*Expected result:* the document becomes searchable.
5. **Ask a question about the PDF.**
Type a query in the interface.
*Expected result:* the app returns an answer linked to a specific page number.
6. **Connect EmbeddingGemma 2 to your RAG pipeline.**
Use the downloaded model to embed text, code, images, video and audio into one vector space. If you need a dedicated vector store, send those vectors to Qdrant or Chroma.
*Expected result:* queries and documents map to the same embedding space, enabling semantic search.
The two posts do not describe a full Qdrant or Chroma installation. For vector store setup, use their official documentation. PDF Desk already gives you page-level search; Qdrant or Chroma becomes useful when you want to index many documents or other content types.
Возможные проблемы и решения
- **PDF Desk only handles one active PDF.**
The post explicitly lists this limitation. If you need another document, replace the active file or re-upload before asking the next question.
- **No Windows commands in the post.**
The macOS/Linux commands are shown, but Windows instructions are in the README. Open the GitHub page and find the Windows section.
- **EmbeddingGemma 2 download fails on Hugging Face.**
Check your network connection and confirm you can access the Hugging Face repository. For large models, make sure your mini PC has enough free disk space and RAM.
FAQ
What is EmbeddingGemma 2?
It is an open multimodal embedding model from Google DeepMind with 740 million parameters. It combines text, code, images, video and audio in one vector space.
Can I run local RAG on a mini PC?
Yes. EmbeddingGemma 2 is open and available on Hugging Face, and PDF Desk runs locally on macOS/Linux. Both are light enough for a mini PC with enough RAM.
Which vector database should I choose, Qdrant or Chroma?
The posts do not compare them. For a simple PDF search, PDF Desk handles indexing itself. If you build a larger RAG system, use Qdrant or Chroma based on your preferences and follow their official setup guides.
How does PDF Desk show source pages?
It links the answer to the specific page of the PDF document. That is the core feature described in the post: the model response includes a reference to the relevant page.
🤖 AI summary
- Updated: 2026-10-08
- Topics: локальный RAG, Qdrant, Chroma, векторная база
- Summary: Собираем локальную RAG-систему без облака: векторная база Qdrant/Chroma, эмбеддинги и подключение к локальной LLM.
- Key facts:
- This guide helps you assemble a local RAG pipeline on a mini PC using the two tools from our channel posts: EmbeddingGemma 2 for embeddings and PDF Desk for page-level PDF search.
- It is suitable for macOS/Linux systems; Windows commands are in the README of the project.
- Because the original posts do not give detailed setup commands for Qdrant or Chroma, the steps below focus on the published components, and we mention vector stores as a possible extension.
- Требования и подготовка Mini PC capable of running a 740M-parameter model locally.
- Python environment for PDF Desk, a Python application.
- If you are on Windows, read the README on GitHub.