I Built a Lightweight Python RAG Orchestrator That Works with SQLite, PGVector and Qdrant

I Built a Lightweight Python RAG Orchestrator That Works with SQLite, PGVector and Qdrant



Tópico: I Built a Lightweight Python RAG Orchestrator That Works with SQLite, PGVector and Qdrant
Categoria: Tutoriais | Programação & Tecnologia
Idioma Principal: Português (Conteúdo de Tecnologia)

Descrição do Conteúdo / Informações:
-------------------------------------------------------------------------
Most RAG frameworks today assume:

• a huge dependency graph

• mandatory LLM orchestration

• opinionated pipelines

• complex configuration

But many real-world systems need something simpler.

Especially when:

• you already have an existing pipeline

• you want local/offline execution

• you need predictable retrieval

• you do not want every step delegated to an LLM

So I built rag-orchestrator.



What makes it different?


The project was designed around one key idea:

RAG infrastructure should be modular, lightweight, and database-agnostic.



Works with multiple vector databases


The orchestrator supports:

• SQLite

• PGVector

• Qdrant

through an abstract storage layer.

This means you can switch backends without rebuilding the whole pipeline.



Fully pluggable architecture


The project provides abstraction layers for:

• Embeddings

• Retrievers

• Cleaners

• Vector stores

• Processing steps

You can easily plug in:

• your own embedding provider

• your own retriever

• custom preprocessing logic

• external pipelines

without rewriting internal logic.



Minimal LLM usage


One important design decision:

The orchestrator works without an LLM for almost the entire pipeline.

LLMs are only required at a single step where they actually add value.

This makes the system:

• cheaper

• faster

• more deterministic

• easier to debug



Minimal configuration


The module intentionally requires very few input parameters.

The goal was:

• fast onboarding

• simple integration

• production-friendly defaults



Tested and production-oriented


The repository already includes:

• integration tests

• runnable scripts

• usage examples

You can inspect them directly in the scripts/ directory.



Easy integration into existing systems


The project was built to integrate into:

• existing RAG pipelines

• enterprise systems

• AI backends

• local AI stacks

• internal search systems

instead of forcing users into a completely new ecosystem.



Installation


```bash id="1b38r0"

pip install rag-orchestrator

## Why this matters

A lot of modern RAG tooling is becoming increasingly framework-heavy.

But many production systems actually need:

* predictability
* portability
* low overhead
* composability

rather than autonomous agent complexity.

This project focuses exactly on that.