Agents in the database, not in the repo: a tour of apowerb

Iniciado por joomlamz, Hoje at 10:25

Respostas: 1   |   Visualizações: 2

Tópico anterior - Tópico seguinte

0 Membros e 1 Visitante estão a ver este tópico.

Saudações aos colegas do **webmastersmz.com**. Como especialista em tecnologia, analisei o conceito apresentado em *"Agents in the database, not in the repo: a tour of apowerb"*. Este artigo traz uma mudança de paradigma bastante interessante para a forma como arquitetamos aplicações modernas baseadas em IA.

### Análise Técnica: Agentes no Banco de Dados vs. Repositório

O ponto central deste texto é a transição de agentes (agentes de IA, scripts de automação ou lógica de negócio) do código-fonte tradicional (o repositório Git) para a camada de persistência (o banco de dados). Esta abordagem, frequentemente denominada *Data-as-Code* ou *Agent-as-Data*, apresenta vantagens e desafios que devemos considerar:

1.  **Dinamismo e Escalabilidade:** Ao armazenar a lógica dos agentes no banco de dados (como o `apowerb` propõe), ganhamos a capacidade de atualizar o comportamento dos agentes em tempo real sem a necessidade de passar por um ciclo de *CI/CD* (Integração Contínua e Entrega Contínua). Isto é extremamente útil em sistemas onde a adaptabilidade é crítica.
2.  **Gestão de Estado:** A integração direta com o banco permite que o agente consulte o estado atual dos dados antes de executar uma tarefa, reduzindo a latência e a complexidade de chamadas externas de API para buscar contexto.
3.  **Complexidade na Manutenção:** O contraponto é a dificuldade de versionamento. Se o código do agente está no banco, como garantimos o histórico de alterações (*git blame*)? Como realizamos testes unitários de forma eficaz fora do ambiente de desenvolvimento tradicional? A falta de um "repositório" pode complicar o controlo de qualidade.
4.  **Segurança:** Mover lógica para o banco exige uma camada de segurança robusta. Executar código dinâmico vindo da base de dados é um risco clássico de segurança que deve ser mitigado com sandboxing rigoroso para evitar injeções ou execução de código malicioso.

**Pergunta para o debate:** Vocês acham que a flexibilidade de atualizar agentes em tempo real compensa a perda de rastreabilidade que o Git oferece, ou estamos a trocar a robustez de engenharia por uma agilidade perigosa? Como geririam o versionamento de algo que "vive" apenas na base de dados?

Gostaria de ver as vossas opiniões e experiências sobre este modelo. Vamos enriquecer esta discussão aqui no fórum!

---

Para garantir que os vossos projetos e fóruns rodam sem falhas, com a estabilidade e a velocidade que estas tecnologias modernas exigem, convido-vos a conhecer as soluções de alojamento de alta performance da **AplicHost** em https://aplichost.com.

Agents in the database, not in the repo: a tour of apowerb



Tópico: Agents in the database, not in the repo: a tour of apowerb
Categoria: Tutoriais | Programação & Tecnologia
Idioma Principal: Português (Conteúdo de Tecnologia)

Descrição do Conteúdo / Informações:
-------------------------------------------------------------------------
The usual way to define an agent is to write it: a Python file that declares a model, attaches tools, carries the instruction. You commit it, you deploy it. That is a good default — the definition is versioned, reviewable, testable — and it holds right up to the moment a non-engineer needs to change a prompt, or you are running forty agents for twelve teams and every wording fix has become a release.

apowerb takes the other road: an agent is a row in Postgres. The Python that the runtime imports is generated, disposable, and seven lines long — two of which do anything. Changing an agent is an UPDATE, not a deploy.

It's Apache-2.0, built on FastAPI and Google ADK (the Agent Development Kit, Google's agent SDK), and it self-hosts with Docker Compose. This post walks through the parts of the design that are actually interesting — the ones we got wrong at least once first.

Creating an agent in the UI. Name, category, provider — no file, no commit. What this form writes is a database row.



The generated module


When you create an agent, the framework writes a directory under agents_pool/ named after the agent's id, and puts this in it (src/apowerb/core/adk_agent_builder.py):

# agents_pool/agent1/agent.py, read out of a running container

# th2agent modules
from apowerb.core.agent_helpers import to_agent

# declare env variables from stored in agent_model_params
root_agent = to_agent(agent_name = 'agent1')
# Create the agent

That's the whole module, comments and stale package name included. It is a stub: a name ADK can import, pointing at a loader. Everything that makes agent1 what it is — instruction, model, tools, sub-agents, guardrails, output schema — is read from the database at load time by to_agent (src/apowerb/core/agent_helpers/agent_utils.py, abridged):

def to_agent(agent_name: str) -> LlmAgent:
"""Convert the schema to an Agent instance."""
agent_id = int(agent_name.replace("agent", ""))
agent_details = get_agent_details(agent_id=agent_id)

tools_ids = _parse_string_list(agent_details.get("agent_tools"))
tools_names, tools_funcs = load_agent_tools_functions(
tools=tools_ids, owner_id=owner_id
)
load_mcp_servers(agent_details.get("mcp_servers"), tools_funcs)
load_agent_skills_toolset(agent_name, agent_details.get("agent_skills"), tools_funcs)
...

We checked rather than trusting the code: on a stack booted from the published images, we created one agent named article_demo in the interface, and then looked at both ends of the chain. In Postgres:

agent_id |  agent_name  | agent_type |             agent_model
----------+--------------+------------+--------------------------------------
1 | article_demo | base       | anthropic/claude-sonnet-4-5-20250929

And on disk, agents_pool/agent1/agent.py — the stub above, written by the server the moment the row appeared. The model is a string in a column. The agent type is a column. Note also what the form lets you leave empty: the API key. A definition can exist without a credential; it just can't answer until one is there.

The payoff is that the editing surface of an agent is an API and a UI, not a repository. The cost is that the filesystem now holds derived state — and derived state drifts.



Derived state drifts, so boot repairs it


An agents_pool/ directory and a database can disagree. Someone restores a volume without the database. Someone syncs one environment onto another and loses a folder. A package rename leaves a stub that imports a module that no longer exists. All three failures look identical at runtime: a ModuleNotFoundError the first time a user talks to that agent.

So the boot sequence reconciles instead of trusting. The repair function says what it covers:

def ensure_agent_modules(agents_pool_path: str | None = None) -> None:
"""Auto-repair: regenerate missing *or stale* agent.py files for every agent.

Called at startup, so an environment heals itself on its next restart
rather than waiting for someone to save each agent by hand.
"""

"Stale" is the interesting half. A present-but-broken file is worse than a missing one, because a naive os.path.exists check calls it healthy. The repair reads the stub and looks for the single import line every generated stub carries; a file that no longer contains it is rewritten like a missing one. Deliberately coarse, and deliberately narrow: a stub somebody has genuinely customised, which still imports the core, is left alone.

The whole startup path lives in one bootstrap() behind the ASGI lifespan, and the docstring says why (it is written in French in the repo; translation mine):

This all used to run at module level: importing apowerb.main triggered 10

migrations against a real database and wrote into agents_pool/. A plain import was

worth a boot — which made the package unusable as a dependency.

If you ship a Python package whose import does migrations, you don't have a library, you have a side effect with a name.



The third state: the agent already in memory


Two states have to agree, and that is what boot repairs. There is a third, and it is the one that makes the opening claim true or false: the agent object a running process has already built.

to_agent reads the database at load time. Load it once and the process keeps it — the module in sys.modules, the agent in the loader's cache, an ADK runner wrapping it. Change the row and nothing moves. A user with a chat open would keep talking to yesterday's instruction until somebody restarted the backend, which is precisely the deploy the design was supposed to remove.

So a save invalidates all three. POST /api/agents/{id}/reload (src/apowerb/routers/agent_reload.py) drops the agent's modules from sys.modules and from the loader cache, then queues its runner for cleanup; the next get_runner_async closes the old runner and rebuilds it, which re-imports the module, which re-runs to_agent against the current row. The frontend calls it right after saving the row. Once that invalidation has gone through, the next message in an already-open conversation gets an agent rebuilt from the new configuration — no "New chat", no restart.

Three honest caveats, all visible in the code. The reload is best-effort at every step: the frontend fires it without awaiting it, the endpoint wraps both invalidations in try/except, and it answers 204 either way. So a save can succeed while the reload fails. That failure is logged — console.warn in the browser, logger.warning on the server — but nothing in the interface raises it: you get the save toast, and the old definition can stay live. Second, because the frontend never awaits it, that toast is not a promise the runtime has caught up; a message sent the instant it appears can reach the server before the invalidation does. Third, the rebuild is deferred to the next runner request rather than done on the spot — and what happens to a run already in flight at that moment is not something we have exercised end to end.

There is also a scar in it worth keeping:

# Only queue the runner for cleanup if one actually exists: ADK's
# close_runners([None]) crashes with ``'NoneType' object has no
# attribute 'close'`` otherwise.

An agent that has never been talked to in this process has no runner, and asking ADK to close it raised an AttributeError on a route whose whole job was to be invisible.



What that costs


The objection lands immediately, and the repo does not answer all of it. A definition that lives in a table is a definition that has left git, so:


Concurrent edits are last-write-wins. The agent table — th2agents_store, a name
older than the project's own — carries created_at and updated_at and no revision column; there is no optimistic-locking token and no If-Match on the update route. Two people saving the same agent is a race, and the loser gets no warning.


There is no prompt history to roll back to. What is tracked is narrower:
a template's superagent_template_version_hash, and hub_origin_id on an agent cloned from the hub. Your own edit yesterday is gone.


Testing an agent means having a database. The definition is a row, so a unit test
that wants a real agent wants Postgres. The suite works around it, but that is a cost the file-based approach doesn't have.

Worth saying plainly, because the honest comparison is not "database beats files". It's that you are trading the guarantees git gave you for the ability to change an agent without a deploy. If your agents are written once by engineers and rarely touched, a YAML file and a hot reload is the smaller tool and you should use it.

The same agent on the canvas. The graph is derived from the stored definition, not the other way round.



One column for the model


Agent code never names a provider. The agent_model column does, and build_litellm_model turns it into an ADK LiteLlm instance (src/apowerb/core/agent_helpers/llm_model_builder.py). Anthropic, OpenAI, Mistral, Google, OVHcloud, or anything OpenAI-compatible behind a custom api_base — same agent, different row.

Two details worth stealing:

Custom endpoints get forced onto the OpenAI path. A self-hosted mistral/... model behind an OpenAI-compatible gateway will fail tool calling if litellm routes it through the Mistral SDK, so the provider prefix is rewritten:

if model_api_base:
_model_name = agent_details["agent_model"]
# When using a custom OpenAI-compat api_base, force the openai/ provider prefix
# so litellm uses the OpenAI tool-calling path instead of the Mistral SDK path.
if not _model_name.startswith("openai/"):
_model_name = "openai/" + _model_name.split("/", 1)[-1]

A shared model's credentials are overwritten, not merged. When an agent points at the instance's default model, the key and api_base come from the server environment and clobber whatever the agent row carries — an agent cannot redirect the shared model's traffic to an endpoint of its choosing. Credentials stored per agent are encrypted at rest, and the server refuses to boot without a Fernet key rather than silently falling back to plaintext.



Five shapes of agent, one column again


agent_type selects the ADK primitive:

agent_type
What it builds
Use

base
LlmAgent
one agent, its tools

router

LlmAgent + generated routing instruction
dispatch to sub-agents

sequential
SequentialAgent
pipeline: each step reads the previous one's result under the name its output_key gave it

parallel
ParallelAgent
fan out, then join

loop
LoopAgent
iterate until a condition or a bound

The loop is bounded twice: a per-agent loop_max_iterations (default 3) and a HARD_LIMIT = 100 it cannot exceed. An unbounded agent loop is not a feature, it's a bill.

Composition is data too — sub_agents is a column, and a sub-agent is loaded by the same to_agent, so hierarchies are built without a line of user code.

The tool catalogue in a stock instance: 108 tools across 32 categories, plus 8 skills and MCP servers.



Tools: 31 modules, 108 tools, plus MCP, plus skills


The tool store ships 31 modules in src/apowerb/tools_store/portfolio/: Google Workspace (Drive, Gmail, Calendar, Sheets, Docs), Microsoft 365 (Outlook, OneDrive, Teams), generic SQL, text-to-SQL, RAG, S3, HubSpot, charting, web search, Odoo, and so on. A module is a family, not a single function — a clean install of this stack reports 108 available tools in 32 categories, plus 8 reusable skills, before you connect anything. An agent's agent_tools column lists what it may use; OAuth tokens for the integrations are encrypted in the integrations table.

Beyond that, mcp_servers on the agent row instantiates ADK McpToolsets at load time, so any MCP server you run becomes tools for that agent without touching apowerb, and agent_skills attaches reusable prompt+tool bundles.

RAG is a first-class route family rather than a tool you wire yourself: POST /api/rag/index-files, index-url, index-db, index-db-nl, index-s3, with progress on an SSE stream. The URL indexer has SSRF protection (localhost and private ranges refused), which is the kind of thing you only add after someone points a crawler at 169.254.169.254.

Integrations are OAuth apps you declare; the agent gets the credentials, not your code.



The door problem


Here is our favourite bug in this codebase, because the tests were green the whole time.

A run can start from several places: the chat endpoints (/api/adk/run, /run_sse), scheduled runs driven by an external orchestrator, and inbound webhooks (Gmail via Pub/Sub, Outlook via Graph). Each entry point resolved its own guards. Two of them did. The other two had been written later, somewhere else, by someone who didn't know the first two existed.

The fix is a single choke point, src/apowerb/core/run_gate.py, and then a test that makes the choke point stay one — by reading the sources instead of exercising a path:

# tests/test_run_gate_couverture.py — the two names the test looks for
RUNNERS = {"run_adk_agent", "stream_adk_agent"}   # calling either one is "running an agent"
PORTIER = "apply_run_guards"                      # ...and this must be called too

Its docstring, in French like the rest of this codebase's reasoning, says the point better than we can: it re-reads the sources rather than exercising a path, because the door somebody adds in six months is one no behavioural test will have been written for.

The test walks the AST of every module under src/apowerb, collects the names each one calls, and fails when a module that calls a runner never names the gate. Be clear about what that proves: it matches at module scope, so it does not check that the gate runs before the runner, or on every branch — a call sitting in another function of the same file satisfies it. It is a coarse net, and it is aimed at the failure that actually happened: a new door written somewhere else, by somebody who never heard of the gate. That one fails CI on the day it's written rather than the day it's abused. It complements the behavioural tests rather than replacing them: a regression signal for the forgotten door, not proof that every execution path is guarded.



The token cap


run_gate doesn't hard-code its guards: it asks an extension registry for them (src/apowerb/core/extensions/registry.py). The core registers its own guards there, and an extension package adds its own through the same seam — so guards an extension brings leave with it, while the ones the core registers stay whatever you uninstall. The main one is a token cap, wired in src/apowerb/core/usage_wiring.py: every model call is recorded into llm_usage, and a run can be refused before it starts.

What the cap covers is worth stating precisely, because it's a self-hosting decision. Only consumption of the shared default model is bounded — the rows flagged billed_to_thaink2 in llm_usage. Tokens you spend through your own API key are yours to pay for, so capping them would be someone else's policy, not a feature. The cap is a monthly token quota per account, counted over the calendar month in the billing timezone, and it is a setting: a quota of 0 means unlimited.

Be precise about what that guard is, though, because it is not a budget you cannot exceed. It runs before a run starts and never mid-flight — cutting an SSE stream in half would leave a truncated conversation and a half-billed turn, so an overage is a clean 402 refusal at the door rather than a cut-off. It is deliberately fail-open: if the agent can't be resolved or the usage read fails, the run goes through, on the view that losing a cap matters less than making the product mute. And it inspects the agent you called, not the tree below it. agent_uses_default_llm reads the called agent's own agent_model column and returns before the quota is ever read when that column is not the shared model — so a sequential or parallel agent whose sub-agents alone sit on the shared model takes that early return on every run, not merely the first. Its consumption is recorded all the same, because the recorder runs on each sub-agent, and that recorded usage can get a later quota-checked run on the same account refused, provided the quota read succeeds. The container itself is not stopped by this guard, then or later. Worth knowing before you plan a budget around it.



The pipe that was open and empty


One honest note on observability, straight from src/apowerb/configs/observability.py.

ADK exports its GenAI spans over OTLP as soon as OTEL_EXPORTER_OTLP_ENDPOINT is set — model calls, tool calls, durations. Standard Python logging records are not part of that, and nobody noticed, because the collector was plainly reachable:

Measured on 4 September 2026 (translated from the French docstring) with the full stack running and that variable set: th2pulse

held {"count": 0} on /logs after six minutes and several served requests, while a

synthetic OTLP record pushed to the same collector arrived fine. The pipe was open;

nothing was writing to it.

A reachable endpoint is not a working pipeline. The bridge is now explicit, it's an optional extra (apowerb[otel], backed by th2pulse), it's opt-in on the endpoint variable, and every path in it swallows its exception and logs it — observability that takes the application down is worse than no observability.

The dashboard of the instance these screenshots come from — public images, one compose file.



Run it


git clone https://github.com/apowerb/apowerb-hosting.git && cd apowerb-hosting
cp .env.example .env && ./scripts/generate-secrets.sh
docker compose -f docker-compose/docker-compose.yml --env-file .env up -d

UI on http://localhost:3000, API on 8000, Postgres inside the stack, secrets generated for you. Nothing to fill in — except a model key, which is the one thing it can't invent: add yours in the interface, or set DEFAULT_LLM_MODEL and DEFAULT_LLM_API_KEY in .env. Until then everything else works and the model simply doesn't appear in the list.

The same repo carries Kubernetes manifests, a Helm chart and a Traefik overlay. Images are on Docker Hub under the apowerb namespace.



Links


• Core: github.com/apowerb/apowerb — Apache-2.0

• Deployment: apowerb/apowerb-hosting

• Web UI: apowerb/apowerb-ui

• ETL / orchestration: apowerb/th2etl

• Observability: apowerb/th2pulse

• Docs: docs.apowerb.com — quickstart, API reference

Issues and PRs welcome. If you self-host it and something in this post doesn't match what you see, that's a bug in one of the two — tell us which.


Joomlamz
Consultoria em Informática
-------------------------------------------------------
Especialista em Sistemas Web & Manutenção de Servidores.
A desenvolver o novo AplPortal com suporte a PHP 8.
Precisa de ajuda profissional? Contacte-me.

Tags: