Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 20 additions & 18 deletions TASKS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# TASKS.md — EGOS Inteligência (SSOT)

> **Updated:** 2026-03-18 | **Patterns:** 10 | **Nodes:** 77.0M | **Rels:** 25.1M | **Tools:** 27 | **Tasks:** 103/141 ✅ | **GitHub Issues:** https://github.com/enioxt/EGOS-Inteligencia/issues
> **Updated:** 2026-03-26 | **Patterns:** 10 | **Nodes:** 77.0M | **Rels:** 25.1M | **Tools:** 27 | **Tasks:** 105/141 ✅ | **GitHub Issues:** https://github.com/enioxt/EGOS-Inteligencia/issues
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

TASKS.md exceeds SSOT size limit and should be split/archived.

The file is now far beyond the allowed backlog size, which makes SSOT maintenance harder and conflicts with the repo rule.

As per coding guidelines "Maintain SSOT registry files: ... TASKS.md (backlog, max 500 lines)".

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@TASKS.md` at line 3, TASKS.md has grown past the SSOT backlog limit (max 500
lines); split it by moving older/closed task entries out of TASKS.md into one or
more archive files (e.g., TASKS_ARCHIVE_YYYY-MM.md) and leave only the
active/backlog items so TASKS.md is <=500 lines, update any top-of-file summary
line and internal links in TASKS.md to point to the new archive files, and
commit with a clear message noting the archive split; search for TASKS.md and
the top summary line (the "**Updated:**" header) to locate where to perform the
truncate-and-archive changes.


---

Expand Down Expand Up @@ -76,13 +76,13 @@
- [ ] Paginação para large result sets
> **Inspiração:** Palantir Gotham

### TASK-007: Investigation Upload + Sharing
- [ ] API: endpoint para upload de investigações (HTML, PDF, JSON)
- [ ] API: listar investigações compartilhadas publicamente
- [ ] Frontend: UI de upload no menu do usuário
- [ ] Frontend: galeria de investigações compartilhadas
- [ ] Frontend: "continuar a partir de" outra investigação (fork)
> **Referência:** Intelink `components/shared/ShareJourneyDialog.tsx`
### TASK-007: Investigation Upload + Sharing ✅ (26/03/2026)
- [x] API: import de bundles JSON exportados + share bundle com anotações e tags
- [x] API: listar investigações compartilhadas publicamente + endpoint de fork por token
- [x] Frontend: upload JSON no painel de pesquisas
- [x] Frontend: galeria pública `/shared` + página compartilhada com findings
- [x] Frontend: "continuar a partir de" outra investigação (fork) para usuários autenticados
> **Arquivos:** `api/src/bracc/routers/investigation.py`, `api/src/bracc/services/investigation_service.py`, `frontend/src/components/investigation/InvestigationPanel.tsx`, `frontend/src/pages/SharedInvestigation.tsx`, `frontend/src/pages/SharedInvestigations.tsx`

### TASK-008: Journey Tracker / Step Counter ✅ (02/03/2026)
- [x] Journey lib: localStorage, 500 entries, dedup, export JSON/MD, Web Share API
Expand Down Expand Up @@ -762,11 +762,13 @@
- [x] _call_openrouter aceita model + api_key params
> **Arquivos:** `chat.py`

### TASK-096: Bug Fixes — DDG Search + PNCP API ✅ (03/03/2026)
- [x] DDG fallback: 3 regex patterns for resilience + graceful empty fallback
- [x] PNCP: try 3 endpoint URLs (API changed), handle 400 gracefully, date normalization
### TASK-096: Bug Fixes — DDG Search + PNCP API ✅ (26/03/2026)
- [x] Brave: retry/backoff em `429/5xx` antes de cair para DDG
- [x] DDG: erro não fica silencioso; fallback retorna `note` explícita com degradação
- [x] PNCP: datas em `AAAAMMDD` + `codigoModalidadeContratacao` iterado por modalidade
- [x] Cobertura dedicada em `tests/unit/test_transparency_tools.py`
> **Issues:** #32 (P1), #33 (P2)
> **Arquivos:** `api/src/bracc/services/transparency_tools.py`
> **Arquivos:** `api/src/bracc/services/transparency_tools.py`, `api/tests/unit/test_transparency_tools.py`

### TASK-097: System Map — API/Routes/Pages Inventory ✅ (03/03/2026)
- [x] Documentar 55+ endpoints em 13 routers
Expand Down Expand Up @@ -1037,7 +1039,7 @@
- [x] `chat.py` reduzido de 1330 → 845 linhas (36% redução)
> **Arquivos:** `chat_models.py`, `chat_tools.py`, `chat_prompt.py`

### TASK-109: Testes Backend — Integration Tests
### TASK-109: Testes Backend — Integration Tests ✅ (26/03/2026)
- [x] Setup pytest + httpx AsyncClient fixtures (conftest.py with mock Neo4j)
- [x] 219 unit tests passing (session 17: fixed 3 stale assertions)
- [x] Test patterns endpoint (list, 503 disabled, 404 invalid, include_probable)
Expand All @@ -1049,11 +1051,11 @@
- [x] Test patterns against live VPS (list 10, invalid 404)
- [x] Test health/meta/activity/cache endpoints
- [x] 955 ETL unit tests passing (0 warnings after Pandas fix)
- [ ] Integration tests with testcontainers Neo4j
- [ ] Test chat tool calling + tier fallback + rate limit
> **Status (session 17-18):** 219 API unit + 18 live integration + 955 ETL = **1,192 tests**
> **Arquivos:** `api/tests/integration/test_live_api.py`, `api/tests/unit/`, `etl/tests/`
> **Esforço restante:** 2h | **Impacto:** Qualidade e confiança
- [x] Integration tests with testcontainers Neo4j (skip gracefully when Docker daemon is unavailable)
- [x] Test chat tool calling + tier fallback + rate limit
> **Status (session 17-18, 25-26/03/2026):** 228 API unit + 18 live integration + 955 ETL = **1,201 tests**. Chat coverage agora inclui tool calling, fallback de modelo, aviso de limite diário e `429` real do endpoint; patterns cobre três detectores concretos no endpoint específico; a suíte `integration` com `Neo4jContainer` não falha mais em máquinas sem daemon Docker.
> **Arquivos:** `api/tests/integration/test_live_api.py`, `api/tests/integration/conftest.py`, `api/tests/unit/test_chat.py`, `api/tests/unit/test_patterns.py`, `api/tests/unit/`, `etl/tests/`
> **Impacto:** Qualidade e confiança

### TASK-110: Neo4j Backup Script (Cron) ✅ (03/03/2026)
- [x] Hot tar backup do volume Docker (sem parar Neo4j) + count snapshot
Expand Down
1 change: 1 addition & 0 deletions api/src/bracc/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ class Settings(BaseSettings):
public_allow_person: bool = False
public_allow_entity_lookup: bool = False
public_allow_investigations: bool = False
investigation_import_max_bytes: int = 2 * 1024 * 1024
pattern_split_threshold_value: float = 80000.0
pattern_split_min_count: int = 3
pattern_share_threshold: float = 0.6
Expand Down
25 changes: 24 additions & 1 deletion api/src/bracc/models/investigation.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ class InvestigationResponse(BaseModel):
description: str | None = None
created_at: str
updated_at: str
entity_ids: list[str] = []
entity_ids: list[str] = Field(default_factory=list)
share_token: str | None = None


Expand Down Expand Up @@ -49,3 +49,26 @@ class Tag(BaseModel):
class TagCreate(BaseModel):
name: str = Field(max_length=50)
color: str = Field(default="#E07A2F", max_length=7)


class SharedInvestigationResponse(InvestigationResponse):
annotations: list[Annotation] = Field(default_factory=list)
tags: list[Tag] = Field(default_factory=list)


class InvestigationExportBundle(BaseModel):
investigation: InvestigationResponse
annotations: list[Annotation] = Field(default_factory=list)
tags: list[Tag] = Field(default_factory=list)


class InvestigationImportResponse(BaseModel):
investigation: InvestigationResponse
imported_entities: int
skipped_entity_ids: list[str] = Field(default_factory=list)
imported_annotations: int
imported_tags: int


class InvestigationForkRequest(BaseModel):
title: str | None = Field(default=None, max_length=200)
8 changes: 8 additions & 0 deletions api/src/bracc/queries/annotation_list_by_token.cypher
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
MATCH (i:Investigation {share_token: $token})-[:HAS_ANNOTATION]->(a:Annotation)
RETURN a.id AS id,
a.entity_id AS entity_id,
i.id AS investigation_id,
a.text AS text,
a.created_at AS created_at
ORDER BY a.created_at DESC
LIMIT 1000
3 changes: 3 additions & 0 deletions api/src/bracc/queries/investigation_shared_count.cypher
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
MATCH (i:Investigation)
WHERE i.share_token IS NOT NULL
RETURN count(i) AS total
18 changes: 18 additions & 0 deletions api/src/bracc/queries/investigation_shared_list.cypher
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
MATCH (i:Investigation)
WHERE i.share_token IS NOT NULL
WITH count(i) AS total
MATCH (i:Investigation)
WHERE i.share_token IS NOT NULL
ORDER BY i.updated_at DESC
SKIP $skip
LIMIT $limit
OPTIONAL MATCH (i)-[:INCLUDES]->(e)
WITH total, i, collect(coalesce(e.cpf, e.cnpj, e.contract_id, e.sanction_id, e.amendment_id, e.cnes_code, e.finance_id, e.embargo_id, e.school_id, e.convenio_id, e.stats_id, elementId(e))) AS eids
RETURN total,
i.id AS id,
i.title AS title,
i.description AS description,
i.created_at AS created_at,
i.updated_at AS updated_at,
i.share_token AS share_token,
[x IN eids WHERE x IS NOT NULL] AS entity_ids
7 changes: 7 additions & 0 deletions api/src/bracc/queries/tag_list_by_token.cypher
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
MATCH (i:Investigation {share_token: $token})-[:HAS_TAG]->(t:Tag)
RETURN t.id AS id,
i.id AS investigation_id,
t.name AS name,
t.color AS color
ORDER BY t.name
LIMIT 1000
100 changes: 87 additions & 13 deletions api/src/bracc/routers/investigation.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,26 @@
import json
from typing import Annotated, Literal

from fastapi import APIRouter, Depends, HTTPException, Query
from fastapi.responses import JSONResponse, Response
from fastapi import APIRouter, Depends, File, HTTPException, Query, UploadFile
from fastapi.responses import Response
from neo4j import AsyncSession
from pydantic import ValidationError

from bracc.config import settings
from bracc.constants import PEP_ROLES
from bracc.dependencies import CurrentUser, get_session
from bracc.middleware.cpf_masking import mask_formatted_cpf, mask_raw_cpf
from bracc.models.investigation import (
Annotation,
AnnotationCreate,
InvestigationCreate,
InvestigationExportBundle,
InvestigationForkRequest,
InvestigationImportResponse,
InvestigationListResponse,
InvestigationResponse,
InvestigationUpdate,
SharedInvestigationResponse,
Tag,
TagCreate,
)
Expand Down Expand Up @@ -58,6 +65,46 @@ async def list_investigations(
return InvestigationListResponse(investigations=investigations, total=total)


@router.post(
"/import",
response_model=InvestigationImportResponse,
status_code=201,
)
async def import_investigation(
file: Annotated[UploadFile, File(...)],
session: Annotated[AsyncSession, Depends(get_session)],
user: CurrentUser,
) -> InvestigationImportResponse:
filename = (file.filename or "").lower()
content_type = (file.content_type or "").lower()
if not filename.endswith(".json") and content_type not in {
"application/json",
"text/json",
}:
raise HTTPException(
status_code=415,
detail="Investigation import currently supports exported JSON bundles only",
)

raw = await file.read(settings.investigation_import_max_bytes + 1)
if len(raw) > settings.investigation_import_max_bytes:
raise HTTPException(
status_code=413,
detail="Investigation import file is too large",
)
try:
payload = json.loads(raw.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise HTTPException(status_code=400, detail="Invalid investigation JSON file") from exc

try:
bundle = InvestigationExportBundle.model_validate(payload)
except ValidationError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc

return await svc.import_investigation_bundle(session, bundle, user.id)


@router.get(
"/{investigation_id}",
response_model=InvestigationResponse,
Expand Down Expand Up @@ -239,36 +286,63 @@ async def generate_share_link(
return {"share_token": token}


@shared_router.get("/api/v1/shared/{token}", response_model=InvestigationResponse)
@shared_router.get("/api/v1/shared", response_model=InvestigationListResponse)
async def list_shared_investigations(
session: Annotated[AsyncSession, Depends(get_session)],
page: Annotated[int, Query(ge=1)] = 1,
size: Annotated[int, Query(ge=1, le=100)] = 20,
) -> InvestigationListResponse:
investigations, total = await svc.list_shared_investigations(session, page, size)
return InvestigationListResponse(investigations=investigations, total=total)


@shared_router.post(
"/api/v1/shared/{token}/fork",
response_model=InvestigationImportResponse,
status_code=201,
)
async def fork_shared_investigation(
token: str,
session: Annotated[AsyncSession, Depends(get_session)],
user: CurrentUser,
body: InvestigationForkRequest | None = None,
) -> InvestigationImportResponse:
title = body.title if body else None
result = await svc.fork_shared_investigation(session, token, user.id, title)
if result is None:
raise HTTPException(status_code=404, detail="Shared investigation not found")
return result


@shared_router.get("/api/v1/shared/{token}", response_model=SharedInvestigationResponse)
async def get_shared_investigation(
token: str,
session: Annotated[AsyncSession, Depends(get_session)],
) -> InvestigationResponse:
result = await svc.get_by_share_token(session, token)
) -> SharedInvestigationResponse:
result = await svc.get_shared_investigation(session, token)
if result is None:
raise HTTPException(status_code=404, detail="Shared investigation not found")
return result


@router.get("/{investigation_id}/export")
@router.get("/{investigation_id}/export", response_model=InvestigationExportBundle)
async def export_investigation(
investigation_id: str,
session: Annotated[AsyncSession, Depends(get_session)],
user: CurrentUser,
) -> JSONResponse:
) -> InvestigationExportBundle:
investigation = await svc.get_investigation(session, investigation_id, user.id)
if investigation is None:
raise HTTPException(status_code=404, detail="Investigation not found")

annotations = await svc.list_annotations(session, investigation_id, user.id)
tags = await svc.list_tags(session, investigation_id, user.id)

export_data = {
"investigation": investigation.model_dump(),
"annotations": [a.model_dump() for a in annotations],
"tags": [t.model_dump() for t in tags],
}
return JSONResponse(content=export_data)
return InvestigationExportBundle(
investigation=investigation,
annotations=annotations,
tags=tags,
)


@router.get("/{investigation_id}/export/pdf")
Expand Down
Loading
Loading