Vadim_Malanov/LegacyHUB
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
55 Commits 2 Branches 0 Tags
main
Commit Graph

5 Commits

Author SHA1 Message Date
Vadim Malanov
fe346d983b fix(pipeline): tolerate duplicate ORIGINAL_PDF artifacts in document lookup
Some checks failed
CI / Backend (lint + tests + compose) (push) Has been cancelled
Details
CI / Frontend (lint + type-check + build) (push) Has been cancelled
Details 
Repeated knowledge-ingest of identical content (same sha256, new
asset_id -> new canonical object key) reuses the Document row but
appends a second ORIGINAL_PDF artifact, because ensure_artifact identity
is (document_id, storage_key). process_document_id then crashed with
MultipleResultsFound on scalar_one_or_none and Celery retried forever.
New latest_artifact helper picks the newest row deterministically
(created_at DESC, id DESC) — every duplicate references byte-identical
objects (sha256 verified at ingest), so the latest reference is always
safe.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-07-11 20:14:52 +03:00
Vadim Malanov
d27dd0ffbb feat: align LegacyHUB with TeamHUB platform contract (D2/D4, assets, security) 
Close 12 audit-driven platform-compliance gaps on a single branch.

- D4 dispatch: app/integrations/dispatch_client.py participant `legacyhub`,
  emits LegacyhubDocumentIndexed + AssetDerivativeReady after the indexing
  commit (idempotent uuid5), http_inbox route (reindex/tombstone) with
  audit-based dedupe; docs/dispatch-contract.md. Celery+Redis stays intra-module.
- D2 SSO: app/integrations/identity.py validates X-TeamHub-* + role/scope
  mapper; security.py adds trusted-header enforcement (AUTH_REQUIRE_IDENTITY)
  and a scope check on /search; docker-compose.teamhub.yml (external teamhub_net
  + internal db net, api not host-published); RUNBOOK network/firewall section.
- Asset standard: SearchHit/Citation carry asset_id/owner_module; buckets
  renamed teamhub-legacyhub-* (+quarantine/tmp/exports); purge-by-asset_id with
  legal-hold guard (app/indexing/projection.py); OCR-markdown derivative event.
- audit_log model + Alembic 0003 + record_audit on writes (same transaction).
- Secret masking: app/common/json_logger.py recursive mask wired into structlog
  (+ensure_ascii=False); event payloads redacted before persistence.
- Service X-API-Key mandatory on ingest endpoints (defence-in-depth).
- Port: host API 8000->8050 (collision with SalesHUB/MailHUB resolved),
  container still listens on 8000.
- Config: no plaintext secret defaults; fail-loud in non-dev (no value leak).
- Docs drift: README PG 5440, layered-auth note, 5173 removed from CORS;
  ingest/folder gated by ENABLE_FOLDER_INGEST (410 by default).
- ADRs: layers mapping, shared-core extraction, UI locale (RU-first).

Tests: 78 passing (ruff, compileall, pytest, tsc, vite build, compose config).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-15 11:44:15 +03:00
Vadim Malanov
9926a4b3fd feat: add asset manifest knowledge ingest
Some checks failed
CI / Backend (lint + tests + compose) (push) Has been cancelled
Details
CI / Frontend (lint + type-check + build) (push) Has been cancelled
Details
2026-06-14 19:22:26 +03:00
Vadim Malanov
a375ca55b9 refactor: extract ensure_artifact into app/storage/artifacts.py 
The artifact-upsert helper was duplicated four times (scanner.py,
table_processor.py, figure_processor.py, pipeline.py) with slightly
different signatures. Consolidates into a single keyword-only function
keyed on (document_id, storage_key) - the identity the schema already
enforces - so re-running the pipeline never creates duplicate rows.

scanner / table_processor / figure_processor now import the shared
helper directly. pipeline.py keeps a thin local wrapper to preserve
the positional call sites at three artifact upsert points (OCR_PDF,
MARKDOWN, DOCLING_JSON).

Tests: 24 passed (5 health + 19 original).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 16:51:54 +03:00
Vadim Malanov
7f72171572 chore: bootstrap repository with governance docs 
Initialize git, add Apache-2.0 LICENSE, .gitattributes (LF line
endings), AGENTS.md (entry points, stack, discovery order, baseline
checks), RUNBOOK.md (dev boot, prod deploy with overlay, ingestion,
failures, rollback, scaling notes), .env.prod.example with rotated
credential placeholders, and dev-only warnings on .env.example.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 16:41:50 +03:00