Vadim Malanov 2806aaf441 fix(ingestion): materialize scanner generator before inline processing ...

The folder scanner commits each document's row inside a per-document
session_scope that only finalizes when the generator advances past its
yield. In inline mode scripts/ingest_folder.py invoked the pipeline
(which opens a fresh session) before that commit landed, so
process_document_id saw "document_missing" and the pipeline aborted at
STORED_ORIGINAL.

Draining the generator with list() forces every scanner commit before
any pipeline run. Verified end to end: a generated PDF now progresses
DISCOVERED -> ... -> CHUNKING_COMPLETED and is searchable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-14 13:41:53 +03:00

2.4 KiB

Raw Blame History

View Raw