fix(indexing): unblock chunk indexing and gate reranker model load

Three defects surfaced during the first live ingest:

1. OpenSearch index mapping is dynamic:"strict", so the first chunk
   carrying metadata.section_heading was rejected with
   strict_dynamic_mapping_exception and zero chunks were indexed. The
   metadata and quality_flags sub-objects now opt into
   dynamic:true so chunker-specific keys insert without a fixed schema,
   while the top-level mapping stays strict.

2. hybrid_search.run_search called get_reranker() unconditionally, so
   every search - even search_mode=lexical with RERANKER_ENABLED=false
   - triggered a multi-GB FlagReranker download and hung the request.
   The loader is now lazy behind the reranker_enabled flag.

3. docker-compose x-common-env never forwarded RERANKER_ENABLED, so the
   container always defaulted to true regardless of .env. Added
   RERANKER_ENABLED: ${RERANKER_ENABLED:-true} to common-env.

After these fixes POST /search (lexical) returns HTTP 200 in ~50ms with
a ranked hit and citation; OpenSearch _count reports the indexed chunk.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Vadim Malanov
2026-06-14 13:42:10 +03:00
parent 2806aaf441
commit 01ba710c08
3 changed files with 19 additions and 10 deletions

View File

@@ -29,6 +29,7 @@ x-common-env: &common-env
EMBEDDING_DEVICE: ${EMBEDDING_DEVICE:-cpu}
RERANKER_MODEL: ${RERANKER_MODEL:-BAAI/bge-reranker-v2-m3}
RERANKER_DEVICE: ${RERANKER_DEVICE:-cpu}
RERANKER_ENABLED: ${RERANKER_ENABLED:-true}
APP_LOG_LEVEL: ${APP_LOG_LEVEL:-INFO}
APP_INPUT_DIR: /data/input
APP_WORK_DIR: /data/work
@@ -44,7 +45,7 @@ services:
POSTGRES_USER: ${POSTGRES_USER:-legacyhub}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-legacyhub}
ports:
- "5432:5432"
- "5440:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck: