ci: install pinned engine via ENGINES_TOKEN, run engine tests
Install teamhub-document-recognition-engine v0.1.0 in the backend CI job through the ENGINES_TOKEN PAT (git insteadOf rewrite, --no-deps plus pdfminer.six, pin matches app/ingestion/ENGINE_SOURCE.md). The step fails with a clear error when the secret is missing. test_chunker.py and the engine adapter tests now run for real in CI instead of skipping. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
24
.github/workflows/ci.yml
vendored
24
.github/workflows/ci.yml
vendored
@@ -33,6 +33,24 @@ jobs:
|
||||
"pytest>=8.2" "pytest-asyncio>=0.23" \
|
||||
ruff
|
||||
|
||||
- name: Install document recognition engine (pinned tag, needs ENGINES_TOKEN)
|
||||
# The engine repo is private; ENGINES_TOKEN is a PAT with read access
|
||||
# to GoodOneFather/TeamHUB_Engines. --no-deps keeps CI light: docling
|
||||
# and ocrmypdf are imported lazily inside engine functions, only
|
||||
# pdfminer.six is needed at import time. Pin must match
|
||||
# app/ingestion/ENGINE_SOURCE.md.
|
||||
env:
|
||||
ENGINES_TOKEN: ${{ secrets.ENGINES_TOKEN }}
|
||||
run: |
|
||||
if [ -z "$ENGINES_TOKEN" ]; then
|
||||
echo "::error::ENGINES_TOKEN secret is not available to Actions (repo Settings -> Secrets and variables -> Actions)"
|
||||
exit 1
|
||||
fi
|
||||
git config --global url."https://x-access-token:${ENGINES_TOKEN}@github.com/".insteadOf "https://github.com/"
|
||||
pip install "pdfminer.six>=20240706"
|
||||
pip install --no-deps \
|
||||
"teamhub-document-recognition-engine @ git+https://github.com/GoodOneFather/TeamHUB_Engines.git@teamhub-document-recognition-engine-v0.1.0#subdirectory=engines/teamhub-document-recognition-engine"
|
||||
|
||||
- name: Ruff
|
||||
run: ruff check app scripts tests
|
||||
|
||||
@@ -43,12 +61,12 @@ jobs:
|
||||
run: python scripts/report_module_sizes.py
|
||||
|
||||
- name: Pytest (unit only — heavy deps excluded)
|
||||
# test_chunker.py skips here: it needs the pinned document recognition
|
||||
# engine (private TeamHUB_Engines repo, no PAT secret configured).
|
||||
# It runs in the Docker image and in dev envs with the engine installed.
|
||||
# The pinned engine is installed above, so test_chunker.py and the
|
||||
# engine adapter tests run for real here; -rs surfaces any skips.
|
||||
run: |
|
||||
pytest tests/test_hashing.py tests/test_quality.py \
|
||||
tests/test_chunker.py tests/test_duplicates.py \
|
||||
tests/test_document_recognition_engine_adapter.py \
|
||||
tests/test_hybrid_search.py -q -rs
|
||||
|
||||
- name: docker compose config
|
||||
|
||||
Reference in New Issue
Block a user