[runtime] Make schema chain test deterministic#41
Merged
Conversation
Why: ensure the end-to-end signing chain test follows cached-only validator warmup behavior in CI instead of hanging on cold schema startup. Contract impact: none
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
/verify?schema=1on cold start instead of following the cached-only warmup path.Description
runtime/tests/runtime-signing.test.mjsupdated thestartServer({ ... })env block for the failing chain test to setVERIFY_SCHEMA_CACHED_ONLY: "1",VERIFY_MAX_MS: "12000",SCHEMA_FETCH_TIMEOUT_MS: "3000", andSCHEMA_VALIDATE_BUDGET_MS: "3000"and removed the previous explicitVERIFY_SCHEMA_CACHED_ONLY: "0"setting.[chain] before clean request,[chain] after clean response,[chain] before summarize request,[chain] after summarize response,[chain] before classify request,[chain] after classify response,[chain] before verify request) intact./verify?schema=1and, if the response is202, logging the warmup, waiting1200ms, retrying once, and if the retry still does not return200throwing an error with full status and body text; otherwise continue with unchanged assertions includingschema_valid === true.runtime/tests/runtime-signing.test.mjsand does not modify any runtime logic or other tests.Testing
npm test -- --runInBand runtime/tests/runtime-signing.test.mjsand the test file passed locally (all subtests inruntime/tests/runtime-signing.test.mjssucceeded).git diff --checkto validate there are no diff-check issues.Exact changes: set cached-only verification environment variables in the failing chain test and add a single 202->retry handling path for
/verify?schema=1while preserving assertions and logging.Codex Task