Skip to content

fix(ci): rewrite Claude Code workflow for working automated PR reviews#43

Merged
carlos-alm merged 1 commit intomainfrom
fix/claude-workflow-review
Feb 23, 2026
Merged

fix(ci): rewrite Claude Code workflow for working automated PR reviews#43
carlos-alm merged 1 commit intomainfrom
fix/claude-workflow-review

Conversation

@carlos-alm
Copy link
Contributor

Summary

  • Fixed: Claude Code review job was running (~$4/run) but never posting comments because allowed_tools is not a valid input on @v1 and the plugin config was broken
  • Switched from anthropics/claude-code-action@v1 to @beta which supports direct_prompt for automated PR reviews
  • Added phased review prompt (root cause analysis → config scrutiny → test integrity → code quality) with strict rating calibration
  • Added bot PR filtering to skip Dependabot/Renovate PRs
  • Downgraded permissions from write to read (beta action handles posting via OIDC)

What was broken

  1. allowed_tools input doesn't exist on the action — silently ignored, Claude's Bash calls were permission-denied
  2. Plugin marketplace config (code-review@claude-code-plugins) wasn't loading
  3. prompt on @v1 is meant for mention-triggered flows, not automated reviews

Test plan

  • Open a test PR and verify the automated-review job posts a review comment
  • Verify interactive-claude still responds to @claude mentions
  • Verify bot PRs (dependabot) are skipped

Switch from broken v1 plugin-based config to @beta with direct_prompt:
- Use direct_prompt instead of prompt + invalid allowed_tools input
- Use @beta action which supports automated review without @claude mention
- Use read permissions (action handles posting via OIDC token)
- Skip bot-generated PRs (dependabot, renovate)
- Add phased review prompt (root cause → config scrutiny → test integrity → code quality)
- Upgrade actions/checkout to v6
@carlos-alm carlos-alm merged commit 85d9ed9 into main Feb 23, 2026
12 of 13 checks passed
@carlos-alm carlos-alm deleted the fix/claude-workflow-review branch February 23, 2026 05:16
@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 23, 2026

Greptile Summary

This PR fixes the broken Claude Code automated review workflow by switching from @v1 (which silently ignored allowed_tools and had broken plugin config) to @beta with direct_prompt for automated reviews.

Major changes:

  • Split workflow into two jobs: automated-review (runs on all PRs) and interactive-claude (runs on @claude mentions)
  • Replaced plugin-based review (code-review@claude-code-plugins) with inline phased review prompt
  • Added bot filtering to skip Dependabot/Renovate PRs
  • Downgraded permissions from write to read (relying on OIDC for posting)
  • Upgraded checkout action from @v4 to @v6

Key improvements:

  • The phased review prompt provides structured guidance (root cause analysis → config scrutiny → test integrity → code quality)
  • Strict rating calibration (default 2-3 stars) to avoid auto-approval
  • Bot filtering prevents wasted CI runs on automated dependency PRs

Considerations:

  • The bot filtering logic uses github.event.pull_request.user.type != 'Bot' which should work for Dependabot but may need verification
  • Permission downgrade assumes @beta action handles posting via OIDC — needs testing to confirm comments appear
  • The inline prompt is quite long (~100 lines) but provides clear structure

Confidence Score: 4/5

  • This PR is safe to merge with low risk — fixes a real CI issue with well-motivated changes
  • Score reflects that this is a thoughtful fix to a broken workflow. The changes are well-documented, the approach is sound (switching from broken plugin config to direct_prompt), and the risk is minimal since this only affects CI automation, not production code. Main uncertainty is whether the permission downgrade will work as expected, but this can be verified in testing.
  • Verify in testing that the @beta action posts comments successfully with read-only permissions via OIDC

Important Files Changed

Filename Overview
.github/workflows/claude.yml Restructures workflow from broken v1 plugin-based review to beta action with direct_prompt; splits into automated-review and interactive-claude jobs; downgrades permissions from write to read

Last reviewed commit: 1ed4121

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 file reviewed, 2 comments

Edit Code Review Agent Settings | Greptile

(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
automated-review:
if: github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bot filtering logic may not work as expected — github.event.pull_request.user.type should check against the string 'Bot' (case-sensitive). GitHub sets this field to 'Bot' for apps like Dependabot, but verify this matches the actual API response structure.

Also note: this won't filter Renovate if it runs as a user account rather than a bot account. Consider checking github.actor against a list of known bot names if needed.

Comment on lines 19 to 23
permissions:
contents: read
pull-requests: write
issues: write
pull-requests: read
issues: read
id-token: write
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Downgrading permissions from write to read — verify that the @beta action can post PR review comments via OIDC without needing pull-requests: write. If the action posting fails, you'll need to restore pull-requests: write.

carlos-alm added a commit that referenced this pull request Mar 21, 2026
- GitNexus overall score corrected from 4.7 to 4.5 to match the
  arithmetic mean of its six sub-scores (5+5+4+4+4+5)/6 = 4.5
- Tier 2 renumbered starting at #38 (was duplicating #37 with Tier 1);
  also resolves the pre-existing duplicate #43 (Bikach/ChrisRoyse now
  #44/#45), with all subsequent entries incremented accordingly
- jelly section header updated from 417 to 423 stars to match the
  ranking table
carlos-alm added a commit that referenced this pull request Mar 23, 2026
…#559)

* docs: update competitive analysis for v3.2.0 and March 2026 landscape

Re-rank codegraph from #8 (4.0) to #5 (4.5) reflecting v3.2.0 features:
41 CLI commands, 32 MCP tools, dataflow across all 11 languages, CFG,
sequence diagrams, architecture boundaries, unified graph model.

Add new competitors: GitNexus (#1, 18k stars), DeusData/codebase-memory-mcp
(#6, 793 stars in 25 days). Update star counts and feature status across
all 85+ ranked projects. Mark 7 roadmap items as DONE. Flag stagnant
projects. Update joern.md (3,021 stars, 75 contributors, 4 community MCP
wrappers) and narsil-mcp.md (129 stars, SPA frontend, +36 security rules,
development paused since Feb 25).

* docs: fix narsil SPA version attribution in competitive analysis overview

Line 18 incorrectly stated "v1.6.1" as the version when the SPA feature
was introduced. The SPA frontend was added in v1.6.0; v1.6.1 is the
current release. Updated to "added v1.6.0, current v1.6.1" to match the
detailed narsil-mcp.md entry.

* docs: remove hardcoded star count from joern comparison table

The "32 stars, growing" value in the Community & maturity row hardcodes
a stale star count. Other comparison tables use "Growing" consistently
for codegraph's community status. Updated to match.

* fix: correct GitNexus score, Tier 2 rank numbering, and jelly star count

- GitNexus overall score corrected from 4.7 to 4.5 to match the
  arithmetic mean of its six sub-scores (5+5+4+4+4+5)/6 = 4.5
- Tier 2 renumbered starting at #38 (was duplicating #37 with Tier 1);
  also resolves the pre-existing duplicate #43 (Bikach/ChrisRoyse now
  #44/#45), with all subsequent entries incremented accordingly
- jelly section header updated from 417 to 423 stars to match the
  ranking table

* fix: correct aider rank and codegraph star count per review feedback

* fix: align scoring breakdown sub-scores with overall rankings for stagnant projects

glimpse: Community 4→2 (stagnant since Jan 2026), avg now 3.83≈3.8 matching ranking.
autodev-codebase: Community 3→1 (stagnant since Jan 2026), avg now 3.33, ranking updated 3.4→3.3.

* fix: align ranking scores with sub-score averages for colbymchenry and axon

* fix: correct ranking inversion at positions #23/#24 (#559)

autodev-codebase (3.3) was ranked #23 above Claude-code-memory (3.4)
at #24. Swapped to maintain descending score order.

* fix: correct score mismatches for code-graph-rag (4.5→4.2) and arbor (3.7→4.2) (#559)

* fix: sync breakdown table row order with ranking table for #23/#24 (#559)

* fix: correct ranking inversions and stale rank references (#559)

* fix: correct sub-score/overall-score mismatches for codexray, loregrep, MATE

* fix: correct score mismatches and aider header rank

* fix: update narsil-mcp Key Metrics to reflect development stagnation (#559)

* fix: add missing "vs arbor" comparison section (#559)

* fix: remove duplicate vs-glimpse section and correct role names in vs-arbor (#559)

The duplicate vs-glimpse block (stale rank #10) was left behind when
vs-arbor was inserted. Removed it — the correct version exists at #11.
Also fixed role vocabulary in vs-arbor: bridge → adapter, added entry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant