PR Reviews & Summaries

Kit AI Code Reviewer & Summarizer

Kit includes a production-ready AI code reviewer and summarizer that provides professional-grade analysis with full repository context. Review GitHub PRs or local git diffs without creating a PR. Use almost any LLM and pay just for tokens. High-quality reviews with SOTA models like Claude Sonnet 4 generally cost about 10 cents, while summaries cost just pennies.

Use per-organization profiles and prioritization for further customization. Use kit’s local output to pipe to other unix tools.

Tip

Want to build a completely custom reviewer? See the Build an AI PR Reviewer tutorial to create your own using kit’s components.

🚀 Quick Start

# 1. Install kit (lightweight - no ML dependencies needed for code review!)
pip install cased-kit

# 2. Set up configuration
kit review --init-config

# 3. Set API keys
export KIT_GITHUB_TOKEN="ghp_your_token"  # For GitHub PRs
export KIT_ANTHROPIC_TOKEN="sk-ant-your_key"  # For Anthropic (default)
export KIT_OPENAI_TOKEN="sk-openai-your_key"  # For OpenAI
export KIT_GOOGLE_API_KEY="AIzaSy-your_google_key"  # For Google

# 4a. Review any GitHub PR
kit review https://github.com/owner/repo/pull/123

# 4b. Review local changes (no PR needed!)
kit review main..feature           # Compare branches
kit review HEAD~3..HEAD            # Review last 3 commits
kit review --staged                # Review staged changes

# 4b. For OpenAI users: specify provider
kit review --model gpt-4.1 https://github.com/owner/repo/pull/123
# Or edit ~/.kit/review-config.yaml to set: llm.provider: openai

# 5. Test without posting (dry run with full formatting)
kit review --dry-run https://github.com/owner/repo/pull/123

# 6. Use custom context profiles for organization standards
kit review --profile company-standards https://github.com/owner/repo/pull/123

# 7. Focus on specific priority levels
kit review --priority=high,medium https://github.com/owner/repo/pull/123

# 8. Quick PR summaries for triage (5-10x cheaper)
kit summarize https://github.com/owner/repo/pull/123

# 9. Add summary to PR description for team visibility
kit summarize --update-pr-body https://github.com/owner/repo/pull/123

Tip

GitHub Actions Users: When running in CI/CD, checkout the repository first and use kit review --repo-path . to avoid authentication issues. See our CI/CD guide for details.

Tip

Just want the PR reviewer? The base pip install cased-kit gives you everything needed for PR reviews without heavy ML dependencies like PyTorch. If you need semantic search features later, install with pip install cased-kit[ml].

💰 Transparent Pricing

Kit now uses real-time pricing data from Helicone’s API (300+ models), ensuring costs are always accurate. No more manual updates when providers change their rates!

Some examples based on real-world testing on production open source PRs:

Model	Typical Cost	Quality	Best For
gemini-1.5-flash-8b	$0.003	⭐⭐⭐	Ultra-budget, high volume
gpt-4.1-nano	$0.0015-0.004	⭐⭐⭐	High-volume, ultra-budget
gpt-4.1-mini	$0.005-0.015	⭐⭐⭐⭐	Budget-friendly, often very good for the price
gemini-2.5-flash	$0.007	⭐⭐⭐⭐	Excellent value, fast
claude-sonnet-4	0.08-$0.14	⭐⭐⭐⭐⭐	Recommended for most

PR Summaries (for triage): ~$0.005-0.02 per summary (5-10x cheaper than reviews)

In Practice

Even without optimizing your model mix, a team doing 500 large PRs a month will generally pay under $50 a month total for reviews with SOTA models.

Tip

Don’t underestimate the smaller models. gpt-4.1-mini delivers surprisingly useful reviews when given the right context via kit. For simple projects, you can get decent AI code reviews for less than $1/month. Here’s an example against kit itself. This review cost half a cent.

🎯 Key Features

Intelligent Analysis

• Repository Context: Full codebase understanding, not just diff analysis
• Symbol Analysis: Identifies when functions/classes are used elsewhere
• Cross-Impact Assessment: Understands how changes affect the broader system
• Multi-Language Support: Works with any language kit supports

Professional Output

• Priority-Based Issues: High/Medium/Low issue categorization with filtering options
• Specific Recommendations: Concrete code suggestions with examples
• GitHub Integration: Clickable links to all referenced files
• Quality Scoring: Objective metrics for review effectiveness

Cost & Transparency

• Real-Time Cost Tracking: See exact LLM usage and costs
• Token Breakdown: Understand what drives costs
• Model Information: Know which AI provided the analysis
• No Hidden Fees: Pay only for actual LLM usage

📄 PR Summaries for Quick Triage

For teams that need to quickly understand PRs before committing to full reviews, kit includes fast, cost-effective PR summarization:

# Generate a quick summary
kit summarize https://github.com/owner/repo/pull/123

# Add the summary directly to the PR description
kit summarize --update-pr-body https://github.com/owner/repo/pull/123

# Use budget models for ultra-low-cost summaries
kit summarize --model gpt-4.1-nano https://github.com/owner/repo/pull/123

Why Use PR Summaries?

• 5-10x cheaper than full reviews (~$0.005-0.02 vs $0.01-0.05+)
• Perfect for triage: Understand what a PR does before deciding on detailed review
• Team visibility: Add AI summaries directly to PR descriptions for everyone to see
• Same repository intelligence: Leverages symbol extraction and dependency analysis

Summary Output Format

Summaries provide structured information in a consistent format:

• What This PR Does: 2-3 sentence overview of the main purpose
• Key Changes: Most important modifications (max 5 bullet points)
• Impact: Areas of codebase affected and potential risks/benefits

PR Body Updates

The --update-pr-body option adds a marked AI summary section to the PR description:

<!-- AI SUMMARY START -->

## What This PR Does
[AI-generated overview]

## Key Changes
- [Key modifications]

## Impact
- [Impact analysis]

*Generated by kit v0.7.1 • Model: claude-sonnet-4-20250514*
<!-- AI SUMMARY END -->

Smart handling: Re-running with --update-pr-body replaces the existing summary instead of duplicating it.

📋 Custom Context Profiles

Store and apply organization-specific coding standards and review guidelines through custom context profiles. Create profiles that automatically inject your company’s coding standards, security requirements, and style guidelines into every PR review.

# Create a profile from your existing coding guidelines
kit review-profile create --name company-standards \
  --file coding-guidelines.md \
  --description "Acme Corp coding standards"

# Use in any review
kit review --profile company-standards https://github.com/owner/repo/pull/123

# List all profiles
kit review-profile list

→ Complete Profiles Guide - Profile management, team workflows, and examples

🏠 Local Diff Reviews (No PR Required!)

Review code changes locally without creating a GitHub PR. Perfect for pre-commit reviews, work-in-progress checks, or private repositories.

How it works: Kit analyzes the git diff in your current repository and provides an AI-powered review with the same intelligence as PR reviews.

# Review changes between branches
kit review main..feature        # Changes from main to feature branch
kit review develop..my-branch   # Changes from develop to my-branch

# Review recent commits
kit review HEAD~3..HEAD         # Last 3 commits
kit review HEAD~1               # Last commit (shorthand for HEAD~1..HEAD)
kit review abc123..def456       # Specific commit range

# Review staged changes before committing
kit review --staged             # Review what's staged for commit

# Combine with other options
kit review --plain main..feature | claude "Fix the high priority issues"
kit review --priority=high HEAD~1..HEAD
kit review --model gpt-4.1-mini --staged

# Review from a different directory
kit review HEAD~1 --repo-path /path/to/repo

Key Benefits:

• Privacy: Review sensitive code without pushing to GitHub
• Speed: Instant reviews without PR creation overhead
• Workflow: Integrate into pre-commit hooks or local development
• Cost: Same transparent pricing as PR reviews

Default Behavior:

• Output: Always goes to stdout (perfect for piping and redirection)
• Format: Standard mode includes headers, metadata, and formatting
• Saving: Reviews are NOT saved to disk by default

Output Formats:

• Standard (default): Full formatted output with headers, diff info, author, and cost tracking
• Plain (--plain): Clean output with just the review content - ideal for piping to other tools
• Dry Run (--dry-run): Preview mode without posting to GitHub (for PR reviews)

Examples:

# Default: outputs formatted review to stdout
kit review HEAD~1                    # See full formatted review in terminal
kit review HEAD~1 > review.md        # Save formatted review to file

# Plain mode: clean output for piping
kit review --plain HEAD~1 | claude "fix these issues"
kit review --plain HEAD~1 | pbcopy   # Copy review to clipboard (macOS)

Note: To automatically save reviews to .kit/reviews/ within your repository, enable save_reviews: true in your config file.

🔄 Output Modes & Integration

Kit provides different output modes for various workflows:

# Standard mode (default) - formatted output with headers and metadata
kit review main..feature
# Shows: header, diff info, author, review content, cost tracking

# Plain mode - clean output for piping to other tools
kit review --plain main..feature | claude "implement these suggestions"
# Shows: only the review content, perfect for piping

# Dry run mode - preview without posting (GitHub PRs only)
kit review --dry-run https://github.com/owner/repo/pull/123

# Combine modes with filtering
kit review --plain --priority=high HEAD~1..HEAD

Output Examples:

Standard Output (default)

🔍 Kit Local Diff Review

Repository: my-project
Diff: main..feature
Author: John Doe

---

## Code Review

### HIGH Priority
- Security issue in auth.py...

---
✨ Cost: $0.0234

Plain Output (—plain)

## Code Review

### HIGH Priority
- Security issue in auth.py...

→ Integration Guide - Output modes, piping workflows, and multi-stage AI analysis

🚀 CI/CD Integration

Add AI code reviews to your GitHub Actions workflow:

name: AI PR Review
on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  ai-review:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      contents: read

    steps:
      - name: AI Code Review
        run: |
          pip install cased-kit
          kit review ${{ github.event.pull_request.html_url }}
        env:
          KIT_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          KIT_ANTHROPIC_TOKEN: ${{ secrets.ANTHROPIC_API_KEY }}

→ CI/CD Guide - GitHub Actions, advanced workflows, and cost optimization strategies

🔧 Configuration

Quick configuration for common setups:

# Override model for specific review
kit review --model gpt-4.1-nano https://github.com/owner/repo/pull/123

# Free local AI with Ollama
kit review --model qwen2.5-coder:latest https://github.com/owner/repo/pull/123

→ Configuration Guide - Model selection, API keys, and configuration files

📊 Examples

See real-world reviews with actual costs and analysis:

• FastAPI Packaging Change ($0.034) - Architectural impact analysis
• React.dev UI Feature ($0.012) - Accessibility-focused review
• Documentation Fix ($0.006) - Proportional response

→ More Examples - Real review examples and use cases

📈 What’s Next: Roadmap

Recently Shipped ✅

• Custom Context Profiles: Store and apply organization-specific coding standards and guidelines
• Priority Filtering: Focus reviews on what matters most

In Development

• Feedback Learning: Simple database to learn from review feedback and improve over time
• Inline Comments: Post comments directly on specific lines instead of summary comments
• Follow-up Review Awareness: Take previous reviews into account for better, more targeted feedback

Future Features

• Multi-Model Consensus: Compare reviews from multiple models for high-stakes changes
• Smart Review Routing: Automatically select the best model based on change type and team preferences

💡 Best Practices

Cost Optimization

• Use free local AI for unlimited reviews with Ollama (requires self-hosted setup)
• Use budget models for routine changes, premium for breaking changes
• Use the --model flag to override models per PR
• Leverage caching - repeat reviews of same repo are 5-10x faster
• Set up profiles to avoid redundant context

Team Adoption

• Start with free local AI to build confidence without costs
• Use budget models initially to control costs
• Create organization-specific guidelines for consistent reviews
• Add to CI/CD for all PRs or just high-impact branches

---

The kit AI PR reviewer provides professional-grade code analysis at costs accessible to any team size, from $0.00/month with free local AI to enterprise-scale deployment. With full repository context and transparent pricing, it’s designed to enhance your development workflow without breaking the budget.