Job Fear Index

What it measures: AI's competitive standing when directly competing against human auditors in the same contests.

Formula: Percentile = (1 - place/total_participants) × 100

Weighting: AI-assisted (human-in-loop) tools receive 0.5× weight; backtesting results receive 0.2× weight (not real-time competition).

Data source: Human contest placements from bot_placements.csv where contestType = "human"

Interpretation:

• 0-30%: AI performs worse than most humans (safe for auditors)
• 30-50%: AI at median level - competes with average auditors
• 50-70%: AI beats majority of humans - elevated threat
• 70%+: AI outperforms most human auditors - high threat

Limitations: Based on contest results only; doesn't capture manual audit quality or client relationships.

What it measures: AI's market presence across different audit platforms and contest types in the recent 90-day window.

Formula: Penetration Score = min(100, (AI_entries/total_contests × 10) + (platforms_entered × 5))

Data sources:

• bot_placements.csv - AI participation records
• real_contests.csv - Total contest counts per platform

Key metrics:

• 90d Market Penetration: % of contests in last 90 days with AI participation
• Platforms Entered: Number of unique platforms (Code4rena, Sherlock, etc.) where AI has competed
• Human Contest Earnings: Prize money won by AI in human contests (not bot races)

Interpretation: Higher penetration = AI is becoming ubiquitous in the audit contest market, reducing opportunities for human-only participation.

What it measures: The highest severity level of vulnerabilities that AI tools have successfully discovered.

Scoring:

• Critical found: 100 points (AI can find fund-draining bugs)
• High found: 70 points (AI can find serious vulnerabilities)
• Medium found: 40 points (AI limited to moderate issues)
• Low/Info only: 10 points (AI only finds minor issues)

Data source: verified_findings.csv - confirmed vulnerabilities found by AI tools in real audits/contests.

Why it matters: If AI can only find low-severity issues, human auditors remain essential for critical security work. Finding Critical/High bugs demonstrates AI can match human expertise on high-impact vulnerabilities.

Limitations: Severity classification varies by platform; some "High" findings may be less impactful than others.

What it measures: Three key factors that determine whether AI could realistically replace human auditors.

Formula: Replacement Score = (Speed × 0.2) + (Cost × 0.3) + (Quality × 0.5)

Component breakdown:

• Speed Advantage (20% weight): Fixed at 90%. AI analyzes code in minutes; humans take days/weeks. Speed alone doesn't replace humans but enables higher throughput.
• Cost Advantage (30% weight): (1 - AI_cost/Human_cost) × 100. Assumes ~$100 AI API costs vs ~$8,000 human auditor cost per contest (40 hrs × $200/hr).
• Quality Parity (50% weight): Uses the Performance Gap percentile. If AI performs at 50th percentile, it matches median human quality.

Interpretation:

• <40: AI is a tool, not a replacement
• 40-60: AI can handle routine work; humans needed for complex audits
• 60-80: AI competitive for most audits; human premium for critical work
• 80+: AI can replace most human auditor functions

What it measures: A progressive scale of AI capability achievements, from basic participation to human-level expertise.

Formula: Score = (current_level / 7) × 100

Level definitions:

• Level 0: AI cannot participate in contests
• Level 1: Wins automated bot races (QA-level findings)
• Level 2: Enters and places in human contests
• Level 3: Top 10 in human contest
• Level 4: Top 5 in human contest (beats 95% of humans)
• Level 5: Podium finish (top 3)
• Level 6: Wins a human contest outright
• Level 7: Discovers novel 0-day vulnerability class

Detection: Automatically calculated from bot_placements.csv by checking placement positions in human vs bot-race contests.

Why it matters: Each level represents a qualitative leap in AI capability that historically required human expertise.

What it measures: Whether AI performance is improving, declining, or stable over time.

Calculation:

1. Sort all human contest placements by date
2. Split into two halves: historical (older) and recent (newer)
3. Calculate average percentile for each half
4. Compare: change = recent_avg - historical_avg

Trend classification:

• Rising (↑): Recent performance > Historical + 5% (AI improving)
• Falling (↓): Recent performance < Historical - 5% (AI regressing)
• Stable (→): Within ±5% (no significant change)

Why it matters: A rising trend suggests AI will continue to close the gap with humans. A falling trend may indicate AI hitting capability limits or humans adapting.

Limitations: Small sample sizes may produce noisy trends; doesn't account for contest difficulty variations.