Skip to content

BENCHMARK.md

Latest commit

 

History

History
179 lines (142 loc) · 4.78 KB

BENCHMARK.md

File metadata and controls

179 lines (142 loc) · 4.78 KB
Raw

SILENTCHAIN AI - Benchmark Report

This document contains performance benchmarks and security scan results for SILENTCHAIN AI across different AI models, providers, and target applications.

Benchmark Results

Test 1: Ollama DeepSeek-V3.1 on ASP.NET TestInvicti
Product SILENTCHAIN AI Professional v1.1.0
AI Provider Ollama
Model deepseek-v3.1:671b-cloud
Target(s) aspnet.testinvicti.com

Performance

Metric Value
Total Scan Time 19.0 minutes
AI Requests 137
Avg Time/Request 8.32s
Total Tokens 138,742
Avg Tokens/Request 1,012

Findings

Severity Count
🔴 High 16
🟠 Medium 37
🟡 Low 63
🔵 Info 35
Total 151
Verified (Phase 2) 20

Scan Coverage

Metric Value
URLs Processed 78
URLs Analyzed 42
Skipped (Dup) 50
Errors 22

Test 2: Ollama gpt-oss:120b on Angular TestInvicti
Product SILENTCHAIN AI Professional v1.3.21
AI Provider Ollama
Model gpt-oss:120b-cloud
Target(s) angular.testinvicti.com

Performance

Metric Value
Total Scan Time 2h 6m
AI Requests 299
Avg Time/Request 25.33s
Total Tokens 407,227
Avg Tokens/Request 1,361

Findings

Severity Count
🔴 High 10
🟠 Medium 80
🟡 Low 170
🔵 Info 39
Total 299
Verified (Phase 2) 12

Scan Coverage

Metric Value
URLs Processed 98
URLs Analyzed 81
Skipped (Dup) 56
Skipped (Low Conf) 0
Errors 0

Phase 2 Telemetry

Metric Value
Phase 2 Attempts 15
Heuristic Matches 204
AI Verification Calls 204
AI Confirmed 4
AI Rejected 147
AI Response Parse Fails 0

Benchmark History

More benchmarks will be added here as additional models, providers, and target applications are tested.

Planned Tests

  • OpenAI GPT-4 on aspnet.testinvicti.com
  • Claude 3.5 Sonnet on aspnet.testinvicti.com
  • Google Gemini 1.5 Pro on aspnet.testinvicti.com
  • Ollama Llama 3.1 on various targets
  • Ollama Qwen 2.5 Coder on various targets
  • Comparative analysis across OWASP Juice Shop
  • Performance testing on large-scale applications

Methodology

Test Environment

  • Burp Suite: Professional Edition
  • SILENTCHAIN: Professional v1.1.0
  • Network: Standard broadband connection
  • Hardware: (To be documented per test)

Test Targets

  • aspnet.testinvicti.com: ASP.NET vulnerable application for security testing
  • More targets will be added in future benchmarks

Metrics Explained

Performance Metrics

  • Total Scan Time: Wall-clock time from first request to last finding
  • AI Requests: Number of API calls made to the AI provider
  • Avg Time/Request: Average response time per AI analysis
  • Total Tokens: Combined input + output tokens used
  • Avg Tokens/Request: Average token consumption per request

Finding Metrics

  • High Severity: Critical vulnerabilities requiring immediate attention
  • Medium Severity: Important security issues with moderate risk
  • Low Severity: Minor vulnerabilities or security weaknesses
  • Info: Informational findings and security notes
  • Verified (Phase 2): Findings confirmed through active exploitation (Professional only)

Coverage Metrics

  • URLs Processed: Total unique URLs encountered
  • URLs Analyzed: URLs that underwent AI analysis
  • Skipped (Dup): Duplicate URLs not re-analyzed
  • Errors: Requests that failed analysis

Contributing Benchmarks

If you'd like to contribute benchmark results:

  1. Run SILENTCHAIN on a public vulnerable application (DVWA, Juice Shop, WebGoat, etc.)
  2. Document your test environment:
    • SILENTCHAIN version
    • AI provider and model
    • Target application
    • Hardware specs (CPU, RAM)
    • Network conditions
  3. Export benchmark report from SILENTCHAIN
  4. Submit via GitHub Issue or Pull Request

Notes

  • All benchmarks are performed on publicly accessible test applications designed for security testing
  • Results may vary based on:
    • AI model version and capabilities
    • Network latency
    • Hardware resources
    • Target application complexity
    • Burp Suite configuration
  • For Ollama benchmarks: Local hardware significantly impacts performance
  • For Cloud AI benchmarks: Network latency and API rate limits affect results

Generated by SILENTCHAIN AI

Copyright © 2026 SN1PERSECURITY LLC. All rights reserved.