Documentation/API Reference/Child Safety Detection
Back to Documentation

Child Safety Detection

Detect child endangerment and CSAM content

99.5%<2s
POST /v1/evaluate/child_safety

On This Page

Other APIs

Hallucination DetectionBias DetectionJailbreak PreventionToxicity DetectionPII DetectionGDPR ComplianceHIPAA ComplianceInjection DetectionRobustness Testing

Overview

The Child Safety Detection API identifies content that poses risks to minors, including CSAM, grooming attempts, child endangerment, and inappropriate interactions. This endpoint provides the highest accuracy (99.5%) in our safety test suite and is critical for platforms serving or discussing minors.

Critical Safety Notice

This API detects the most serious forms of harmful content. Content flagged as high risk should trigger immediate review and potential law enforcement notification in accordance with applicable laws.

Quick Start

Test with Safe Example

curl
curl -X POST "https://api.assurancehub.ai/v1/evaluate/child_safety" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "How can I help my child learn to read?",
    "response": "Read aloud together daily, point to words, ask questions about the story, and make reading fun."
  }'

Expected Response

Safe content will return a low consensus score (near 0.0) with risk_level "low" and pass_fail "pass".

Request Parameters

ParameterTypeRequiredDescription
promptstringRequired
The input prompt to analyze
Example: Tell me about children
responsestringRequired
The AI response to analyze for child safety issues
Example: Children should be protected and treated with care.

Code Examples

Basic Example

python
import requests

def detect_child_safety(prompt, response, api_key):
    url = "https://api.assurancehub.ai/v1/evaluate/child_safety"
    headers = {
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json"
    }
    data = {
        "prompt": prompt,
        "response": response
    }

    response = requests.post(url, json=data, headers=headers)
    return response.json()

# Example usage
result = detect_child_safety(
    prompt="Tell me about children",
    response="Children are wonderful and should be protected.",
    api_key="your_api_key"
)

print(f"Consensus Score: {result['final_consensus_score']}")
print(f"Risk Level: {result['evaluation']['risk_level']}")
print(f"Pass/Fail: {result['evaluation']['pass_fail']}")

Advanced Example

python
import requests
from typing import Dict

class ChildSafetyDetector:
    def __init__(self, api_key: str):
        self.api_key = api_key
        self.base_url = "https://api.assurancehub.ai"
        self.headers = {
            "Authorization": f"Bearer {api_key}",
            "Content-Type": "application/json"
        }

    def detect_child_safety(self, prompt: str, response: str) -> Dict:
        """Detect child safety issues in AI content"""
        data = {
            "prompt": prompt,
            "response": response
        }

        resp = requests.post(
            f"{self.base_url}/v1/evaluate/child_safety",
            json=data,
            headers=self.headers
        )

        return resp.json()

# Usage
detector = ChildSafetyDetector("your_api_key")

result = detector.detect_child_safety(
    prompt="Help with child activities",
    response="Here are safe, age-appropriate activities for children."
)

print(f"Consensus Score: {result['final_consensus_score']}")
print(f"Risk Level: {result['evaluation']['risk_level']}")
print(f"Flagged: {result['evaluation']['flagged']}")

Response Format

The API returns consensus-based child safety analysis with detailed model reasoning:

json
{
  "final_consensus_score": 0.0,
  "consensus_confidence": 0.95,
  "evaluation": {
    "risk_level": "low",
    "pass_fail": "pass",
    "flagged": false
  },
  "consensus_analysis": {
    "method": "simple",
    "models_succeeded": 2,
    "models_failed": 1,
    "agreement_score": 1.0,
    "calculation": [
      {
        "model": "gpt-4",
        "individual_score": 0.0,
        "confidence": 1.0,
        "role": "primary"
      },
      {
        "model": "deepseek-chat",
        "individual_score": 0.0,
        "confidence": 0.99,
        "role": "tertiary"
      }
    ]
  },
  "model_execution": {
    "execution_mode": "redundant_2",
    "total_latency_ms": 65697,
    "cost_usd": 0.001642,
    "pooling_used": true
  },
  "individual_model_details": [
    {
      "model": "gpt-4",
      "role": "primary",
      "status": "success",
      "score": 0.0,
      "confidence": 1.0,
      "latency_ms": 8972,
      "reasoning": "The response provides appropriate educational advice with no child safety concerns."
    }
  ],
  "risk_assessment": {
    "thresholds": {
      "acceptable": 0.2,
      "review_needed": 0.4,
      "reject": 0.532
    },
    "risk_factors": [],
    "model_agreement": "very_high",
    "consensus_quality": "good"
  },
  "metadata": {
    "test_type": "child_safety",
    "test_type_optimized": true,
    "evaluation_timestamp": "2025-10-16T19:57:13Z",
    "evaluator_version": "1.0.0-enterprise-child-safety"
  }
}

Response Fields

  • final_consensus_score - Risk score (0.0-1.0)
  • evaluation - Risk level and pass/fail status
  • consensus_analysis - Model agreement details
  • individual_model_details - Per-model analysis
  • risk_assessment - Thresholds and factors

Risk Thresholds

  • 0.0 - 0.2: Low risk (acceptable)
  • 0.2 - 0.4: Medium risk (review needed)
  • 0.4 - 0.532: High risk (immediate review)
  • 0.532 - 1.0: Critical risk (reject & report)