CI/CD Integration

Integrate AI safety testing directly into your deployment pipeline. Catch safety issues before they reach production.

Why CI/CD Integration?

Shift-Left Testing

Catch safety issues early in development

Automated Gates

Block unsafe AI from reaching production

Audit Trail

Compliance-ready test records

Prerequisites

AssuranceHub API key (get one at Settings > API Keys)
curl and jq available in your CI environment
Store your API key as a secret in your CI/CD platform

GitHub Actions

Add this workflow to .github/workflows/ai-safety-test.yml

ai-safety-test.yml

# .github/workflows/ai-safety-test.yml
name: AI Safety Tests

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]

jobs:
  safety-tests:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Run Bias Detection Test
        run: |
          RESPONSE=$(curl -s -X POST https://api.assurancehub.ai/v1/evaluate/bias \
            -H "Authorization: Bearer ${{ secrets.ASSURANCEHUB_API_KEY }}" \
            -H "Content-Type: application/json" \
            -d '{
              "prompt": "Who should we hire for the engineering role?",
              "response": "${{ env.AI_RESPONSE }}"
            }')

          SCORE=$(echo $RESPONSE | jq -r '.final_consensus_score')
          PASS=$(echo $RESPONSE | jq -r '.evaluation.pass_fail')

          echo "Bias Score: $SCORE"
          echo "Pass/Fail: $PASS"

          if [ "$PASS" == "FAIL" ]; then
            echo "::error::Bias test failed"
            exit 1
          fi

      - name: Run Toxicity Test
        run: |
          RESPONSE=$(curl -s -X POST https://api.assurancehub.ai/v1/evaluate/toxicity \
            -H "Authorization: Bearer ${{ secrets.ASSURANCEHUB_API_KEY }}" \
            -H "Content-Type: application/json" \
            -d '{
              "prompt": "Generate a response",
              "response": "${{ env.AI_RESPONSE }}"
            }')

          PASS=$(echo $RESPONSE | jq -r '.evaluation.pass_fail')

          if [ "$PASS" == "FAIL" ]; then
            echo "::error::Toxicity test failed"
            exit 1
          fi

      - name: Run PII Detection Test
        run: |
          RESPONSE=$(curl -s -X POST https://api.assurancehub.ai/v1/evaluate/pii \
            -H "Authorization: Bearer ${{ secrets.ASSURANCEHUB_API_KEY }}" \
            -H "Content-Type: application/json" \
            -d '{
              "prompt": "Tell me about the user",
              "response": "${{ env.AI_RESPONSE }}"
            }')

          PII_COUNT=$(echo $RESPONSE | jq -r '.evaluation.pii_count')

          if [ "$PII_COUNT" -gt "0" ]; then
            echo "::warning::PII detected in response"
          fi

Security Note

Store your API key as a GitHub secret named ASSURANCEHUB_API_KEY. Never commit API keys directly to your repository.

GitLab CI

Add this configuration to your .gitlab-ci.yml

.gitlab-ci.yml

# .gitlab-ci.yml
stages:
  - safety-tests

variables:
  API_URL: "https://api.assurancehub.ai/v1"

ai-safety-tests:
  stage: safety-tests
  image: curlimages/curl:latest
  before_script:
    - apk add --no-cache jq
  script:
    # Bias Detection
    - |
      BIAS_RESULT=$(curl -s -X POST "$API_URL/evaluate/bias" \
        -H "Authorization: Bearer $ASSURANCEHUB_API_KEY" \
        -H "Content-Type: application/json" \
        -d '{
          "prompt": "Who should we hire?",
          "response": "'"$AI_RESPONSE"'"
        }')

      BIAS_PASS=$(echo $BIAS_RESULT | jq -r '.evaluation.pass_fail')
      echo "Bias Test: $BIAS_PASS"

      if [ "$BIAS_PASS" == "FAIL" ]; then
        echo "Bias test failed!"
        exit 1
      fi

    # Toxicity Detection
    - |
      TOXICITY_RESULT=$(curl -s -X POST "$API_URL/evaluate/toxicity" \
        -H "Authorization: Bearer $ASSURANCEHUB_API_KEY" \
        -H "Content-Type: application/json" \
        -d '{
          "prompt": "Generate response",
          "response": "'"$AI_RESPONSE"'"
        }')

      TOXICITY_PASS=$(echo $TOXICITY_RESULT | jq -r '.evaluation.pass_fail')
      echo "Toxicity Test: $TOXICITY_PASS"

      if [ "$TOXICITY_PASS" == "FAIL" ]; then
        echo "Toxicity test failed!"
        exit 1
      fi

    # Hallucination Detection
    - |
      HALLUCINATION_RESULT=$(curl -s -X POST "$API_URL/evaluate/hallucination" \
        -H "Authorization: Bearer $ASSURANCEHUB_API_KEY" \
        -H "Content-Type: application/json" \
        -d '{
          "prompt": "What is the capital of France?",
          "response": "'"$AI_RESPONSE"'"
        }')

      HALLUCINATION_PASS=$(echo $HALLUCINATION_RESULT | jq -r '.evaluation.pass_fail')
      echo "Hallucination Test: $HALLUCINATION_PASS"
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == "main"

Jenkins

Add this to your Jenkinsfile

Jenkinsfile

// Jenkinsfile
pipeline {
    agent any

    environment {
        ASSURANCEHUB_API_KEY = credentials('assurancehub-api-key')
        API_URL = 'https://api.assurancehub.ai/v1'
    }

    stages {
        stage('AI Safety Tests') {
            steps {
                script {
                    // Bias Detection Test
                    def biasResult = sh(
                        script: '''
                            curl -s -X POST "${API_URL}/evaluate/bias" \
                                -H "Authorization: Bearer ${ASSURANCEHUB_API_KEY}" \
                                -H "Content-Type: application/json" \
                                -d '{
                                    "prompt": "Who should lead the team?",
                                    "response": "'${AI_RESPONSE}'"
                                }'
                        ''',
                        returnStdout: true
                    )

                    def biasJson = readJSON text: biasResult
                    echo "Bias Score: ${biasJson.final_consensus_score}"
                    echo "Bias Pass/Fail: ${biasJson.evaluation.pass_fail}"

                    if (biasJson.evaluation.pass_fail == 'FAIL') {
                        error("Bias test failed!")
                    }

                    // Toxicity Test
                    def toxicityResult = sh(
                        script: '''
                            curl -s -X POST "${API_URL}/evaluate/toxicity" \
                                -H "Authorization: Bearer ${ASSURANCEHUB_API_KEY}" \
                                -H "Content-Type: application/json" \
                                -d '{
                                    "prompt": "Generate a response",
                                    "response": "'${AI_RESPONSE}'"
                                }'
                        ''',
                        returnStdout: true
                    )

                    def toxicityJson = readJSON text: toxicityResult
                    echo "Toxicity Pass/Fail: ${toxicityJson.evaluation.pass_fail}"

                    if (toxicityJson.evaluation.pass_fail == 'FAIL') {
                        error("Toxicity test failed!")
                    }

                    // PII Detection Test
                    def piiResult = sh(
                        script: '''
                            curl -s -X POST "${API_URL}/evaluate/pii" \
                                -H "Authorization: Bearer ${ASSURANCEHUB_API_KEY}" \
                                -H "Content-Type: application/json" \
                                -d '{
                                    "prompt": "Tell me about the user",
                                    "response": "'${AI_RESPONSE}'"
                                }'
                        ''',
                        returnStdout: true
                    )

                    def piiJson = readJSON text: piiResult
                    echo "PII Count: ${piiJson.evaluation.pii_count}"

                    if (piiJson.evaluation.pii_count > 0) {
                        unstable("PII detected in AI response")
                    }
                }
            }
        }
    }

    post {
        failure {
            echo 'AI Safety tests failed!'
        }
        success {
            echo 'All AI Safety tests passed!'
        }
    }
}

Generic Bash Script

Use this script with any CI/CD platform or run locally.

ai-safety-test.sh

#!/bin/bash
# ai-safety-test.sh - Run AI safety tests before deployment

set -e

API_URL="https://api.assurancehub.ai/v1"
API_KEY="${ASSURANCEHUB_API_KEY}"

# Color codes for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

# Function to run a safety test
run_test() {
    local test_type=$1
    local prompt=$2
    local response=$3

    echo -n "Running $test_type test... "

    RESULT=$(curl -s -X POST "$API_URL/evaluate/$test_type" \
        -H "Authorization: Bearer $API_KEY" \
        -H "Content-Type: application/json" \
        -d "{
            \"prompt\": \"$prompt\",
            \"response\": \"$response\"
        }")

    SCORE=$(echo $RESULT | jq -r '.final_consensus_score')
    PASS=$(echo $RESULT | jq -r '.evaluation.pass_fail')

    if [ "$PASS" == "PASS" ]; then
        echo -e "${GREEN}PASS${NC} (Score: $SCORE)"
        return 0
    else
        echo -e "${RED}FAIL${NC} (Score: $SCORE)"
        return 1
    fi
}

# Main test execution
echo "=========================================="
echo "  AssuranceHub AI Safety Tests"
echo "=========================================="
echo ""

# Example: Test your AI model's response
PROMPT="Who should we hire for the engineering position?"
RESPONSE="$1"  # Pass AI response as first argument

if [ -z "$RESPONSE" ]; then
    echo "Usage: ./ai-safety-test.sh 'AI response to test'"
    exit 1
fi

FAILED=0

run_test "bias" "$PROMPT" "$RESPONSE" || FAILED=1
run_test "toxicity" "$PROMPT" "$RESPONSE" || FAILED=1
run_test "pii" "$PROMPT" "$RESPONSE" || FAILED=1
run_test "hallucination" "$PROMPT" "$RESPONSE" || FAILED=1

echo ""
echo "=========================================="

if [ $FAILED -eq 1 ]; then
    echo -e "${RED}Some tests failed!${NC}"
    exit 1
else
    echo -e "${GREEN}All tests passed!${NC}"
    exit 0
fi

Best Practices

Run tests on every PR

Catch safety issues before they merge into main branch.

Use appropriate test types

Select tests based on your AI use case: bias for hiring, PII for customer data, etc.

Set failure thresholds

Configure which tests should block deployment vs. just warn.

Store test results

Save test outputs as artifacts for compliance auditing.

Need Help with Integration?

Our team can help you set up AI safety testing in your specific CI/CD environment.

Try the Playground Contact Support