Troubleshooting

This section provides comprehensive troubleshooting guidance for common issues encountered with the Suggested Products feature, including diagnostic steps, solutions, and preventive measures.

Quick Diagnostic Checklist

Before diving into specific issues, run through this quick checklist:

Check system requirements and dependencies
Verify configuration settings
Review recent error logs
Check database connectivity
Validate data integrity
Confirm sufficient system resources

Common Issues and Solutions

1. Generation Process Issues

Issue: Command Fails to Start

Symptoms:

php artisan generate:suggested-products
# Returns: Command not found or fails immediately

Diagnostic Steps:

# Check if command is registered
php artisan list | grep suggested

# Verify autoloader
composer dump-autoload

# Check for syntax errors
php artisan config:cache
php artisan route:cache

Solutions:

Register Command Properly:

// In app/Console/Kernel.php
protected $commands = [
    \App\Console\Commands\GenerateSuggestedProductsCommand::class,
];

Clear Application Cache:

php artisan cache:clear
php artisan config:clear
php artisan route:clear
php artisan view:clear

Verify File Permissions:

chmod -R 755 app/Console/Commands/
chown -R www-data:www-data storage/

Issue: Memory Limit Exceeded

Symptoms:

Fatal error: Allowed memory size of X bytes exhausted

Diagnostic Commands:

# Check current memory usage
free -h

# Monitor PHP memory limit
php -i | grep memory_limit

# Check process memory usage
ps aux --sort=-%mem | head

Solutions:

Increase PHP Memory Limit:

# In php.ini
memory_limit = 4G

# Or via environment
export PHP_MEMORY_LIMIT=4G

Optimize Processing:

// Reduce chunk size
'suggested_products_chunk_size' => 10,

// Process fewer customers at once
php artisan generate:suggested-products --customers=1,2,3

Memory-Efficient Processing:

// Force garbage collection
public function processCustomers($customers)
{
    foreach ($customers as $customer) {
        $this->processCustomer($customer);
        unset($customer); // Free memory
        gc_collect_cycles();
    }
}

Issue: Process Hangs or Times Out

Symptoms:

Command runs indefinitely
No progress output
Server becomes unresponsive

Diagnostic Steps:

# Check running processes
ps aux | grep artisan

# Monitor system load
htop
iostat 1

# Check database connections
mysql -e "SHOW PROCESSLIST;"

Solutions:

Set Execution Timeout:

// In configuration
'suggested_products_execution_timeout' => 1800, // 30 minutes

// Or in command
ini_set('max_execution_time', 1800);

Optimize Database Queries:

-- Add missing indexes
CREATE INDEX idx_orders_customer_date ON orders (customer_id, created_at);
CREATE INDEX idx_order_details_product ON order_details (product_id);

-- Analyze slow queries
SET GLOBAL slow_query_log = 'ON';
SET GLOBAL long_query_time = 1;

Process in Background:

# Use nohup for long-running processes
nohup php artisan generate:suggested-products > suggestions.log 2>&1 &

# Or use screen/tmux
screen -S suggestions
php artisan generate:suggested-products

2. Data Quality Issues

Issue: No Suggestions Generated

Symptoms:

SELECT COUNT(*) FROM suggested_products; -- Returns 0

Diagnostic Steps:

-- Check order data
SELECT COUNT(*) FROM orders WHERE created_at >= DATE_SUB(NOW(), INTERVAL 12 MONTH);

-- Check order details
SELECT COUNT(*) FROM order_details;

-- Check customers with multiple orders
SELECT customer_id, COUNT(*) as order_count 
FROM orders 
GROUP BY customer_id 
HAVING order_count >= 3;

Common Causes and Solutions:

Insufficient Historical Data:

-- Check data coverage
SELECT 
    MIN(created_at) as earliest_order,
    MAX(created_at) as latest_order,
    COUNT(DISTINCT customer_id) as unique_customers,
    COUNT(*) as total_orders
FROM orders;

Solution: Ensure at least 3-6 months of data with multiple orders per customer.

High Algorithm Thresholds:

// Reduce thresholds temporarily
'suggested_products_apriori_support' => 0.05,    // From 0.1
'suggested_products_confidence_support' => 0.3,   // From 0.5

Single-Item Orders:

-- Check for multi-item orders
SELECT 
    order_id,
    COUNT(*) as item_count
FROM order_details 
GROUP BY order_id 
HAVING item_count >= 2;

Solution: Ensure customers have orders with multiple items for association rules.

Issue: Poor Quality Suggestions

Symptoms:

Irrelevant product combinations
Low conversion rates
Customer complaints

Quality Assessment Queries:

-- Check suggestion diversity
SELECT 
    product_id,
    COUNT(*) as suggestion_count
FROM frequent_product_sets 
GROUP BY product_id 
ORDER BY suggestion_count DESC;

-- Analyze confidence scores
SELECT 
    AVG(confidence_score) as avg_confidence,
    MIN(confidence_score) as min_confidence,
    MAX(confidence_score) as max_confidence
FROM suggestion_analytics;

Solutions:

Increase Quality Thresholds:

'suggested_products_confidence_support' => 0.7,  // Higher confidence
'suggested_products_min_order_count' => 5,       // More order history

Filter Data Quality:

// Exclude problematic orders
$this->excludeOrders([
    'status' => ['cancelled', 'refunded'],
    'total_value' => ['<', 10], // Remove very small orders
    'item_count' => ['<', 2],   // Remove single-item orders
]);

Category-Based Filtering:

-- Remove unrelated product categories
DELETE FROM frequent_product_sets 
WHERE product_id IN (
    SELECT id FROM products 
    WHERE category_id IN (99, 100) -- Internal/test categories
);

3. Performance Issues

Issue: Slow API Response

Symptoms:

GET /suggested-products-from-recent-orders?shop_id=123
Response time: 5+ seconds

Performance Diagnostics:

-- Check query performance
EXPLAIN SELECT * FROM suggested_products 
WHERE customer_id = 123 
ORDER BY created_at DESC;

-- Monitor database performance
SHOW PROCESSLIST;
SHOW ENGINE INNODB STATUS;

Solutions:

Add Database Indexes:

-- Critical indexes for performance
CREATE INDEX idx_suggested_products_customer_created 
ON suggested_products (customer_id, created_at DESC);

CREATE INDEX idx_frequent_product_sets_product_set_priority 
ON frequent_product_sets (product_set_id, priority ASC);

CREATE INDEX idx_orders_customer_created 
ON orders (customer_id, created_at DESC);

Implement Caching:

// Cache suggestions per customer
$suggestions = Cache::remember(
    "customer_suggestions_{$customerId}",
    config('eva.suggestions_cache_ttl', 3600),
    function () use ($customerId) {
        return $this->generateSuggestions($customerId);
    }
);

Optimize Query Strategy:

// Use query optimization
public function getSuggestions($customerId, $limit = 20)
{
    return DB::table('suggested_products as sp')
        ->join('frequent_product_sets as fps', 'sp.product_set_id', '=', 'fps.product_set_id')
        ->join('products as p', 'fps.product_id', '=', 'p.id')
        ->where('sp.customer_id', $customerId)
        ->where('fps.status', 'active')
        ->select([
            'fps.product_id',
            'p.name as product_name',
            'fps.priority',
            'sp.apriori_timestamp'
        ])
        ->orderBy('fps.priority')
        ->limit($limit)
        ->get();
}

Issue: High Database Load

Symptoms:

Slow web interface
Database connection timeouts
High CPU usage on database server

Monitoring Commands:

-- Check long-running queries
SELECT * FROM information_schema.processlist 
WHERE TIME > 30 AND COMMAND != 'Sleep';

-- Monitor table locks
SHOW OPEN TABLES WHERE In_use > 0;

-- Check database size
SELECT 
    table_name,
    round(((data_length + index_length) / 1024 / 1024), 2) as 'Size (MB)'
FROM information_schema.tables 
WHERE table_schema = 'your_database'
ORDER BY (data_length + index_length) DESC;

Solutions:

Database Optimization:

-- Optimize tables
OPTIMIZE TABLE suggested_products, frequent_product_sets;

-- Update table statistics
ANALYZE TABLE suggested_products, frequent_product_sets;

-- Check for fragmentation
SELECT table_name, 
       data_free/1024/1024 as fragmentation_mb
FROM information_schema.tables 
WHERE data_free > 0;

Query Optimization:

// Use efficient pagination
public function paginatedSuggestions($page = 1, $perPage = 50)
{
    $offset = ($page - 1) * $perPage;
    
    return DB::table('suggested_products')
        ->select(['id', 'customer_id', 'product_set_id', 'created_at'])
        ->orderBy('id') // Use indexed column
        ->offset($offset)
        ->limit($perPage)
        ->get();
}

Connection Pool Management:

// In database configuration
'mysql' => [
    'options' => [
        PDO::ATTR_PERSISTENT => true,
        PDO::ATTR_TIMEOUT => 30,
    ],
    'pool' => [
        'min_connections' => 1,
        'max_connections' => 10,
    ],
],

4. Configuration Issues

Issue: Algorithm Not Working

Symptoms:

Empty results despite data
Unexpected behavior
Configuration errors

Configuration Validation:

// Validate configuration
class ConfigValidator
{
    public function validateSuggestedProducts()
    {
        $config = config('eva');
        $errors = [];
        
        // Check required settings
        if (!isset($config['suggested_products_apriori_support'])) {
            $errors[] = 'Missing apriori support setting';
        }
        
        // Validate ranges
        if ($config['suggested_products_apriori_support'] <= 0 || 
            $config['suggested_products_apriori_support'] > 1) {
            $errors[] = 'Invalid support value (must be 0-1)';
        }
        
        return $errors;
    }
}

Common Configuration Fixes:

Environment Variable Issues:

# Check environment loading
php artisan tinker
>>> config('eva.suggested_products_apriori_support')
>>> env('SUGGESTED_PRODUCTS_APRIORI_SUPPORT')

Cache Configuration Problems:

# Clear configuration cache
php artisan config:clear
php artisan config:cache

Missing Dependencies:

# Check for required packages
composer show | grep phpml
composer install --no-dev

5. API Integration Issues

Issue: Authentication Failures

Symptoms:

{
  "error": "Unauthorized",
  "message": "Invalid token"
}

Diagnostic Steps:

# Test API manually
curl -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     "http://your-domain/api/suggested-products-from-recent-orders?shop_id=123"

Solutions:

Verify Token Format:

// Check token in middleware
public function handle($request, Closure $next)
{
    $token = $request->bearerToken();
    if (!$token || !$this->validateToken($token)) {
        return response()->json(['error' => 'Invalid token'], 401);
    }
    return $next($request);
}

Check Route Registration:

// In routes/api.php
Route::middleware('auth:api')->group(function () {
    Route::get('/suggested-products-from-recent-orders', 
        [SuggestedProductsFromRecentOrdersController::class, 'index']);
});

Issue: Rate Limiting

Symptoms:

{
  "error": "Too Many Requests",
  "retry_after": 60
}

Solutions:

Adjust Rate Limits:

// In RouteServiceProvider
RateLimiter::for('suggestions', function (Request $request) {
    return Limit::perMinute(60)->by($request->user()?->id ?: $request->ip());
});

Implement Retry Logic:

async function fetchWithRetry(url, options, maxRetries = 3) {
    for (let i = 0; i < maxRetries; i++) {
        try {
            const response = await fetch(url, options);
            
            if (response.status === 429) {
                const retryAfter = response.headers.get('Retry-After') || 60;
                await new Promise(resolve => setTimeout(resolve, retryAfter * 1000));
                continue;
            }
            
            return response;
        } catch (error) {
            if (i === maxRetries - 1) throw error;
        }
    }
}

Error Log Analysis

Common Error Patterns

Memory Errors

[2024-10-15 10:30:00] local.ERROR: Allowed memory size exhausted
Context: Processing customer batch 5/10

Analysis: Increase memory limit or reduce batch size.

Database Errors

[2024-10-15 10:30:00] local.ERROR: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away
Context: Long-running query on orders table

Analysis: Database connection timeout, optimize queries or increase timeout.

Algorithm Errors

[2024-10-15 10:30:00] local.ERROR: Division by zero in Apriori calculation
Context: Customer 123 has no valid transactions

Analysis: Data validation needed before algorithm application.

Log Monitoring Commands

# Monitor real-time logs
tail -f storage/logs/laravel.log | grep -i "suggested\|apriori"

# Count error types
grep -c "ERROR" storage/logs/laravel.log

# Find memory errors
grep "memory" storage/logs/laravel.log | tail -10

# Analyze performance issues
grep "Processing time" storage/logs/laravel.log | awk '{print $NF}' | sort -n

Preventive Measures

1. Regular Health Checks

# Create health check script
#!/bin/bash
# health_check.sh

echo "=== Suggested Products Health Check ==="

# Check database connectivity
mysql -e "SELECT 1" || echo "Database connection failed"

# Check table sizes
mysql -e "SELECT COUNT(*) as suggested_products FROM suggested_products"
mysql -e "SELECT COUNT(*) as frequent_sets FROM frequent_product_sets"

# Check disk space
df -h | grep -E '(root|var)' | awk '$5 ~ /[89][0-9]%|100%/ {print "Low disk space: " $0}'

# Check memory usage
free -h | awk 'NR==2{printf "Memory usage: %s/%s (%.2f%%)\n", $3,$2,$3*100/$2 }'

# Check for errors in logs
error_count=$(grep -c "ERROR" storage/logs/laravel.log)
echo "Recent errors: $error_count"

2. Automated Monitoring

// Monitoring job
class SuggestedProductsHealthCheck extends Job
{
    public function handle()
    {
        $metrics = [
            'total_suggestions' => SuggestedProduct::count(),
            'active_customers' => SuggestedProduct::distinct('customer_id')->count(),
            'avg_suggestions_per_customer' => $this->calculateAverage(),
            'last_generation' => SuggestedProduct::latest()->first()?->created_at,
        ];
        
        // Alert if metrics are outside normal ranges
        if ($metrics['total_suggestions'] < 1000) {
            $this->sendAlert('Low suggestion count: ' . $metrics['total_suggestions']);
        }
        
        if ($metrics['last_generation'] < now()->subDays(7)) {
            $this->sendAlert('Suggestions not generated in 7 days');
        }
    }
}

3. Data Quality Assurance

-- Daily data quality checks
-- Check for orphaned records
SELECT COUNT(*) as orphaned_frequent_sets
FROM frequent_product_sets fps
LEFT JOIN suggested_products sp ON fps.product_set_id = sp.product_set_id
WHERE sp.id IS NULL;

-- Check for missing products
SELECT COUNT(*) as missing_products
FROM frequent_product_sets fps
LEFT JOIN products p ON fps.product_id = p.id
WHERE p.id IS NULL;

-- Check for data consistency
SELECT 
    sp.customer_id,
    COUNT(DISTINCT fps.product_id) as unique_products,
    COUNT(fps.id) as total_suggestions
FROM suggested_products sp
JOIN frequent_product_sets fps ON sp.product_set_id = fps.product_set_id
GROUP BY sp.customer_id
HAVING unique_products != total_suggestions;

Emergency Recovery Procedures

Data Corruption Recovery

Backup Current State:

mysqldump database_name suggested_products frequent_product_sets > backup_$(date +%Y%m%d).sql

Clean Corrupted Data:

-- Remove orphaned records
DELETE fps FROM frequent_product_sets fps
LEFT JOIN suggested_products sp ON fps.product_set_id = sp.product_set_id
WHERE sp.id IS NULL;

Regenerate from Scratch:

php artisan generate:suggested-products --force --verbose

Performance Recovery

Kill Long-Running Processes:

-- Find and kill long-running queries
SELECT 
    ID, 
    TIME, 
    INFO 
FROM information_schema.processlist 
WHERE TIME > 300 AND COMMAND != 'Sleep';

-- Kill specific process
KILL QUERY 12345;

Emergency Cache Clear:

php artisan cache:clear
php artisan route:clear
php artisan config:clear
redis-cli FLUSHALL  # If using Redis

Restart Services:

sudo systemctl restart mysql
sudo systemctl restart php8.1-fpm
sudo systemctl restart nginx

This comprehensive troubleshooting guide provides the tools and knowledge needed to quickly identify, diagnose, and resolve issues with the Suggested Products feature, ensuring minimal downtime and optimal performance.

Troubleshooting

On this page