API & Synchronization

This document covers the API endpoints, synchronization processes, and data flow between portal and client systems.

API Endpoints

Public Endpoints (Authenticated)

• GET /api/reports - Retrieve usage reports with filtering
• GET /api/aggregates - Get aggregated statistics
• GET /api/export - Export reports to Excel
• GET /api/get-last-executed - Get last sync timestamp
• GET /api/get-account-managers - Get account manager list

Client Endpoints (Client Authentication)

• POST /api/process-client-provided-sat-usage-report - Process client-provided data

Request/Response Examples

GET /api/reports

// Request Parameters
{
  "client_id": 123,
  "start_date": "2025-10-01",
  "end_date": "2025-10-31",
  "search": "client name",
  "per_page": 25
}

// Response
{
  "data": [
    {
      "id": 1,
      "client_name": "Example Client",
      "date": "2025-10-29",
      "total_licences": 100,
      "assigned_licences": 85,
      "total_users": 80,
      "sat_usage_percentage": 94.12,
      "portal_licence_no": 100,
      "downgraded_licences": 0
    }
  ],
  "meta": {
    "current_page": 1,
    "total": 1,
    "per_page": 25
  }
}

GET /api/aggregates

// Response
{
  "data": [
    {
      "total_licences": "1,250",
      "assigned_licences": "1,100",
      "active_users_in_backend": "150",
      "active_users_in_field": "950",
      "total_users": "1,100",
      "sat_usage": 95.45
    }
  ]
}

Synchronization Process

Complete Data Flow Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                    End-to-End Synchronization Process                        │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  Client Side                    Network                    Portal Side      │
│  ┌─────────────┐                                          ┌─────────────┐   │
│  │SAT Usage    │  ────────────────────────────────────►   │Portal Sync  │   │
│  │Radar        │  ◄────────────────────────────────────   │Service      │   │
│  │             │                                          │             │   │
│  │ 1. Collect  │                                          │ 4. Process  │   │
│  │ 2. Aggregate│                                          │ 5. Store    │   │
│  │ 3. Transmit │                                          │ 6. Confirm  │   │
│  └─────────────┘                                          └─────────────┘   │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

1. Client-Side Data Collection (SAT Usage Radar)

Hourly Data Aggregation Process

// Client Service Execution
$filters = (object) [
    'start_date' => Carbon::now()->subHour()->startOfHour()->format('Y-m-d H:i:s'),
    'end_date' => Carbon::now()->subHour()->endOfHour()->format('Y-m-d H:i:s')
];

$service = new GenerateHourlySatUsageReportsService();
$result = $service->handle($filters);

Data Collection Sources

1. Orders & Sales: Query both V1 and V2 systems for transaction data
2. Visit Tracking: Aggregate regular and off-route visit metrics
3. User Activity: Monitor active users across different roles
4. License Management: Track allocation and usage patterns
5. Delivery Operations: Count completed deliveries
6. System Metadata: Extract branch info and leave management data

Client-Side Transmission Process

1. Data Packaging: Structure data into standardized JSON format
2. Authentication: Generate secure tokens for portal communication
3. HTTP Transmission: Send data via authenticated POST request
4. Status Logging: Record transmission status (SENT/CONFIRMED/FAILED)

2. Portal-Side Sync Trigger

Hourly Sync Command Execution

// Portal Command: sync:sat-usage-reports
$dates = [
    'start_date' => Carbon::now()->subHour()->startOfHour()->format('Y-m-d H:i:s'),
    'end_date' => Carbon::now()->subHour()->endOfHour()->format('Y-m-d H:i:s')
];

Log::channel('sat-usage')->info("satusage: sync started for dates between " . $dates['start_date'] . " and " . $dates['end_date']);

(new TriggerSatUsageReportsSync)->handle($dates);

Client Selection Process

Portal identifies active clients for data collection:

• Clients with active licenses
• Accounts with recent SAT sync activity
• Excludes deleted clients/accounts
• Filters inactive account IDs

3. Portal-to-Client Data Request

Standard Sync Process

1. Client Enumeration: Get all active client accounts
2. Request Generation: Create data request for specified time period
3. HTTP Request: Send authenticated request to client endpoint
4. Response Processing: Handle client response data
5. Status Management: Update sync status based on response

Fallback Mechanism (Client Callback)

When standard sync fails:

1. Trigger Client Callback: Request client to push data to portal
2. Wait Period: Allow up to 30 seconds for client response (3 × 10 seconds)
3. Status Check: Monitor hourly report status changes
4. Failure Handling: Mark as failed if no response within timeout

4. Sync Status Management

Four-Stage Status Flow

┌─────────────┐    ┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   PENDING   │───►│   SUCCESS   │───►│ CONFIRMED   │    │   FAILED    │
│             │    │             │    │             │    │             │
│ Initial     │    │ Data        │    │ Processing  │    │ Error at    │
│ Request     │    │ Received    │    │ Confirmed   │    │ Any Stage   │
│ Sent        │    │ from Client │    │ to Client   │    │             │
└─────────────┘    └─────────────┘    └─────────────┘    └─────────────┘

Status Definitions

• PENDING: Initial log created, request sent to client
• SUCCESS: Data successfully received from client system
• CONFIRMED: Portal confirmed successful processing to client
• FAILED: Sync failed at any stage (network, validation, processing)

5. Data Processing and Storage

Portal Data Processing Pipeline

1. Data Validation: Verify data structure and content integrity
2. User Categorization: Process active users by role type
   • Backend users = backend_users + supervisors + other_users
   • Field users = field_users count
3. Metric Calculation: Compute derived metrics and percentages
4. Hourly Storage: Store raw hourly data in sat_usage_hourly_reports
5. Daily Aggregation: Update or create daily summary records
6. License Correlation: Match with portal license data for analysis

Daily Report Aggregation Logic

// For new daily reports
$create_data = array_merge($client_data, $licences, $orders, $sales, $visits, $deliveries, $current_branch);
SatUsageDailyReport::create($create_data);

// For existing daily reports (sum transactions, latest user counts)
$formatted_daily_data = $this->formatDailyData($hourlyReport, $data);
$dailyReport->updateOrCreate($client_data, $aggregated_data);

6. Confirmation and Callback

Client Confirmation Process

1. Callback Data Preparation: Include log_id and success status
2. Client Notification: Send confirmation to client callback endpoint
3. Status Update: Mark hourly report as CONFIRMED
4. Error Handling: Handle callback failures gracefully

Callback Data Structure

$callback_data = [
    'log_id' => $response['response']['meta']['log_id'],
    'status' => SyncStatusEnum::SUCCESS
];

7. Error Handling and Recovery

Error Scenarios

1. Client Unavailable: Network timeouts or connection failures
2. Authentication Errors: Invalid or expired tokens
3. Data Format Issues: Malformed or incomplete data
4. Processing Errors: Internal portal processing failures

Recovery Mechanisms

• Automatic Retry: Built-in retry logic for transient failures
• Fallback Callback: Alternative data collection method
• Comprehensive Logging: Detailed error tracking for debugging
• Status Tracking: Persistent status management for recovery operations

8. Performance Optimization

Client-Side Optimizations

• Query Efficiency: Optimized database queries for data collection
• Memory Management: Efficient handling of large datasets
• Concurrent Processing: Parallel data collection where possible

Portal-Side Optimizations

• Batch Processing: Process multiple clients concurrently
• Database Indexing: Optimized indexes for fast query performance
• Queue Management: Asynchronous processing for scalability

Troubleshooting Sync Issues

Common Sync Problems

1. Authentication Failures

• Symptoms: 401 responses, authentication errors in logs
• Solutions: Verify client tokens, check token expiry, validate credentials

2. Network Timeouts

• Symptoms: Connection timeouts, no response from client
• Solutions: Check network connectivity, verify client system status, increase timeout values

3. Data Format Issues

• Symptoms: 422 validation errors, malformed data warnings
• Solutions: Validate JSON structure, check required fields, verify data types

4. Sync Status Stuck

• Symptoms: Reports stuck in PENDING status
• Solutions: Check client system health, verify callback endpoints, manual re-sync

Debugging Tools

Log Analysis

# Check portal sync logs
tail -f storage/logs/sat-usage.log

# Check client transmission logs
tail -f storage/logs/sat-usage-radar.log

Status Monitoring

// Check sync status for specific client
$hourlyReports = SatUsageHourlyReport::where('client_id', $clientId)
    ->where('status', 'pending')
    ->orderBy('created_at', 'desc')
    ->get();

Related Documentation

• Architecture & Components: System design and components overview
• Configuration & Deployment: Setup and configuration details