Sync Queue
File de synchronisation pour les opérations échouées hors ligne. Les actions sont mises en queue, priorisées et retentées automatiquement au retour du réseau avec backoff exponentiel.
Overview
La Sync Queue est un composant essentiel de l'architecture offline de YaniPay. Lorsqu'une opération échoue en raison d'une absence de réseau, elle est automatiquement ajoutée à une file d'attente persistante. Dès que la connectivité est rétablie, les opérations sont traitées dans l'ordre de priorité avec un système de retry intelligent.
Priority Queue
4 niveaux de priorité
Auto Retry
Backoff exponentiel
Persistent
AsyncStorage durable
Monitored
Compteurs en temps réel
Queue Flow
Voici le flux complet d'une opération depuis son échec jusqu'à sa résolution.
Implementation
L'implémentation de la Sync Queue gère l'ajout, le traitement et le nettoyage des opérations en attente.
import AsyncStorage from '@react-native-async-storage/async-storage';
import NetInfo from '@react-native-community/netinfo';
// Queue item priority
export type QueuePriority = 'critical' | 'high' | 'normal' | 'low';
// Sync status for each item
export type SyncStatus =
| 'pending'
| 'processing'
| 'completed'
| 'failed'
| 'permanently_failed';
// Queue item interface
export interface QueueItem<T = unknown> {
id: string;
operation: string; // e.g., 'payment.transfer', 'profile.update'
payload: T;
priority: QueuePriority;
status: SyncStatus;
createdAt: number;
lastAttempt: number | null;
attemptCount: number;
maxRetries: number;
nextRetryAt: number | null;
error: string | null;
metadata?: Record<string, unknown>;
}
// Priority weights for sorting
const PRIORITY_WEIGHTS: Record<QueuePriority, number> = {
critical: 0,
high: 1,
normal: 2,
low: 3,
};
const QUEUE_KEY = 'yanipay_sync_queue';
const MAX_QUEUE_SIZE = 100;
// Operation handlers registry
type OperationHandler<T = unknown> = (payload: T) => Promise<void>;
const handlers = new Map<string, OperationHandler>();
export class OfflineQueue {
private static processing = false;
// Register an operation handler
static registerHandler<T>(
operation: string,
handler: OperationHandler<T>
): void {
handlers.set(operation, handler as OperationHandler);
}
// Add operation to queue
static async enqueue<T>(
operation: string,
payload: T,
options: {
priority?: QueuePriority;
maxRetries?: number;
metadata?: Record<string, unknown>;
} = {}
): Promise<QueueItem<T>> {
const queue = await this.getQueue();
// Check queue size limit
if (queue.length >= MAX_QUEUE_SIZE) {
// Remove oldest low-priority completed/failed items
const pruned = queue
.filter(
(item) =>
item.status !== 'completed' &&
item.status !== 'permanently_failed'
)
.slice(0, MAX_QUEUE_SIZE - 1);
await this.saveQueue(pruned);
}
const item: QueueItem<T> = {
id: `queue_${Date.now()}_${Math.random().toString(36).slice(2)}`,
operation,
payload,
priority: options.priority ?? 'normal',
status: 'pending',
createdAt: Date.now(),
lastAttempt: null,
attemptCount: 0,
maxRetries: options.maxRetries ?? 5,
nextRetryAt: null,
error: null,
metadata: options.metadata,
};
const updatedQueue = await this.getQueue();
updatedQueue.push(item as QueueItem);
await this.saveQueue(updatedQueue);
// Try to process immediately if online
const state = await NetInfo.fetch();
if (state.isConnected && state.isInternetReachable) {
this.processQueue();
}
return item;
}
// Process all pending items in priority order
static async processQueue(): Promise<void> {
if (this.processing) return;
this.processing = true;
try {
const queue = await this.getQueue();
// Sort by priority then by creation time
const pending = queue
.filter(
(item) =>
item.status === 'pending' &&
(item.nextRetryAt === null ||
Date.now() >= item.nextRetryAt)
)
.sort((a, b) => {
const priorityDiff =
PRIORITY_WEIGHTS[a.priority] -
PRIORITY_WEIGHTS[b.priority];
if (priorityDiff !== 0) return priorityDiff;
return a.createdAt - b.createdAt;
});
for (const item of pending) {
const handler = handlers.get(item.operation);
if (!handler) {
item.status = 'permanently_failed';
item.error = `No handler for operation: ${item.operation}`;
continue;
}
// Check network before each item
const state = await NetInfo.fetch();
if (!state.isConnected) {
break; // Stop processing if offline
}
item.status = 'processing';
item.lastAttempt = Date.now();
item.attemptCount++;
try {
await handler(item.payload);
item.status = 'completed';
item.error = null;
} catch (error) {
const errMsg =
error instanceof Error ? error.message : 'Unknown error';
if (item.attemptCount >= item.maxRetries) {
item.status = 'permanently_failed';
item.error = `Max retries exceeded: ${errMsg}`;
} else {
item.status = 'pending';
item.error = errMsg;
// Exponential backoff: 1s, 2s, 4s, 8s, 16s
const delay = Math.pow(2, item.attemptCount - 1) * 1000;
item.nextRetryAt = Date.now() + delay;
}
}
}
// Save updated queue
await this.saveQueue(
queue.filter(
(item) => item.status !== 'completed'
)
);
} finally {
this.processing = false;
}
}
// Clear the entire queue
static async clearQueue(): Promise<void> {
await AsyncStorage.removeItem(QUEUE_KEY);
}
// Get queue from storage
static async getQueue(): Promise<QueueItem[]> {
const raw = await AsyncStorage.getItem(QUEUE_KEY);
return raw ? JSON.parse(raw) : [];
}
// Save queue to storage
private static async saveQueue(
queue: QueueItem[]
): Promise<void> {
await AsyncStorage.setItem(QUEUE_KEY, JSON.stringify(queue));
}
// Get queue statistics
static async getStats(): Promise<{
pending: number;
processing: number;
failed: number;
permanentlyFailed: number;
total: number;
oldestItem: number | null;
byPriority: Record<QueuePriority, number>;
}> {
const queue = await this.getQueue();
const stats = {
pending: 0,
processing: 0,
failed: 0,
permanentlyFailed: 0,
total: queue.length,
oldestItem: queue.length > 0
? Math.min(...queue.map((i) => i.createdAt))
: null,
byPriority: {
critical: 0,
high: 0,
normal: 0,
low: 0,
},
};
for (const item of queue) {
if (item.status === 'pending') stats.pending++;
if (item.status === 'processing') stats.processing++;
if (item.status === 'failed') stats.failed++;
if (item.status === 'permanently_failed')
stats.permanentlyFailed++;
stats.byPriority[item.priority]++;
}
return stats;
}
}Priority Levels
Chaque opération en queue se voit assigner un niveau de priorité qui détermine l'ordre de traitement au retour du réseau.
Critical
Max 5 retriesTraité en premier. Max 5 retries. Délai minimum entre retries.
Exemples : Paiements en attente, confirmations de virement
High
Max 5 retriesTraité après les critiques. Backoff standard.
Exemples : Mises à jour du profil, changements de paramètres de carte
Normal
Max 3 retriesTraitement standard. Retry modéré.
Exemples : Préférences utilisateur, abonnements newsletter
Low
Max 2 retriesTraité en dernier. Supprimé si la queue est pleine.
Exemples : Événements analytics, logs non critiques, synchro préférences UI
Ordre de traitement
Retry Strategy
Le système utilise un backoff exponentiel pour éviter de surcharger le serveur lors du retour en ligne.
Exponential Backoff Timeline
Après 5 tentatives échouées (31 secondes au total), l'opération est marquée comme "permanently_failed" et nécessite une intervention manuelle.
// Retry strategy configuration per priority
export interface RetryConfig {
maxRetries: number;
baseDelayMs: number;
maxDelayMs: number;
backoffMultiplier: number;
}
export const RETRY_CONFIGS: Record<QueuePriority, RetryConfig> = {
critical: {
maxRetries: 5,
baseDelayMs: 1000, // 1 second
maxDelayMs: 16000, // 16 seconds max
backoffMultiplier: 2,
},
high: {
maxRetries: 5,
baseDelayMs: 1000,
maxDelayMs: 30000, // 30 seconds max
backoffMultiplier: 2,
},
normal: {
maxRetries: 3,
baseDelayMs: 2000, // 2 seconds
maxDelayMs: 60000, // 1 minute max
backoffMultiplier: 2,
},
low: {
maxRetries: 2,
baseDelayMs: 5000, // 5 seconds
maxDelayMs: 120000, // 2 minutes max
backoffMultiplier: 3,
},
};
// Calculate next retry delay
export function calculateRetryDelay(
attemptCount: number,
config: RetryConfig
): number {
const delay = Math.min(
config.baseDelayMs *
Math.pow(config.backoffMultiplier, attemptCount - 1),
config.maxDelayMs
);
// Add jitter (10% random variation) to prevent thundering herd
const jitter = delay * 0.1 * Math.random();
return Math.floor(delay + jitter);
}
// Determine if error is temporary or permanent
export function isTemporaryError(error: unknown): boolean {
if (error instanceof Error) {
const message = error.message.toLowerCase();
// Permanent errors - do not retry
const permanentPatterns = [
'unauthorized',
'401',
'forbidden',
'403',
'not found',
'404',
'validation',
'invalid',
'duplicate',
'already exists',
];
if (permanentPatterns.some((p) => message.includes(p))) {
return false;
}
}
// Temporary by default (network, timeout, 5xx)
return true;
}Conflict Resolution
Lorsque des opérations en queue entrent en conflit avec des changements effectués sur le serveur pendant l'absence de réseau, différentes stratégies sont appliquées selon le type de donnée.
Last-Write-Wins
Pour les préférences utilisateur (thème, langue, notifications). La dernière modification écrase les précédentes.
Appliqué à : Theme, language, notification settings
Server-Authoritative
Pour les données financières (soldes, transactions). Le serveur est toujours la source de vérité.
Appliqué à : Balance, transactions, card status
Merge Non-Conflicting
Pour les mises à jour partielles (profil). Les champs non conflictés sont fusionnés.
Appliqué à : Profile fields, address, preferences
export type ConflictStrategy =
| 'last-write-wins'
| 'server-authoritative'
| 'merge';
export interface ConflictResolution<T> {
strategy: ConflictStrategy;
localData: T;
serverData: T;
resolvedData: T;
conflictFields: string[];
}
// Resolve conflicts based on strategy
export function resolveConflict<T extends Record<string, unknown>>(
localData: T,
serverData: T,
strategy: ConflictStrategy,
localTimestamp: number,
serverTimestamp: number
): ConflictResolution<T> {
const conflictFields: string[] = [];
// Find conflicting fields
for (const key of Object.keys(localData)) {
if (
JSON.stringify(localData[key]) !==
JSON.stringify(serverData[key])
) {
conflictFields.push(key);
}
}
let resolvedData: T;
switch (strategy) {
case 'last-write-wins':
resolvedData =
localTimestamp > serverTimestamp ? localData : serverData;
break;
case 'server-authoritative':
resolvedData = serverData;
break;
case 'merge':
// Merge non-conflicting changes, server wins on conflicts
resolvedData = { ...serverData };
for (const key of Object.keys(localData)) {
if (!conflictFields.includes(key)) {
(resolvedData as Record<string, unknown>)[key] =
localData[key];
}
}
break;
default:
resolvedData = serverData;
}
return {
strategy,
localData,
serverData,
resolvedData,
conflictFields,
};
}
// Map operations to conflict strategies
export const CONFLICT_STRATEGIES: Record<string, ConflictStrategy> = {
'profile.update': 'merge',
'preferences.update': 'last-write-wins',
'card.settings': 'server-authoritative',
'wallet.transfer': 'server-authoritative',
'loyalty.redeem': 'server-authoritative',
'notification.preferences': 'last-write-wins',
};Queue Persistence
La queue est persistée dans AsyncStorage pour survivre aux redémarrages de l'application. Chaque modification de la queue déclenche une sauvegarde immédiate.
import AsyncStorage from '@react-native-async-storage/async-storage';
import { QueueItem } from './offlineQueue';
const QUEUE_STORAGE_KEY = 'yanipay_sync_queue';
const QUEUE_METADATA_KEY = 'yanipay_sync_queue_meta';
interface QueueMetadata {
lastProcessed: number | null;
totalProcessed: number;
totalFailed: number;
createdAt: number;
version: number;
}
// Persist queue with metadata
export async function persistQueue(
queue: QueueItem[]
): Promise<void> {
await AsyncStorage.setItem(
QUEUE_STORAGE_KEY,
JSON.stringify(queue)
);
}
// Restore queue from storage
export async function restoreQueue(): Promise<QueueItem[]> {
const raw = await AsyncStorage.getItem(QUEUE_STORAGE_KEY);
if (!raw) return [];
const queue: QueueItem[] = JSON.parse(raw);
// Reset any items stuck in 'processing' state
// (app was killed during processing)
return queue.map((item) => ({
...item,
status:
item.status === 'processing' ? 'pending' : item.status,
}));
}
// Update metadata after processing
export async function updateMetadata(
processed: number,
failed: number
): Promise<void> {
const raw = await AsyncStorage.getItem(QUEUE_METADATA_KEY);
const meta: QueueMetadata = raw
? JSON.parse(raw)
: {
lastProcessed: null,
totalProcessed: 0,
totalFailed: 0,
createdAt: Date.now(),
version: 1,
};
meta.lastProcessed = Date.now();
meta.totalProcessed += processed;
meta.totalFailed += failed;
await AsyncStorage.setItem(
QUEUE_METADATA_KEY,
JSON.stringify(meta)
);
}
// Get metadata
export async function getMetadata(): Promise<QueueMetadata | null> {
const raw = await AsyncStorage.getItem(QUEUE_METADATA_KEY);
return raw ? JSON.parse(raw) : null;
}Limite de taille de la queue
Queue Monitoring
Un hook React permet de surveiller l'état de la queue en temps réel et d'afficher les compteurs dans l'interface utilisateur.
import { useEffect, useState, useCallback } from 'react';
import { OfflineQueue, QueuePriority, SyncStatus } from '@/lib/offline/offlineQueue';
import { useNetworkStatus } from './useNetworkStatus';
export interface QueueMonitorState {
pendingCount: number;
failedCount: number;
processingCount: number;
totalCount: number;
isProcessing: boolean;
byPriority: Record<QueuePriority, number>;
oldestItemAge: number | null; // in milliseconds
lastRefresh: Date;
}
export function useQueueMonitor(
pollIntervalMs = 5000
): QueueMonitorState & { refresh: () => Promise<void> } {
const [state, setState] = useState<QueueMonitorState>({
pendingCount: 0,
failedCount: 0,
processingCount: 0,
totalCount: 0,
isProcessing: false,
byPriority: { critical: 0, high: 0, normal: 0, low: 0 },
oldestItemAge: null,
lastRefresh: new Date(),
});
const { isConnected } = useNetworkStatus();
const refresh = useCallback(async () => {
const stats = await OfflineQueue.getStats();
setState({
pendingCount: stats.pending,
failedCount: stats.permanentlyFailed,
processingCount: stats.processing,
totalCount: stats.total,
isProcessing: stats.processing > 0,
byPriority: stats.byPriority,
oldestItemAge: stats.oldestItem
? Date.now() - stats.oldestItem
: null,
lastRefresh: new Date(),
});
}, []);
useEffect(() => {
refresh();
const interval = setInterval(refresh, pollIntervalMs);
return () => clearInterval(interval);
}, [refresh, pollIntervalMs]);
// Auto-process when network returns
useEffect(() => {
if (isConnected && state.pendingCount > 0) {
OfflineQueue.processQueue().then(refresh);
}
}, [isConnected, state.pendingCount, refresh]);
return { ...state, refresh };
}
// Component to display queue status badge
// Usage: <QueueStatusBadge /> in header or status bar
export function getQueueStatusInfo(state: QueueMonitorState): {
label: string;
variant: 'success' | 'warning' | 'error' | 'neutral';
showBadge: boolean;
} {
if (state.totalCount === 0) {
return {
label: 'Queue vide',
variant: 'success',
showBadge: false,
};
}
if (state.failedCount > 0) {
return {
label: `${state.failedCount} opération(s) échouée(s)`,
variant: 'error',
showBadge: true,
};
}
if (state.pendingCount > 0) {
return {
label: `${state.pendingCount} en attente de sync`,
variant: 'warning',
showBadge: true,
};
}
if (state.isProcessing) {
return {
label: 'Synchronisation...',
variant: 'neutral',
showBadge: true,
};
}
return {
label: 'Synchronisé',
variant: 'success',
showBadge: false,
};
}Error Handling: Permanent vs Temporary Failures
La queue distingue deux types d'échecs pour adapter son comportement de retry.
Temporary Failures
L'opération sera retentée automatiquement avec backoff exponentiel.
- Network timeout (ETIMEDOUT)
- Server error (500, 502, 503)
- Rate limit (429)
- DNS resolution failure
- Connection reset
Permanent Failures
L'opération est marquée comme définitivement échouée. Pas de retry.
- Unauthorized (401)
- Forbidden (403)
- Not found (404)
- Validation error (422)
- Duplicate operation