No backoff or remote state refresh on data_start does not match error causes continuous retry loop

[BAPBC]

Repository: emoncms/sync (affects sync_upload.php, Sync module v3.2.0 through v3.4.0)

Describe the bug

When sync_upload.php receives a success==false response from the remote emoncms.org server with the message data_start does not match local npoints, the service enters a continuous retry loop. The loop runs once per minute for as long as the mismatch persists — observed durations of 12+ hours on production systems.

The root cause is two missing behaviours in the $result->success==false handler:

1. No remote state refresh — after a failed upload, the in-memory $feeds array retains the stale remote->npoints value. Every subsequent retry uses this stale value, guaranteeing the same failure again without any possibility of self-recovery.

2. Insufficient backoff — the existing sleep(60) applies to all failure types equally. For a feed state mismatch caused by server-side maintenance, 60 seconds is far too short and results in the client hammering the server continuously until the server-side state normalizes.

To reproduce

The condition occurs reliably during the emoncms.org server maintenance window (approximately 07:00–09:00 UTC daily). Any client with a PHPFina feed whose remote data_start diverges from local npoints during that window will enter the loop.

Observed in journal:
emoncms_sync: - Upload size: 1680
emoncms_sync: - Upload data_start does not match local npoints for feedid=XXXXXX
emoncms_sync: - Upload size: 1680
emoncms_sync: - Upload data_start does not match local npoints for feedid=XXXXXX
Repeating every ~60 seconds for hours with no recovery.

Impact

On systems connected via metered or cellular WAN connections, each failed retry downloads remote feed metadata from emoncms.org before the mismatch is detected. Observed download volume during a single 12-hour error window: ~57 MiB/hour, sustained across the full maintenance window. On a cellular connection this causes significant unplanned data usage and associated cost. Multiple production sites confirmed affected simultaneously.

Proposed fix

In the $result->success==false block in sync_upload.php, add a specific handler for the data_start does not match condition that:

1. Re-fetches the remote feed list via $sync->get_feed_list($userid) to refresh in-memory state before the next retry
2. Backs off for max(300, $upload_interval) seconds rather than 60 seconds, aligning the retry interval with the normal upload cycle and avoiding server hammering during maintenance windows

PHP

if ($result->success==false) {
    print "- ".$result->message."\n";

    if (strpos($result->message, 'data_start does not match') !== false) {
        print "- Refreshing remote feed list after mismatch\n";
        $feeds = $sync->get_feed_list($userid);

        $remote_id_map = array();
        foreach ($feeds as $tagname=>$f) {
            if ($feeds[$tagname]->remote->exists) {
                $remote_id_map[$feeds[$tagname]->remote->id] = $tagname;
            }
        }

        $backoff = max(300, $upload_interval);
        print "- Sleeping ".$backoff."s before retry\n";
        if (!$background_service) die();
        sleep($backoff);
        continue;
    }

    if (!$background_service) die();
    sleep(60);
    continue;
}

Environment

• EmonCMS v11.9.10, Sync module v3.2.0 through v3.4.0 (all confirmed affected, identical code path)
• Raspberry Pi 4, emonSD-01Feb24 image
• PHP 8.1, Apache 2.4

Additional context

The fix has been deployed and verified on 6 production systems. The data_start does not match error resolves within one retry cycle after the fix is applied. No data loss or feed corruption results from the fix — the remote feed list refresh simply re-reads the current server state before the next upload attempt.