light-backup

A LightOS module that automatically/periodically backs up a Light Phone user's data to their own cloud storage (Google Drive, Dropbox, any WebDAV server, OneDrive + more to come). Each tool is responsible for managing which of its data gets backed up and what that data looks like when stored. At backup time, a tool will be presented with the timestamp of each of its paths' last successful backup, and based on that it will provide a list of files to be copied onto the user's cloud storage.

Though this will be primarily used by LightOS, we will put as much of the actual code that consumes it inside the server module of the SDK, and we will at least hook up one dummy cloud provider in the LightOS emulator so you can test your own tools' backups.

This repo is open source, but is depended on by shipping Light Phone products - see CONTRIBUTING.md for what kinds of changes we're currently able to accept.

Architecture

• RemoteBackup - the interface a cloud provider implements: create a directory, upload/list/download files, and report each path's last-successful-backup time (getMostRecentBackupDates()). See GoogleDriveRemoteBackup and DropboxRemoteBackup for examples, or WebDavRemoteBackup for a non-OAuth one (Nextcloud, ownCloud, Synology, ...) built on dav4jvm.
• BackupDataSource - the interface a host app implements to say what to back up (getPathsToBackUp, getFilesToBackUpForPath, readFile, hashForFile, getEarliestPossibleBackupDate). This library ships no implementation, it only knows how to move bytes once told which ones.
• BackupRunner - orchestrates one backup pass: for each path, splits the time since its last backup into bounded, independently-completable windows (one day by default - see chunkWindows), uploads what's changed within each window, writes its checksum manifest, and records a _meta entry for any window that made progress (these are used to determine time of last successful backup). Windowing means a long catch-up period doesn't have to be redone in full after a partial failure - only the windows after the last one that completed do. Platform-agnostic - on Android it's invoked by BackupWorker, an Android WorkManager CoroutineWorker.
• RemoteAccessTokenProvider - provider-agnostic interface for access token caching and invalidation. Implementations (and the OAuth linking flow, which talks to the companion light-oauth-relay worker) live in the SDK server module.
• RemoteConnection - how to reach a linked account. The host's BackupDependencyProvider should return one per kind: createOAuthConnection (a RemoteAccessTokenProvider) for Google/Dropbox/other OAuth providers, createWebDavConnection (server URL plus Basic or Bearer WebDavCredentials), and optionally createUnspecifiedConnection for a Custom backend. There's no linking flow for WebDAV, so the host should call WebDavRemoteBackup.verify() to validate a server URL and credentials before saving them.

Project layout

Most code lives in the jvmShared[x] source sets since it does not need to be Android-specific. Integration tests (which write to actual cloud storage) can be run on a PC.

library/src/
  jvmSharedMain/  # shared between jvm() and androidTarget() - most of the library lives here
  jvmSharedTest/  # tests for the above, run on the jvm() target
  jvmMain/        # jvm()-only code
  androidMain/    # Android-only, mostly boilerplate/wrappers to run backups using WorkManager

Building & testing

./gradlew check

Most tests are plain unit/fake-based tests. A handful are contract tests that run against a real Google Drive or Dropbox account - RemoteBackupContractTest and BackupRunnerContractTest, each run against both providers via GoogleDriveRemoteBackupContractTest/ DropboxRemoteBackupContractTest and GoogleBackupRunnerContractTest/ DropboxBackupRunnerContractTest - they write real files into real cloud storage. They're skipped automatically unless you set GOOGLE_DRIVE_TEST_ACCESS_TOKEN and/or DROPBOX_TEST_ACCESS_TOKEN to a valid access token for a disposable/test account. Use with caution.

The WebDAV equivalents (WebDav*ContractTest) run whenever WEBDAV_TEST_URL (plus WEBDAV_TEST_USERNAME/WEBDAV_TEST_PASSWORD) is set. They need no real account - CI runs them against a throwaway local server, and .github/webdav-test-server.yaml shows how to run the same one yourself.