A crash-safe storage, savegames and daily backups

- Writes go to a flushed temporary file and are renamed into place; the last
  intact version stays as .bak and reads fall back to it, so a power cut
  never leaves a damaged save. Same file format as before.
- Savegames per account project (newest 30, one per day for 90 days), kept
  before restores and deletions; deleted projects restorable for 30 days.
  Projects dialog: savegames per project and "Recently deleted".
- Daily gzip backup of data/ into backups/ (14 newest + monthly), restore
  script scripts/restore-backup.mjs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Fabian @ Blax Software 2026-10-20 13:20:55 +02:00
parent 5e17826ba5
commit 08b086a71e
24 changed files with 740 additions and 19 deletions

View File

@ -18,3 +18,4 @@ localhost-key.pem
*.map *.map
*.log *.log
data data
backups

1
.gitignore vendored
View File

@ -36,3 +36,4 @@ playwright-report/
# Shared projects (server storage) # Shared projects (server storage)
data/ data/
backups/

View File

@ -41,7 +41,26 @@ The editor UI is in [app/components/editor/](app/components/editor/) with state
## Accounts ## Accounts
[server/api/auth/](server/api/auth/) (register, login, logout, me, password, delete) and [server/api/me/projects/](server/api/me/projects/) (list, get, put, delete). Passwords are hashed with scrypt; the session is a random token in an httpOnly, SameSite=Lax cookie, stored server-side as a SHA-256. API calls that change something are refused when their Origin is another site. Storage (Nitro fs): `data/users`, `data/sessions`, `data/accounts/<uid>/` with an index (revision per project, deletions for a year) and one file per project. Each save sends the revision it started from; a mismatch answers 409 and the browser keeps the other version as a copy. Limits: 1000 projects and 100 MB per account, 2 MB per project, login 20 tries per 10 minutes per IP, 10 new accounts per hour per IP. [server/api/auth/](server/api/auth/) (register, login, logout, me, password, delete) and [server/api/me/projects/](server/api/me/projects/) (list, get, put, delete). Passwords are hashed with scrypt; the session is a random token in an httpOnly, SameSite=Lax cookie, stored server-side as a SHA-256. API calls that change something are refused when their Origin is another site. Storage: `data/users`, `data/sessions`, `data/accounts/<uid>/` with an index (revision per project, deletions for a year), one file per project, and savegames in `h/<project>/`. Each save sends the revision it started from; a mismatch answers 409 and the browser keeps the other version as a copy. Limits: 1000 projects and 100 MB per account, 2 MB per project, login 20 tries per 10 minutes per IP, 10 new accounts per hour per IP.
## Power cuts, savegames, backups
No database: everything is JSON files under `data/`, which keeps memory use flat. Three layers protect it:
- **Crash-safe writes** ([server/utils/safe-fs.ts](server/utils/safe-fs.ts), mounted in [server/plugins/storage.ts](server/plugins/storage.ts)). A save writes a temporary file, flushes it to disk, keeps the current intact version as `<file>.bak` and then renames the new file into place. A power cut at any moment leaves the old or the new version; reading falls back to `.bak` when a file is missing or damaged, and leftover temporary files are removed at start.
- **Savegames** ([server/utils/history.ts](server/utils/history.ts)). Before a save replaces an account project, its state is kept as a version (at most one per 15 minutes; always before a restore and on deletion). Kept: the newest 30 plus the last of each day for 90 days. Deleted projects stay restorable for 30 days. In the app: Projects dialog, clock icon (restore, or open as copy), and "Recently deleted".
- **Daily backup** ([server/utils/backup.ts](server/utils/backup.ts), [server/plugins/backup.ts](server/plugins/backup.ts)). Once a day the production server writes all of `data/` into `backups/logigate-YYYY-MM-DD.ndjson.gz`, leaving out damaged files. Kept: the newest 14 plus the first of each month for a year. `backups/` sits next to `data/` on the same disk, so it protects against damaged files and mistakes, not against losing the disk; copy it elsewhere for that.
Restore a backup on edgerealm (the current `data/` is moved aside first, so this can be undone):
```bash
cd /mnt/HC_Volume_102040804/srv/logigate
docker compose stop frontend
docker run --rm -v "$PWD":/app -w /app node:current-slim node scripts/restore-backup.mjs backups/logigate-2026-10-20.ndjson.gz
docker compose start frontend
```
In the browser, each localStorage write is atomic, so a power cut there loses at most the last few seconds.
## Compatibility ## Compatibility
@ -59,4 +78,4 @@ pnpm build # .output/
## Deploy ## Deploy
Same pattern as autismus-spektrum. On edgerealm the repo lives in `/mnt/HC_Volume_102040804/srv/logigate`; run `./deploy.sh` there. It pulls, builds in a one-shot container and recreates the `logigate` container. Share links and accounts live in `data/` next to the repo (mounted into the container), so redeploys keep them; include that folder in backups. Host nginx terminates TLS with the `*.blax.at` wildcard cert and forwards to Traefik, which routes `logicate.blax.at` and `logigate.blax.at` by the labels in [docker-compose.yml](docker-compose.yml). Same pattern as autismus-spektrum. On edgerealm the repo lives in `/mnt/HC_Volume_102040804/srv/logigate`; run `./deploy.sh` there. It pulls, builds in a one-shot container and recreates the `logigate` container. Share links and accounts live in `data/` and the daily backups in `backups/`, both next to the repo and mounted into the container, so redeploys keep them. Host nginx terminates TLS with the `*.blax.at` wildcard cert and forwards to Traefik, which routes `logicate.blax.at` and `logigate.blax.at` by the labels in [docker-compose.yml](docker-compose.yml).

View File

@ -31,5 +31,6 @@ onMounted(() => {
<EditorProjectsDialog v-if="ed.dialog.value === 'projects'" /> <EditorProjectsDialog v-if="ed.dialog.value === 'projects'" />
<EditorShareDialog v-if="ed.dialog.value === 'share'" /> <EditorShareDialog v-if="ed.dialog.value === 'share'" />
<EditorAccountDialog v-if="ed.dialog.value === 'account'" /> <EditorAccountDialog v-if="ed.dialog.value === 'account'" />
<EditorVersionsDialog v-if="ed.dialog.value === 'versions'" />
</div> </div>
</template> </template>

View File

@ -6,6 +6,17 @@ const renameText = ref('')
const confirmDelete = ref<string | null>(null) const confirmDelete = ref<string | null>(null)
const close = () => (ed.dialog.value = null) const close = () => (ed.dialog.value = null)
const showTrash = ref(false)
const trash = ref<{ id: string; name: string; deleted: number; v: number }[] | null>(null)
async function toggleTrash() {
showTrash.value = !showTrash.value
if (showTrash.value) trash.value = await ed.listTrash().catch(() => [])
}
async function restoreDeleted(t: { id: string; v: number }) {
await ed.restoreDeleted(t)
close()
}
const when = (t: number) => new Intl.DateTimeFormat(undefined, { dateStyle: 'medium', timeStyle: 'short' }).format(t) const when = (t: number) => new Intl.DateTimeFormat(undefined, { dateStyle: 'medium', timeStyle: 'short' }).format(t)
function open(id: string) { function open(id: string) {
@ -69,6 +80,14 @@ function onFile(e: Event) {
<template v-else-if="!ed.account.value"> · <Icon name="lucide:laptop" class="size-3" /> this browser only</template> <template v-else-if="!ed.account.value"> · <Icon name="lucide:laptop" class="size-3" /> this browser only</template>
</span> </span>
</button> </button>
<button
v-if="ed.account.value && m.cloud?.owner === ed.account.value.id"
class="icon-btn"
title="Savegames: earlier versions"
@click="ed.versionsFor.value = m.id; ed.dialog.value = 'versions'"
>
<Icon name="lucide:history" class="size-3.5" />
</button>
<button class="icon-btn" title="Rename" @click="startRename(m.id, m.name)"><Icon name="lucide:pencil" class="size-3.5" /></button> <button class="icon-btn" title="Rename" @click="startRename(m.id, m.name)"><Icon name="lucide:pencil" class="size-3.5" /></button>
<button class="icon-btn" title="Duplicate" @click="ed.duplicateProject(m.id); close()"><Icon name="lucide:copy" class="size-3.5" /></button> <button class="icon-btn" title="Duplicate" @click="ed.duplicateProject(m.id); close()"><Icon name="lucide:copy" class="size-3.5" /></button>
<button v-if="confirmDelete !== m.id" class="icon-btn" title="Delete" @click="confirmDelete = m.id"><Icon name="lucide:trash-2" class="size-3.5" /></button> <button v-if="confirmDelete !== m.id" class="icon-btn" title="Delete" @click="confirmDelete = m.id"><Icon name="lucide:trash-2" class="size-3.5" /></button>
@ -76,6 +95,26 @@ function onFile(e: Event) {
</li> </li>
</ul> </ul>
<template v-if="ed.account.value">
<button class="section-toggle mt-4" @click="toggleTrash">
<Icon :name="showTrash ? 'lucide:chevron-down' : 'lucide:chevron-right'" class="size-4" />
Recently deleted
</button>
<template v-if="showTrash">
<p v-if="!trash" class="hint mt-2">Loading…</p>
<p v-else-if="!trash.length" class="hint mt-2">Nothing deleted in the last 30 days.</p>
<ul v-else class="project-list">
<li v-for="t in trash" :key="t.id">
<div class="pick">
<span class="name">{{ t.name }}</span>
<span class="meta">deleted {{ when(t.deleted) }}</span>
</div>
<button class="btn-sm" @click="restoreDeleted(t)">Restore</button>
</li>
</ul>
</template>
</template>
<div class="mt-4 flex flex-wrap gap-2"> <div class="mt-4 flex flex-wrap gap-2">
<button class="btn" @click="ed.newProject(false); close()"><Icon name="lucide:file" class="size-4" /> New (NAND only)</button> <button class="btn" @click="ed.newProject(false); close()"><Icon name="lucide:file" class="size-4" /> New (NAND only)</button>
<button class="btn" @click="ed.newProject(true); close()"><Icon name="lucide:file-plus" class="size-4" /> New with examples</button> <button class="btn" @click="ed.newProject(true); close()"><Icon name="lucide:file-plus" class="size-4" /> New with examples</button>

View File

@ -0,0 +1,66 @@
<script setup lang="ts">
const ed = useEditor()
const close = () => {
ed.dialog.value = 'projects'
ed.versionsFor.value = null
}
const id = computed(() => ed.versionsFor.value)
const name = computed(() => ed.projects.value.find(m => m.id === id.value)?.name ?? '')
const versions = ref<{ v: number; at: number; rev: number; name: string; size: number }[] | null>(null)
const error = ref('')
const busy = ref<number | null>(null)
const when = (t: number) => new Intl.DateTimeFormat(undefined, { dateStyle: 'medium', timeStyle: 'short' }).format(t)
const kb = (n: number) => `${Math.max(1, Math.round(n / 1024))} KB`
onMounted(async () => {
if (!id.value) return
try {
versions.value = await ed.listVersions(id.value)
} catch (e) {
error.value = (e as Error).message
}
})
async function restore(v: number, asCopy: boolean) {
if (!id.value) return
busy.value = v
try {
await ed.restoreVersion(id.value, v, asCopy)
ed.dialog.value = null
} catch (e) {
error.value = (e as Error).message
} finally {
busy.value = null
}
}
</script>
<template>
<div class="modal-backdrop" @click.self="close">
<div class="modal" role="dialog" aria-labelledby="versions-title">
<div class="flex items-start justify-between gap-4">
<h2 id="versions-title">Savegames of "{{ name }}"</h2>
<button class="tb-btn" title="Back" @click="close"><Icon name="lucide:x" class="size-4" /></button>
</div>
<p class="hint mt-1">
Kept automatically in your account while you work: a version at most every 15 minutes, the newest 30 plus one per day for 90 days.
Restoring keeps the current state as a savegame too, so nothing is lost.
</p>
<p v-if="error" class="result fail"><Icon name="lucide:circle-x" class="size-4 shrink-0" /> {{ error }}</p>
<p v-else-if="!versions" class="hint">Loading…</p>
<p v-else-if="!versions.length" class="hint">No savegames yet. The first one is kept the next time this project changes, 15 minutes after the last save.</p>
<ul v-else class="project-list">
<li v-for="v in versions" :key="v.v">
<div class="pick">
<span class="name">{{ when(v.at) }}</span>
<span class="meta">{{ v.name }} · revision {{ v.rev }} · {{ kb(v.size) }}</span>
</div>
<button class="btn-sm" :disabled="busy !== null" title="Replace the project with this version" @click="restore(v.v, false)">Restore</button>
<button class="btn-sm" :disabled="busy !== null" title="Open this version as a separate project" @click="restore(v.v, true)">As copy</button>
</li>
</ul>
</div>
</div>
</template>

View File

@ -105,7 +105,9 @@ function create() {
const notice = ref<{ text: string; kind: 'ok' | 'error' } | null>(null) const notice = ref<{ text: string; kind: 'ok' | 'error' } | null>(null)
const labelFocus = ref(0) const labelFocus = ref(0)
const showHelp = ref(false) const showHelp = ref(false)
const dialog = ref<null | 'projects' | 'share' | 'account'>(null) const dialog = ref<null | 'projects' | 'share' | 'account' | 'versions'>(null)
/** Project whose savegames the versions dialog shows. */
const versionsFor = ref<string | null>(null)
let noticeTimer: ReturnType<typeof setTimeout> | undefined let noticeTimer: ReturnType<typeof setTimeout> | undefined
function notify(text: string, kind: 'ok' | 'error' = 'ok') { function notify(text: string, kind: 'ok' | 'error' = 'ok') {
@ -855,14 +857,14 @@ function create() {
async function upload(m: ProjectMeta, uid: string) { async function upload(m: ProjectMeta, uid: string) {
const p = localProject(m) const p = localProject(m)
if (!p) return if (!p) return
const body = { project: p, baseRev: m.cloud?.owner === uid ? m.cloud.rev : 0, link: m.link } const body = { project: p, baseRev: m.cloud?.owner === uid ? m.cloud.rev : 0, link: m.link, snapshot: m.snapshot === true }
let res = await authCall<{ meta?: CloudMeta; conflict?: boolean; status: number }>('PUT', `/api/me/projects/${m.id}`, body) let res = await authCall<{ meta?: CloudMeta; conflict?: boolean; status: number }>('PUT', `/api/me/projects/${m.id}`, body)
if (res.conflict) { if (res.conflict) {
await keepRemoteCopy(m.id) await keepRemoteCopy(m.id)
res = await authCall('PUT', `/api/me/projects/${m.id}`, { ...body, force: true }) res = await authCall('PUT', `/api/me/projects/${m.id}`, { ...body, force: true })
notify(`"${m.name}" was also changed on another device. That version is kept as a copy.`) notify(`"${m.name}" was also changed on another device. That version is kept as a copy.`)
} }
if (res.meta) updateMeta(m.id, { cloud: { owner: uid, rev: res.meta.rev }, dirty: false }) if (res.meta) updateMeta(m.id, { cloud: { owner: uid, rev: res.meta.rev }, dirty: false, snapshot: undefined })
} }
let syncingAccount = false let syncingAccount = false
@ -999,6 +1001,64 @@ function create() {
notify('Account deleted. Your projects are still in this browser.') notify('Account deleted. Your projects are still in this browser.')
} }
// ── Savegames ─────────────────────────────────────────────────────
interface VersionInfo {
v: number
at: number
rev: number
name: string
size: number
}
async function listVersions(projectId: string): Promise<VersionInfo[]> {
const res = await authCall<{ versions: VersionInfo[]; status: number }>('GET', `/api/me/projects/${projectId}/versions`)
return res.versions ?? []
}
async function fetchVersion(projectId: string, v: number): Promise<Project> {
const res = await authCall<{ version?: { project: unknown }; status: number }>('GET', `/api/me/projects/${projectId}/versions/${v}`)
if (!res.version) throw new Error('Version not found')
return toProject(res.version.project)
}
/**
* Brings back a savegame: as a new project, or in place of the current
* state (which the server keeps as a savegame first, so this can be undone).
*/
async function restoreVersion(projectId: string, v: number, asCopy: boolean) {
if (projectId === meta.value.id && saveTimer) persistNow()
const p = await fetchVersion(projectId, v)
if (asCopy) {
p.name = `${p.name} (${new Date(v).toLocaleDateString()})`
activate(addProject(p), p)
notify(`Opened the saved version as "${p.name}"`)
return
}
const m = listProjects().find(x => x.id === projectId)
if (!m) return
const next: ProjectMeta = { ...m, name: p.name, updated: Date.now(), dirty: true, snapshot: true }
saveProject(next, p)
activate(next, p)
scheduleCloud()
notify('Restored. The state before is kept as a savegame too.')
}
async function listTrash(): Promise<{ id: string; name: string; deleted: number; v: number }[]> {
const res = await authCall<{ projects: { id: string; name: string; deleted: number; v: number }[]; status: number }>('GET', '/api/me/trash')
return res.projects ?? []
}
async function restoreDeleted(item: { id: string; v: number }) {
const p = await fetchVersion(item.id, item.v)
persistNow()
setPendingDeletes(pendingDeletes().filter(d => d.id !== item.id))
const m: ProjectMeta = { id: item.id, name: p.name, updated: Date.now(), dirty: true }
saveProject(m, p)
activate(m, p)
scheduleCloud()
notify(`"${p.name}" is back`)
}
/** Checks the session with the server, then syncs; again on focus and every minute. */ /** Checks the session with the server, then syncs; again on focus and every minute. */
async function startAccount() { async function startAccount() {
try { try {
@ -1114,6 +1174,11 @@ function create() {
changePassword, changePassword,
deleteAccount, deleteAccount,
startAccount, startAccount,
versionsFor,
listVersions,
restoreVersion,
listTrash,
restoreDeleted,
copyText, copyText,
shape, shape,
} }

View File

@ -21,6 +21,8 @@ export interface ProjectMeta {
dirty?: boolean dirty?: boolean
/** The example project a new browser starts with, never touched yet. */ /** The example project a new browser starts with, never touched yet. */
starter?: boolean starter?: boolean
/** Ask the server to keep the state before the next upload as a savegame (after restoring an old version). */
snapshot?: boolean
} }
export interface AccountUser { export interface AccountUser {

View File

@ -68,7 +68,7 @@ if [ ! -f .env ]; then
fi fi
echo "==> Preparing docker data directories..." echo "==> Preparing docker data directories..."
mkdir -p docker-data/node_modules data/shares mkdir -p docker-data/node_modules data/shares backups
if [ "$(id -u)" -eq 0 ]; then if [ "$(id -u)" -eq 0 ]; then
echo "==> Fixing ownership on writable directories (root mode)..." echo "==> Fixing ownership on writable directories (root mode)..."

View File

@ -27,8 +27,10 @@ services:
- .env - .env
volumes: volumes:
- ./.output:/app/.output:ro - ./.output:/app/.output:ro
# Projects shared by link (one JSON file each). Survives redeploys. # Share links and accounts (JSON files). Survives redeploys.
- ./data:/app/data - ./data:/app/data
# Daily backups of data/, see scripts/restore-backup.mjs.
- ./backups:/app/backups
labels: labels:
traefik.enable: "true" traefik.enable: "true"
traefik.docker.network: "web" traefik.docker.network: "web"

View File

@ -28,15 +28,8 @@ export default defineNuxtConfig({
clientBundle: { scan: true }, clientBundle: { scan: true },
}, },
// Share links and accounts live as JSON files in data/ (a volume in production). // Share links and accounts live as JSON files in data/ (a volume in
nitro: { // production), written crash-safe; see server/plugins/storage.ts.
storage: {
shares: { driver: 'fs', base: './data/shares' },
users: { driver: 'fs', base: './data/users' },
sessions: { driver: 'fs', base: './data/sessions' },
accounts: { driver: 'fs', base: './data/accounts' },
},
},
runtimeConfig: { runtimeConfig: {
public: { public: {

47
scripts/restore-backup.mjs Executable file
View File

@ -0,0 +1,47 @@
#!/usr/bin/env node
// Restores data/ from a backup made by server/utils/backup.ts.
//
// node scripts/restore-backup.mjs backups/logigate-2026-10-20.ndjson.gz [target-dir]
//
// The target (default: data) is moved aside to data.before-restore-<time>
// first, so a restore can itself be undone. Stop the container before
// restoring and start it again afterwards.
import { createReadStream, existsSync } from 'node:fs'
import { mkdir, open, rename } from 'node:fs/promises'
import { dirname, join, resolve } from 'node:path'
import { createInterface } from 'node:readline'
import { createGunzip } from 'node:zlib'
const [file, target = 'data'] = process.argv.slice(2)
if (!file) {
console.error('Usage: node scripts/restore-backup.mjs <backup.ndjson.gz> [target-dir]')
process.exit(1)
}
const dir = resolve(target)
if (existsSync(dir)) {
const aside = `${dir}.before-restore-${new Date().toISOString().replace(/[:.]/g, '-')}`
await rename(dir, aside)
console.log(`Moved the current ${target} to ${aside}`)
}
const lines = createInterface({ input: createReadStream(file).pipe(createGunzip()), crlfDelay: Infinity })
let count = 0
let header = null
for await (const line of lines) {
if (!line.trim()) continue
const entry = JSON.parse(line)
if (!header) {
header = entry
if (entry.logigate !== 'backup') throw new Error('Not a Logigate backup')
continue
}
if (typeof entry.path !== 'string' || entry.path.split('/').includes('..')) throw new Error(`Bad path in backup: ${entry.path}`)
const out = join(dir, entry.path)
await mkdir(dirname(out), { recursive: true })
const fh = await open(out, 'w')
await fh.writeFile(entry.data, 'utf8')
await fh.sync()
await fh.close()
count++
}
console.log(`Restored ${count} files from ${file} (made ${header?.created})`)

View File

@ -4,12 +4,17 @@ export default defineEventHandler(async event => {
if (!isProjectId(id)) throw createError({ statusCode: 404, message: 'Project not found' }) if (!isProjectId(id)) throw createError({ statusCode: 404, message: 'Project not found' })
return withLock(user.id, async () => { return withLock(user.id, async () => {
const index = await readIndex(user.id) const index = await readIndex(user.id)
// The last state goes to the trash (kept as a version for 30 days).
const meta = index.projects.find(p => p.id === id)
const last = meta && (await readProject(user.id, id))
if (meta && last) await keepVersion(user.id, id, { project: last, rev: meta.rev, at: meta.updated, name: meta.name }, true)
index.projects = index.projects.filter(p => p.id !== id) index.projects = index.projects.filter(p => p.id !== id)
// Remember the deletion for a year, so other devices drop their copy too. // Remember the deletion for a year, so other devices drop their copy too.
const yearAgo = Date.now() - 365 * 86_400_000 const yearAgo = Date.now() - 365 * 86_400_000
index.deleted = [{ id, at: Date.now() }, ...index.deleted.filter(d => d.id !== id && d.at > yearAgo)].slice(0, 2000) index.deleted = [{ id, at: Date.now() }, ...index.deleted.filter(d => d.id !== id && d.at > yearAgo)].slice(0, 2000)
await accounts().removeItem(projectKey(user.id, id)) await accounts().removeItem(projectKey(user.id, id))
await writeIndex(user.id, index) await writeIndex(user.id, index)
await emptyOldTrash(user.id, index)
return { ok: true } return { ok: true }
}) })
}) })

View File

@ -1,5 +1,5 @@
import { toProject } from '../../../../app/lib/logic/project' import { toProject } from '../../../../../app/lib/logic/project'
import type { CloudMeta } from '../../../utils/accounts' import type { CloudMeta } from '../../../../utils/accounts'
// Save a project to the account. `baseRev` is the revision the device last // Save a project to the account. `baseRev` is the revision the device last
// had; if another device saved in between, this answers 409 with the newer // had; if another device saved in between, this answers 409 with the newer
// version's meta, unless `force` is set. // version's meta, unless `force` is set.
@ -10,7 +10,7 @@ export default defineEventHandler(async event => {
if (!isProjectId(id)) throw createError({ statusCode: 400, message: 'Bad project id' }) if (!isProjectId(id)) throw createError({ statusCode: 400, message: 'Bad project id' })
const raw = await readRawBody(event, 'utf8') const raw = await readRawBody(event, 'utf8')
if (!raw || raw.length > 2_100_000) throw createError({ statusCode: 413, message: 'Project too large' }) if (!raw || raw.length > 2_100_000) throw createError({ statusCode: 413, message: 'Project too large' })
let body: { project?: unknown; baseRev?: unknown; force?: unknown; link?: unknown } let body: { project?: unknown; baseRev?: unknown; force?: unknown; link?: unknown; snapshot?: unknown }
try { try {
body = JSON.parse(raw) body = JSON.parse(raw)
} catch { } catch {
@ -39,6 +39,11 @@ export default defineEventHandler(async event => {
const meta: CloudMeta = { id, name: project.name, updated: Date.now(), rev: (existing?.rev ?? 0) + 1, size } const meta: CloudMeta = { id, name: project.name, updated: Date.now(), rev: (existing?.rev ?? 0) + 1, size }
if (cleanLink) meta.link = cleanLink if (cleanLink) meta.link = cleanLink
else if (existing?.link && body.link === undefined) meta.link = existing.link else if (existing?.link && body.link === undefined) meta.link = existing.link
// Savegame: keep the state this save replaces.
const previous = existing && (await readProject(user.id, id))
if (existing && previous) {
await keepVersion(user.id, id, { project: previous, rev: existing.rev, at: existing.updated, name: existing.name }, body.snapshot === true)
}
await accounts().setItem(projectKey(user.id, id), project) await accounts().setItem(projectKey(user.id, id), project)
index.projects = [meta, ...index.projects.filter(p => p.id !== id)] index.projects = [meta, ...index.projects.filter(p => p.id !== id)]
index.deleted = index.deleted.filter(d => d.id !== id) index.deleted = index.deleted.filter(d => d.id !== id)

View File

@ -0,0 +1,13 @@
// Savegames of one project, newest first (also works for a deleted project).
export default defineEventHandler(async event => {
const user = await requireUser(event)
const id = getRouterParam(event, 'id')
if (!isProjectId(id)) throw createError({ statusCode: 404, message: 'Project not found' })
const versions = []
for (const t of await versionTimes(user.id, id)) {
const v = await readVersion(user.id, id, t)
if (v) versions.push({ v: t, at: v.at, rev: v.rev, name: v.name, size: JSON.stringify(v.project).length })
}
setHeader(event, 'cache-control', 'no-store')
return { versions }
})

View File

@ -0,0 +1,10 @@
export default defineEventHandler(async event => {
const user = await requireUser(event)
const id = getRouterParam(event, 'id')
const t = Number(getRouterParam(event, 'v'))
if (!isProjectId(id) || !Number.isFinite(t)) throw createError({ statusCode: 404, message: 'Version not found' })
const version = await readVersion(user.id, id, t)
if (!version) throw createError({ statusCode: 404, message: 'Version not found' })
setHeader(event, 'cache-control', 'no-store')
return { version }
})

View File

@ -0,0 +1,15 @@
// Projects deleted in the last 30 days that can still be restored.
export default defineEventHandler(async event => {
const user = await requireUser(event)
const index = await readIndex(user.id)
const cutoff = Date.now() - TRASH_DAYS * 86_400_000
const projects = []
for (const d of index.deleted) {
if (d.at < cutoff) continue
const [latest] = await versionTimes(user.id, d.id)
const v = latest && (await readVersion(user.id, d.id, latest))
if (v) projects.push({ id: d.id, name: v.name, deleted: d.at, v: latest })
}
setHeader(event, 'cache-control', 'no-store')
return { projects }
})

10
server/plugins/backup.ts Normal file
View File

@ -0,0 +1,10 @@
// Production only: a backup of data/ once a day (checked hourly, and a
// minute after start so a restart never skips a day).
import { resolve } from 'node:path'
export default defineNitroPlugin(() => {
if (process.env.NODE_ENV !== 'production') return
const run = () => backupIfDue(resolve('data'), resolve('backups')).catch(e => console.error('[backup] failed', e))
setTimeout(run, 60_000)
setInterval(run, 60 * 60_000)
})

11
server/plugins/storage.ts Normal file
View File

@ -0,0 +1,11 @@
// Mounts the crash-safe file storage for share links and accounts.
import { join, resolve } from 'node:path'
export default defineNitroPlugin(() => {
const root = resolve(process.cwd(), 'data')
const storage = useStorage()
for (const name of ['shares', 'users', 'sessions', 'accounts']) {
storage.mount(name, safeFsDriver({ base: join(root, name) }))
}
void removeStaleTemp(root)
})

75
server/utils/backup.ts Normal file
View File

@ -0,0 +1,75 @@
// Nightly backup of the whole data/ folder into backups/, one gzip file per
// day (one JSON line per stored file). Restore with scripts/restore-backup.mjs.
import { createWriteStream, existsSync } from 'node:fs'
import { mkdir, readdir, readFile, rename, unlink } from 'node:fs/promises'
import { join, relative, sep } from 'node:path'
import { createGzip } from 'node:zlib'
export const backupName = (date: Date) => `logigate-${date.toISOString().slice(0, 10)}.ndjson.gz`
async function files(dir: string, out: string[] = []): Promise<string[]> {
for (const e of await readdir(dir, { withFileTypes: true }).catch(() => [])) {
const p = join(dir, e.name)
if (e.isDirectory()) await files(p, out)
else out.push(p)
}
return out
}
/** Writes today's backup (if missing). Returns its path and how many files it holds. */
export async function writeBackup(dataDir: string, backupDir: string, date = new Date()): Promise<{ file: string; count: number }> {
await mkdir(backupDir, { recursive: true })
const file = join(backupDir, backupName(date))
const tmp = `${file}.tmp`
const gzip = createGzip()
const out = createWriteStream(tmp)
const done = new Promise<void>((resolve, reject) => {
out.on('finish', resolve)
out.on('error', reject)
gzip.on('error', reject)
})
gzip.pipe(out)
const write = (line: unknown) =>
new Promise<void>(resolve => (gzip.write(`${JSON.stringify(line)}\n`) ? resolve() : gzip.once('drain', resolve)))
await write({ logigate: 'backup', version: 1, created: date.toISOString() })
let count = 0
const all = await files(dataDir)
const present = new Set(all)
for (const f of all) {
if (f.includes('.tmp-')) continue
// A .bak only matters when its file is gone (a crash between two renames).
if (f.endsWith('.bak') && present.has(f.slice(0, -4))) continue
const real = f.endsWith('.bak') ? f.slice(0, -4) : f
const data = await readFile(f, 'utf8')
try {
JSON.parse(data)
} catch {
continue // Damaged file: leave it out rather than preserve the damage.
}
await write({ path: relative(dataDir, real).split(sep).join('/'), data })
count++
}
gzip.end()
await done
await rename(tmp, file)
return { file, count }
}
/** Keeps the newest 14 backups plus the first backup of each of the last 12 months. */
export async function pruneBackups(backupDir: string) {
const names = (await readdir(backupDir).catch(() => [])).filter(n => /^logigate-\d{4}-\d{2}-\d{2}\.ndjson\.gz$/.test(n)).sort().reverse()
const keep = new Set(names.slice(0, 14))
const months = new Map<string, string>()
for (const n of names) months.set(n.slice(9, 16), n) // ends on the earliest of each month
for (const n of [...months.values()].slice(0, 12)) keep.add(n)
for (const n of names) if (!keep.has(n)) await unlink(join(backupDir, n)).catch(() => {})
}
export async function backupIfDue(dataDir: string, backupDir: string) {
if (!existsSync(dataDir) || existsSync(join(backupDir, backupName(new Date())))) return
const { file, count } = await writeBackup(dataDir, backupDir)
await pruneBackups(backupDir)
console.log(`[backup] ${count} files to ${file}`)
}

71
server/utils/history.ts Normal file
View File

@ -0,0 +1,71 @@
import type { AccountIndex } from './accounts'
// Savegames: earlier versions of each account project.
//
// Before a save replaces a project, its current state is kept as a version,
// at most one every 15 minutes (always before a restore and on deletion).
// Kept: the newest 30, plus the last version of each day for 90 days.
// Versions of a deleted project stay 30 days, so it can be restored.
import type { Project } from '../../app/lib/logic/types'
export interface Version {
project: Project
/** Revision and save time of the state that was kept. */
rev: number
at: number
name: string
}
const DAY = 86_400_000
export const SNAPSHOT_EVERY = 15 * 60_000
export const KEEP_NEWEST = 30
export const KEEP_DAILY_DAYS = 90
export const TRASH_DAYS = 30
const prefix = (uid: string, pid: string) => `${uid}:h:${pid}:`
/** Which versions (by time) to drop: all but the newest 30 and the last of each day for 90 days. */
export function versionsToDrop(times: number[], now: number): number[] {
const sorted = [...times].sort((a, b) => b - a)
const keep = new Set(sorted.slice(0, KEEP_NEWEST))
const days = new Set<string>()
for (const t of sorted) {
if (now - t > KEEP_DAILY_DAYS * DAY) continue
const day = new Date(t).toISOString().slice(0, 10)
if (!days.has(day)) {
days.add(day)
keep.add(t)
}
}
return sorted.filter(t => !keep.has(t))
}
export async function versionTimes(uid: string, pid: string): Promise<number[]> {
const keys = await accounts().getKeys(prefix(uid, pid))
return keys
.filter(k => k.startsWith(prefix(uid, pid)))
.map(k => Number(k.slice(prefix(uid, pid).length)))
.filter(Number.isFinite)
.sort((a, b) => b - a)
}
export const readVersion = (uid: string, pid: string, t: number) => accounts().getItem<Version>(`${prefix(uid, pid)}${t}`)
/** Keeps the project's current state as a version, unless one was kept less than 15 minutes ago. */
export async function keepVersion(uid: string, pid: string, current: Version, force = false) {
const now = Date.now()
const times = await versionTimes(uid, pid)
if (!force && times[0] !== undefined && now - times[0] < SNAPSHOT_EVERY) return
await accounts().setItem(`${prefix(uid, pid)}${now}`, current)
for (const t of versionsToDrop([now, ...times], now)) await accounts().removeItem(`${prefix(uid, pid)}${t}`)
}
export async function dropVersions(uid: string, pid: string) {
for (const t of await versionTimes(uid, pid)) await accounts().removeItem(`${prefix(uid, pid)}${t}`)
}
/** Deleted projects older than 30 days lose their versions for good. */
export async function emptyOldTrash(uid: string, index: AccountIndex) {
const cutoff = Date.now() - TRASH_DAYS * DAY
for (const d of index.deleted) if (d.at < cutoff) await dropVersions(uid, d.id)
}

121
server/utils/safe-fs.ts Normal file
View File

@ -0,0 +1,121 @@
// File storage that survives power cuts. Every write goes to a temporary
// file, is flushed to disk, and only then replaces the real file; the last
// good version stays next to it as "<file>.bak". Reading falls back to the
// .bak when the file is missing or not valid JSON. Same file format as
// Nitro's built-in "fs" driver, so existing data reads unchanged.
import { existsSync } from 'node:fs'
import { mkdir, open, readdir, readFile, rename, rm, unlink } from 'node:fs/promises'
import { dirname, join, relative, resolve, sep } from 'node:path'
const TMP = '.tmp-'
function validJson(text: string): boolean {
try {
JSON.parse(text)
return true
} catch {
return false
}
}
async function syncDir(dir: string) {
try {
const fh = await open(dir, 'r')
try {
await fh.sync()
} finally {
await fh.close()
}
} catch {
// Not every file system can sync a directory; the file itself is synced.
}
}
/** Writes `data` to `file` so that a crash at any moment leaves either the old or the new content. */
export async function atomicWrite(file: string, data: string) {
const dir = dirname(file)
await mkdir(dir, { recursive: true })
const tmp = `${file}${TMP}${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`
const fh = await open(tmp, 'w')
try {
await fh.writeFile(data, 'utf8')
await fh.sync()
} finally {
await fh.close()
}
// Keep the current version as the backup, but only if it is intact.
const current = await readFile(file, 'utf8').catch(() => null)
if (current !== null && validJson(current)) await rename(file, `${file}.bak`)
await rename(tmp, file)
await syncDir(dir)
}
/** Reads a JSON file, falling back to its .bak when it is missing or damaged. */
export async function safeRead(file: string): Promise<string | null> {
for (const f of [file, `${file}.bak`]) {
const text = await readFile(f, 'utf8').catch(() => null)
if (text !== null && validJson(text)) return text
}
return null
}
async function walk(dir: string, out: string[]) {
const entries = await readdir(dir, { withFileTypes: true }).catch(() => [])
for (const e of entries) {
const p = join(dir, e.name)
if (e.isDirectory()) await walk(p, out)
else out.push(p)
}
}
/** Removes temporary files left behind by writes that a crash interrupted. */
export async function removeStaleTemp(base: string) {
const files: string[] = []
await walk(base, files)
for (const f of files) if (f.includes(TMP)) await unlink(f).catch(() => {})
}
/** Storage driver for Nitro's useStorage(), mounted in server/plugins/storage.ts. */
export function safeFsDriver(opts: { base: string }) {
const base = resolve(opts.base)
const path = (key: string) => {
if (/(^|:)\.\.(:|$)/.test(key)) throw new Error(`Invalid storage key ${JSON.stringify(key)}`)
return join(base, key.replace(/:/g, '/'))
}
return {
name: 'safe-fs',
options: opts,
async hasItem(key: string) {
return (await safeRead(path(key))) !== null
},
getItem(key: string) {
return safeRead(path(key))
},
setItem(key: string, value: string) {
return atomicWrite(path(key), value)
},
async removeItem(key: string) {
const p = path(key)
await unlink(p).catch(() => {})
await unlink(`${p}.bak`).catch(() => {})
await syncDir(dirname(p))
},
async getKeys(prefix?: string) {
// Only walk the folder the prefix points at, not the whole store.
const dir = prefix ? join(base, prefix.replace(/:$/, '').replace(/:/g, '/')) : base
const files: string[] = []
await walk(dir, files)
const keys = new Set<string>()
for (const f of files) {
if (f.includes(TMP)) continue
const real = f.endsWith('.bak') ? f.slice(0, -4) : f
keys.add(relative(base, real).split(sep).join(':'))
}
return [...keys]
},
async clear() {
await rm(base, { recursive: true, force: true })
},
}
}

149
tests/durability.test.ts Normal file
View File

@ -0,0 +1,149 @@
import { execFileSync } from 'node:child_process'
import { existsSync, mkdtempSync, readFileSync, readdirSync, writeFileSync } from 'node:fs'
import { mkdir, readFile, rm, unlink, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { backupName, pruneBackups, writeBackup } from '../server/utils/backup'
import { versionsToDrop } from '../server/utils/history'
import { atomicWrite, removeStaleTemp, safeFsDriver, safeRead } from '../server/utils/safe-fs'
let dir = ''
beforeEach(() => {
dir = mkdtempSync(join(tmpdir(), 'logigate-'))
})
afterEach(async () => {
await rm(dir, { recursive: true, force: true })
})
describe('crash-safe files', () => {
it('keeps the previous good version as .bak', async () => {
const f = join(dir, 'a', 'p.json')
await atomicWrite(f, '{"v":1}')
await atomicWrite(f, '{"v":2}')
expect(await safeRead(f)).toBe('{"v":2}')
expect(await readFile(`${f}.bak`, 'utf8')).toBe('{"v":1}')
})
it('reads the .bak when the file was cut off by a power failure', async () => {
const f = join(dir, 'p.json')
await atomicWrite(f, '{"v":1}')
await atomicWrite(f, '{"v":2}')
await writeFile(f, '{"v":3, "nodes": [{"id": "n1", "ki') // truncated
expect(await safeRead(f)).toBe('{"v":1}')
})
it('reads the .bak when the power went out between the two renames', async () => {
const f = join(dir, 'p.json')
await atomicWrite(f, '{"v":1}')
await atomicWrite(f, '{"v":2}')
await unlink(f)
expect(await safeRead(f)).toBe('{"v":1}')
})
it('ignores a half-written temporary file and cleans it up', async () => {
const f = join(dir, 'p.json')
await atomicWrite(f, '{"v":1}')
await writeFile(`${f}.tmp-123-456-abc`, '{"v":2, "trunc')
expect(await safeRead(f)).toBe('{"v":1}')
await removeStaleTemp(dir)
expect(readdirSync(dir).sort()).toEqual(['p.json'])
})
it('does not overwrite a good .bak with a damaged file', async () => {
const f = join(dir, 'p.json')
await atomicWrite(f, '{"v":1}')
await atomicWrite(f, '{"v":2}')
await writeFile(f, 'garbage')
await atomicWrite(f, '{"v":3}')
expect(await readFile(`${f}.bak`, 'utf8')).toBe('{"v":1}')
expect(await safeRead(f)).toBe('{"v":3}')
})
it('works as a storage driver: keys by prefix, without .bak and temp files', async () => {
const d = safeFsDriver({ base: dir })
await d.setItem('u1:index', '{"projects":[]}')
await d.setItem('u1:p:abc', '{"name":"A"}')
await d.setItem('u1:p:abc', '{"name":"B"}')
await d.setItem('u2:index', '{}')
await writeFile(join(dir, 'u1', 'p', 'abc.tmp-1-2-x'), 'x')
expect((await d.getKeys('u1:')).sort()).toEqual(['u1:index', 'u1:p:abc'])
expect(await d.getItem('u1:p:abc')).toBe('{"name":"B"}')
expect(await d.hasItem('u1:p:nope')).toBe(false)
await d.removeItem('u1:p:abc')
expect(await d.hasItem('u1:p:abc')).toBe(false)
expect(() => d.getItem('u1:..:x')).toThrow()
})
it('reads files written by the old storage the same way', async () => {
await mkdir(join(dir, 'user'), { recursive: true })
writeFileSync(join(dir, 'user', 'abc'), '{"id":"abc","email":"a@b.c"}')
const d = safeFsDriver({ base: dir })
expect(await d.getItem('user:abc')).toBe('{"id":"abc","email":"a@b.c"}')
})
})
describe('savegames', () => {
const DAY = 86_400_000
const now = Date.UTC(2026, 9, 20, 12)
it('keeps the newest 30 versions', () => {
const times = Array.from({ length: 40 }, (_, i) => now - i * 60_000)
const drop = versionsToDrop(times, now)
// All on one day, whose last version is among the newest anyway.
expect(drop).toEqual(times.slice(30))
})
it('keeps the last version of each day for 90 days, nothing older', () => {
const times = [now, ...Array.from({ length: 100 }, (_, i) => now - (i + 1) * DAY - 3_600_000), ...Array.from({ length: 100 }, (_, i) => now - (i + 1) * DAY - 7_200_000)]
const drop = new Set(versionsToDrop(times, now))
const sorted = [...times].sort((a, b) => b - a)
const older = sorted.slice(30).filter(t => !drop.has(t))
// Beyond the newest 30: one per day, none older than 90 days.
expect(new Set(older.map(t => new Date(t).toISOString().slice(0, 10))).size).toBe(older.length)
expect(older.every(t => now - t <= 90 * DAY)).toBe(true)
expect(older.length).toBeGreaterThan(70)
})
})
describe('daily backup', () => {
it('backs up every file and restores it with the script', async () => {
const data = join(dir, 'data')
const backups = join(dir, 'backups')
await atomicWrite(join(data, 'users', 'user', 'u1'), '{"id":"u1"}')
await atomicWrite(join(data, 'accounts', 'u1', 'index'), '{"projects":[]}')
await atomicWrite(join(data, 'accounts', 'u1', 'p', 'abc'), '{"name":"v1"}')
await atomicWrite(join(data, 'accounts', 'u1', 'p', 'abc'), '{"name":"v2"}')
await writeFile(join(data, 'accounts', 'u1', 'broken'), '{"half')
await atomicWrite(join(data, 'shares', 'X'), '{"project":0}')
await atomicWrite(join(data, 'shares', 'X'), '{"project":1}')
await unlink(join(data, 'shares', 'X')) // crash between renames: only X.bak (the previous version) left
const { file, count } = await writeBackup(data, backups, new Date(Date.UTC(2026, 9, 20)))
expect(file.endsWith(backupName(new Date(Date.UTC(2026, 9, 20))))).toBe(true)
expect(count).toBe(4)
const target = join(dir, 'restored')
execFileSync('node', ['scripts/restore-backup.mjs', file, target])
const read = (p: string) => readFileSync(join(target, p), 'utf8')
expect(read('users/user/u1')).toBe('{"id":"u1"}')
expect(read('accounts/u1/p/abc')).toBe('{"name":"v2"}')
expect(read('shares/X')).toBe('{"project":0}')
expect(existsSync(join(target, 'accounts', 'u1', 'broken'))).toBe(false)
})
it('keeps 14 daily backups and one per month', async () => {
await mkdir(dir, { recursive: true })
for (let i = 0; i < 60; i++) {
const d = new Date(Date.UTC(2026, 9, 20) - i * 86_400_000)
writeFileSync(join(dir, backupName(d)), '')
}
await pruneBackups(dir)
const left = readdirSync(dir).sort()
expect(left).toContain(backupName(new Date(Date.UTC(2026, 9, 20))))
expect(left).toContain(backupName(new Date(Date.UTC(2026, 7, 22))))
expect(left).toContain(backupName(new Date(Date.UTC(2026, 8, 1))))
expect(left).toContain(backupName(new Date(Date.UTC(2026, 9, 1))))
expect(left.length).toBe(14 + 3) // 14 newest + first of October, September, August
})
})