A crash-safe storage, savegames and daily backups
- Writes go to a flushed temporary file and are renamed into place; the last intact version stays as .bak and reads fall back to it, so a power cut never leaves a damaged save. Same file format as before. - Savegames per account project (newest 30, one per day for 90 days), kept before restores and deletions; deleted projects restorable for 30 days. Projects dialog: savegames per project and "Recently deleted". - Daily gzip backup of data/ into backups/ (14 newest + monthly), restore script scripts/restore-backup.mjs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
5e17826ba5
commit
08b086a71e
|
|
@ -18,3 +18,4 @@ localhost-key.pem
|
||||||
*.map
|
*.map
|
||||||
*.log
|
*.log
|
||||||
data
|
data
|
||||||
|
backups
|
||||||
|
|
|
||||||
|
|
@ -36,3 +36,4 @@ playwright-report/
|
||||||
|
|
||||||
# Shared projects (server storage)
|
# Shared projects (server storage)
|
||||||
data/
|
data/
|
||||||
|
backups/
|
||||||
|
|
|
||||||
23
README.md
23
README.md
|
|
@ -41,7 +41,26 @@ The editor UI is in [app/components/editor/](app/components/editor/) with state
|
||||||
|
|
||||||
## Accounts
|
## Accounts
|
||||||
|
|
||||||
[server/api/auth/](server/api/auth/) (register, login, logout, me, password, delete) and [server/api/me/projects/](server/api/me/projects/) (list, get, put, delete). Passwords are hashed with scrypt; the session is a random token in an httpOnly, SameSite=Lax cookie, stored server-side as a SHA-256. API calls that change something are refused when their Origin is another site. Storage (Nitro fs): `data/users`, `data/sessions`, `data/accounts/<uid>/` with an index (revision per project, deletions for a year) and one file per project. Each save sends the revision it started from; a mismatch answers 409 and the browser keeps the other version as a copy. Limits: 1000 projects and 100 MB per account, 2 MB per project, login 20 tries per 10 minutes per IP, 10 new accounts per hour per IP.
|
[server/api/auth/](server/api/auth/) (register, login, logout, me, password, delete) and [server/api/me/projects/](server/api/me/projects/) (list, get, put, delete). Passwords are hashed with scrypt; the session is a random token in an httpOnly, SameSite=Lax cookie, stored server-side as a SHA-256. API calls that change something are refused when their Origin is another site. Storage: `data/users`, `data/sessions`, `data/accounts/<uid>/` with an index (revision per project, deletions for a year), one file per project, and savegames in `h/<project>/`. Each save sends the revision it started from; a mismatch answers 409 and the browser keeps the other version as a copy. Limits: 1000 projects and 100 MB per account, 2 MB per project, login 20 tries per 10 minutes per IP, 10 new accounts per hour per IP.
|
||||||
|
|
||||||
|
## Power cuts, savegames, backups
|
||||||
|
|
||||||
|
No database: everything is JSON files under `data/`, which keeps memory use flat. Three layers protect it:
|
||||||
|
|
||||||
|
- **Crash-safe writes** ([server/utils/safe-fs.ts](server/utils/safe-fs.ts), mounted in [server/plugins/storage.ts](server/plugins/storage.ts)). A save writes a temporary file, flushes it to disk, keeps the current intact version as `<file>.bak` and then renames the new file into place. A power cut at any moment leaves the old or the new version; reading falls back to `.bak` when a file is missing or damaged, and leftover temporary files are removed at start.
|
||||||
|
- **Savegames** ([server/utils/history.ts](server/utils/history.ts)). Before a save replaces an account project, its state is kept as a version (at most one per 15 minutes; always before a restore and on deletion). Kept: the newest 30 plus the last of each day for 90 days. Deleted projects stay restorable for 30 days. In the app: Projects dialog, clock icon (restore, or open as copy), and "Recently deleted".
|
||||||
|
- **Daily backup** ([server/utils/backup.ts](server/utils/backup.ts), [server/plugins/backup.ts](server/plugins/backup.ts)). Once a day the production server writes all of `data/` into `backups/logigate-YYYY-MM-DD.ndjson.gz`, leaving out damaged files. Kept: the newest 14 plus the first of each month for a year. `backups/` sits next to `data/` on the same disk, so it protects against damaged files and mistakes, not against losing the disk; copy it elsewhere for that.
|
||||||
|
|
||||||
|
Restore a backup on edgerealm (the current `data/` is moved aside first, so this can be undone):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /mnt/HC_Volume_102040804/srv/logigate
|
||||||
|
docker compose stop frontend
|
||||||
|
docker run --rm -v "$PWD":/app -w /app node:current-slim node scripts/restore-backup.mjs backups/logigate-2026-10-20.ndjson.gz
|
||||||
|
docker compose start frontend
|
||||||
|
```
|
||||||
|
|
||||||
|
In the browser, each localStorage write is atomic, so a power cut there loses at most the last few seconds.
|
||||||
|
|
||||||
## Compatibility
|
## Compatibility
|
||||||
|
|
||||||
|
|
@ -59,4 +78,4 @@ pnpm build # .output/
|
||||||
|
|
||||||
## Deploy
|
## Deploy
|
||||||
|
|
||||||
Same pattern as autismus-spektrum. On edgerealm the repo lives in `/mnt/HC_Volume_102040804/srv/logigate`; run `./deploy.sh` there. It pulls, builds in a one-shot container and recreates the `logigate` container. Share links and accounts live in `data/` next to the repo (mounted into the container), so redeploys keep them; include that folder in backups. Host nginx terminates TLS with the `*.blax.at` wildcard cert and forwards to Traefik, which routes `logicate.blax.at` and `logigate.blax.at` by the labels in [docker-compose.yml](docker-compose.yml).
|
Same pattern as autismus-spektrum. On edgerealm the repo lives in `/mnt/HC_Volume_102040804/srv/logigate`; run `./deploy.sh` there. It pulls, builds in a one-shot container and recreates the `logigate` container. Share links and accounts live in `data/` and the daily backups in `backups/`, both next to the repo and mounted into the container, so redeploys keep them. Host nginx terminates TLS with the `*.blax.at` wildcard cert and forwards to Traefik, which routes `logicate.blax.at` and `logigate.blax.at` by the labels in [docker-compose.yml](docker-compose.yml).
|
||||||
|
|
|
||||||
|
|
@ -31,5 +31,6 @@ onMounted(() => {
|
||||||
<EditorProjectsDialog v-if="ed.dialog.value === 'projects'" />
|
<EditorProjectsDialog v-if="ed.dialog.value === 'projects'" />
|
||||||
<EditorShareDialog v-if="ed.dialog.value === 'share'" />
|
<EditorShareDialog v-if="ed.dialog.value === 'share'" />
|
||||||
<EditorAccountDialog v-if="ed.dialog.value === 'account'" />
|
<EditorAccountDialog v-if="ed.dialog.value === 'account'" />
|
||||||
|
<EditorVersionsDialog v-if="ed.dialog.value === 'versions'" />
|
||||||
</div>
|
</div>
|
||||||
</template>
|
</template>
|
||||||
|
|
|
||||||
|
|
@ -6,6 +6,17 @@ const renameText = ref('')
|
||||||
const confirmDelete = ref<string | null>(null)
|
const confirmDelete = ref<string | null>(null)
|
||||||
|
|
||||||
const close = () => (ed.dialog.value = null)
|
const close = () => (ed.dialog.value = null)
|
||||||
|
|
||||||
|
const showTrash = ref(false)
|
||||||
|
const trash = ref<{ id: string; name: string; deleted: number; v: number }[] | null>(null)
|
||||||
|
async function toggleTrash() {
|
||||||
|
showTrash.value = !showTrash.value
|
||||||
|
if (showTrash.value) trash.value = await ed.listTrash().catch(() => [])
|
||||||
|
}
|
||||||
|
async function restoreDeleted(t: { id: string; v: number }) {
|
||||||
|
await ed.restoreDeleted(t)
|
||||||
|
close()
|
||||||
|
}
|
||||||
const when = (t: number) => new Intl.DateTimeFormat(undefined, { dateStyle: 'medium', timeStyle: 'short' }).format(t)
|
const when = (t: number) => new Intl.DateTimeFormat(undefined, { dateStyle: 'medium', timeStyle: 'short' }).format(t)
|
||||||
|
|
||||||
function open(id: string) {
|
function open(id: string) {
|
||||||
|
|
@ -69,6 +80,14 @@ function onFile(e: Event) {
|
||||||
<template v-else-if="!ed.account.value"> · <Icon name="lucide:laptop" class="size-3" /> this browser only</template>
|
<template v-else-if="!ed.account.value"> · <Icon name="lucide:laptop" class="size-3" /> this browser only</template>
|
||||||
</span>
|
</span>
|
||||||
</button>
|
</button>
|
||||||
|
<button
|
||||||
|
v-if="ed.account.value && m.cloud?.owner === ed.account.value.id"
|
||||||
|
class="icon-btn"
|
||||||
|
title="Savegames: earlier versions"
|
||||||
|
@click="ed.versionsFor.value = m.id; ed.dialog.value = 'versions'"
|
||||||
|
>
|
||||||
|
<Icon name="lucide:history" class="size-3.5" />
|
||||||
|
</button>
|
||||||
<button class="icon-btn" title="Rename" @click="startRename(m.id, m.name)"><Icon name="lucide:pencil" class="size-3.5" /></button>
|
<button class="icon-btn" title="Rename" @click="startRename(m.id, m.name)"><Icon name="lucide:pencil" class="size-3.5" /></button>
|
||||||
<button class="icon-btn" title="Duplicate" @click="ed.duplicateProject(m.id); close()"><Icon name="lucide:copy" class="size-3.5" /></button>
|
<button class="icon-btn" title="Duplicate" @click="ed.duplicateProject(m.id); close()"><Icon name="lucide:copy" class="size-3.5" /></button>
|
||||||
<button v-if="confirmDelete !== m.id" class="icon-btn" title="Delete" @click="confirmDelete = m.id"><Icon name="lucide:trash-2" class="size-3.5" /></button>
|
<button v-if="confirmDelete !== m.id" class="icon-btn" title="Delete" @click="confirmDelete = m.id"><Icon name="lucide:trash-2" class="size-3.5" /></button>
|
||||||
|
|
@ -76,6 +95,26 @@ function onFile(e: Event) {
|
||||||
</li>
|
</li>
|
||||||
</ul>
|
</ul>
|
||||||
|
|
||||||
|
<template v-if="ed.account.value">
|
||||||
|
<button class="section-toggle mt-4" @click="toggleTrash">
|
||||||
|
<Icon :name="showTrash ? 'lucide:chevron-down' : 'lucide:chevron-right'" class="size-4" />
|
||||||
|
Recently deleted
|
||||||
|
</button>
|
||||||
|
<template v-if="showTrash">
|
||||||
|
<p v-if="!trash" class="hint mt-2">Loading…</p>
|
||||||
|
<p v-else-if="!trash.length" class="hint mt-2">Nothing deleted in the last 30 days.</p>
|
||||||
|
<ul v-else class="project-list">
|
||||||
|
<li v-for="t in trash" :key="t.id">
|
||||||
|
<div class="pick">
|
||||||
|
<span class="name">{{ t.name }}</span>
|
||||||
|
<span class="meta">deleted {{ when(t.deleted) }}</span>
|
||||||
|
</div>
|
||||||
|
<button class="btn-sm" @click="restoreDeleted(t)">Restore</button>
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
</template>
|
||||||
|
</template>
|
||||||
|
|
||||||
<div class="mt-4 flex flex-wrap gap-2">
|
<div class="mt-4 flex flex-wrap gap-2">
|
||||||
<button class="btn" @click="ed.newProject(false); close()"><Icon name="lucide:file" class="size-4" /> New (NAND only)</button>
|
<button class="btn" @click="ed.newProject(false); close()"><Icon name="lucide:file" class="size-4" /> New (NAND only)</button>
|
||||||
<button class="btn" @click="ed.newProject(true); close()"><Icon name="lucide:file-plus" class="size-4" /> New with examples</button>
|
<button class="btn" @click="ed.newProject(true); close()"><Icon name="lucide:file-plus" class="size-4" /> New with examples</button>
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,66 @@
|
||||||
|
<script setup lang="ts">
|
||||||
|
const ed = useEditor()
|
||||||
|
const close = () => {
|
||||||
|
ed.dialog.value = 'projects'
|
||||||
|
ed.versionsFor.value = null
|
||||||
|
}
|
||||||
|
|
||||||
|
const id = computed(() => ed.versionsFor.value)
|
||||||
|
const name = computed(() => ed.projects.value.find(m => m.id === id.value)?.name ?? '')
|
||||||
|
const versions = ref<{ v: number; at: number; rev: number; name: string; size: number }[] | null>(null)
|
||||||
|
const error = ref('')
|
||||||
|
const busy = ref<number | null>(null)
|
||||||
|
|
||||||
|
const when = (t: number) => new Intl.DateTimeFormat(undefined, { dateStyle: 'medium', timeStyle: 'short' }).format(t)
|
||||||
|
const kb = (n: number) => `${Math.max(1, Math.round(n / 1024))} KB`
|
||||||
|
|
||||||
|
onMounted(async () => {
|
||||||
|
if (!id.value) return
|
||||||
|
try {
|
||||||
|
versions.value = await ed.listVersions(id.value)
|
||||||
|
} catch (e) {
|
||||||
|
error.value = (e as Error).message
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
async function restore(v: number, asCopy: boolean) {
|
||||||
|
if (!id.value) return
|
||||||
|
busy.value = v
|
||||||
|
try {
|
||||||
|
await ed.restoreVersion(id.value, v, asCopy)
|
||||||
|
ed.dialog.value = null
|
||||||
|
} catch (e) {
|
||||||
|
error.value = (e as Error).message
|
||||||
|
} finally {
|
||||||
|
busy.value = null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<template>
|
||||||
|
<div class="modal-backdrop" @click.self="close">
|
||||||
|
<div class="modal" role="dialog" aria-labelledby="versions-title">
|
||||||
|
<div class="flex items-start justify-between gap-4">
|
||||||
|
<h2 id="versions-title">Savegames of "{{ name }}"</h2>
|
||||||
|
<button class="tb-btn" title="Back" @click="close"><Icon name="lucide:x" class="size-4" /></button>
|
||||||
|
</div>
|
||||||
|
<p class="hint mt-1">
|
||||||
|
Kept automatically in your account while you work: a version at most every 15 minutes, the newest 30 plus one per day for 90 days.
|
||||||
|
Restoring keeps the current state as a savegame too, so nothing is lost.
|
||||||
|
</p>
|
||||||
|
<p v-if="error" class="result fail"><Icon name="lucide:circle-x" class="size-4 shrink-0" /> {{ error }}</p>
|
||||||
|
<p v-else-if="!versions" class="hint">Loading…</p>
|
||||||
|
<p v-else-if="!versions.length" class="hint">No savegames yet. The first one is kept the next time this project changes, 15 minutes after the last save.</p>
|
||||||
|
<ul v-else class="project-list">
|
||||||
|
<li v-for="v in versions" :key="v.v">
|
||||||
|
<div class="pick">
|
||||||
|
<span class="name">{{ when(v.at) }}</span>
|
||||||
|
<span class="meta">{{ v.name }} · revision {{ v.rev }} · {{ kb(v.size) }}</span>
|
||||||
|
</div>
|
||||||
|
<button class="btn-sm" :disabled="busy !== null" title="Replace the project with this version" @click="restore(v.v, false)">Restore</button>
|
||||||
|
<button class="btn-sm" :disabled="busy !== null" title="Open this version as a separate project" @click="restore(v.v, true)">As copy</button>
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
@ -105,7 +105,9 @@ function create() {
|
||||||
const notice = ref<{ text: string; kind: 'ok' | 'error' } | null>(null)
|
const notice = ref<{ text: string; kind: 'ok' | 'error' } | null>(null)
|
||||||
const labelFocus = ref(0)
|
const labelFocus = ref(0)
|
||||||
const showHelp = ref(false)
|
const showHelp = ref(false)
|
||||||
const dialog = ref<null | 'projects' | 'share' | 'account'>(null)
|
const dialog = ref<null | 'projects' | 'share' | 'account' | 'versions'>(null)
|
||||||
|
/** Project whose savegames the versions dialog shows. */
|
||||||
|
const versionsFor = ref<string | null>(null)
|
||||||
|
|
||||||
let noticeTimer: ReturnType<typeof setTimeout> | undefined
|
let noticeTimer: ReturnType<typeof setTimeout> | undefined
|
||||||
function notify(text: string, kind: 'ok' | 'error' = 'ok') {
|
function notify(text: string, kind: 'ok' | 'error' = 'ok') {
|
||||||
|
|
@ -855,14 +857,14 @@ function create() {
|
||||||
async function upload(m: ProjectMeta, uid: string) {
|
async function upload(m: ProjectMeta, uid: string) {
|
||||||
const p = localProject(m)
|
const p = localProject(m)
|
||||||
if (!p) return
|
if (!p) return
|
||||||
const body = { project: p, baseRev: m.cloud?.owner === uid ? m.cloud.rev : 0, link: m.link }
|
const body = { project: p, baseRev: m.cloud?.owner === uid ? m.cloud.rev : 0, link: m.link, snapshot: m.snapshot === true }
|
||||||
let res = await authCall<{ meta?: CloudMeta; conflict?: boolean; status: number }>('PUT', `/api/me/projects/${m.id}`, body)
|
let res = await authCall<{ meta?: CloudMeta; conflict?: boolean; status: number }>('PUT', `/api/me/projects/${m.id}`, body)
|
||||||
if (res.conflict) {
|
if (res.conflict) {
|
||||||
await keepRemoteCopy(m.id)
|
await keepRemoteCopy(m.id)
|
||||||
res = await authCall('PUT', `/api/me/projects/${m.id}`, { ...body, force: true })
|
res = await authCall('PUT', `/api/me/projects/${m.id}`, { ...body, force: true })
|
||||||
notify(`"${m.name}" was also changed on another device. That version is kept as a copy.`)
|
notify(`"${m.name}" was also changed on another device. That version is kept as a copy.`)
|
||||||
}
|
}
|
||||||
if (res.meta) updateMeta(m.id, { cloud: { owner: uid, rev: res.meta.rev }, dirty: false })
|
if (res.meta) updateMeta(m.id, { cloud: { owner: uid, rev: res.meta.rev }, dirty: false, snapshot: undefined })
|
||||||
}
|
}
|
||||||
|
|
||||||
let syncingAccount = false
|
let syncingAccount = false
|
||||||
|
|
@ -999,6 +1001,64 @@ function create() {
|
||||||
notify('Account deleted. Your projects are still in this browser.')
|
notify('Account deleted. Your projects are still in this browser.')
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── Savegames ─────────────────────────────────────────────────────
|
||||||
|
interface VersionInfo {
|
||||||
|
v: number
|
||||||
|
at: number
|
||||||
|
rev: number
|
||||||
|
name: string
|
||||||
|
size: number
|
||||||
|
}
|
||||||
|
|
||||||
|
async function listVersions(projectId: string): Promise<VersionInfo[]> {
|
||||||
|
const res = await authCall<{ versions: VersionInfo[]; status: number }>('GET', `/api/me/projects/${projectId}/versions`)
|
||||||
|
return res.versions ?? []
|
||||||
|
}
|
||||||
|
|
||||||
|
async function fetchVersion(projectId: string, v: number): Promise<Project> {
|
||||||
|
const res = await authCall<{ version?: { project: unknown }; status: number }>('GET', `/api/me/projects/${projectId}/versions/${v}`)
|
||||||
|
if (!res.version) throw new Error('Version not found')
|
||||||
|
return toProject(res.version.project)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Brings back a savegame: as a new project, or in place of the current
|
||||||
|
* state (which the server keeps as a savegame first, so this can be undone).
|
||||||
|
*/
|
||||||
|
async function restoreVersion(projectId: string, v: number, asCopy: boolean) {
|
||||||
|
if (projectId === meta.value.id && saveTimer) persistNow()
|
||||||
|
const p = await fetchVersion(projectId, v)
|
||||||
|
if (asCopy) {
|
||||||
|
p.name = `${p.name} (${new Date(v).toLocaleDateString()})`
|
||||||
|
activate(addProject(p), p)
|
||||||
|
notify(`Opened the saved version as "${p.name}"`)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const m = listProjects().find(x => x.id === projectId)
|
||||||
|
if (!m) return
|
||||||
|
const next: ProjectMeta = { ...m, name: p.name, updated: Date.now(), dirty: true, snapshot: true }
|
||||||
|
saveProject(next, p)
|
||||||
|
activate(next, p)
|
||||||
|
scheduleCloud()
|
||||||
|
notify('Restored. The state before is kept as a savegame too.')
|
||||||
|
}
|
||||||
|
|
||||||
|
async function listTrash(): Promise<{ id: string; name: string; deleted: number; v: number }[]> {
|
||||||
|
const res = await authCall<{ projects: { id: string; name: string; deleted: number; v: number }[]; status: number }>('GET', '/api/me/trash')
|
||||||
|
return res.projects ?? []
|
||||||
|
}
|
||||||
|
|
||||||
|
async function restoreDeleted(item: { id: string; v: number }) {
|
||||||
|
const p = await fetchVersion(item.id, item.v)
|
||||||
|
persistNow()
|
||||||
|
setPendingDeletes(pendingDeletes().filter(d => d.id !== item.id))
|
||||||
|
const m: ProjectMeta = { id: item.id, name: p.name, updated: Date.now(), dirty: true }
|
||||||
|
saveProject(m, p)
|
||||||
|
activate(m, p)
|
||||||
|
scheduleCloud()
|
||||||
|
notify(`"${p.name}" is back`)
|
||||||
|
}
|
||||||
|
|
||||||
/** Checks the session with the server, then syncs; again on focus and every minute. */
|
/** Checks the session with the server, then syncs; again on focus and every minute. */
|
||||||
async function startAccount() {
|
async function startAccount() {
|
||||||
try {
|
try {
|
||||||
|
|
@ -1114,6 +1174,11 @@ function create() {
|
||||||
changePassword,
|
changePassword,
|
||||||
deleteAccount,
|
deleteAccount,
|
||||||
startAccount,
|
startAccount,
|
||||||
|
versionsFor,
|
||||||
|
listVersions,
|
||||||
|
restoreVersion,
|
||||||
|
listTrash,
|
||||||
|
restoreDeleted,
|
||||||
copyText,
|
copyText,
|
||||||
shape,
|
shape,
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -21,6 +21,8 @@ export interface ProjectMeta {
|
||||||
dirty?: boolean
|
dirty?: boolean
|
||||||
/** The example project a new browser starts with, never touched yet. */
|
/** The example project a new browser starts with, never touched yet. */
|
||||||
starter?: boolean
|
starter?: boolean
|
||||||
|
/** Ask the server to keep the state before the next upload as a savegame (after restoring an old version). */
|
||||||
|
snapshot?: boolean
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface AccountUser {
|
export interface AccountUser {
|
||||||
|
|
|
||||||
|
|
@ -68,7 +68,7 @@ if [ ! -f .env ]; then
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "==> Preparing docker data directories..."
|
echo "==> Preparing docker data directories..."
|
||||||
mkdir -p docker-data/node_modules data/shares
|
mkdir -p docker-data/node_modules data/shares backups
|
||||||
|
|
||||||
if [ "$(id -u)" -eq 0 ]; then
|
if [ "$(id -u)" -eq 0 ]; then
|
||||||
echo "==> Fixing ownership on writable directories (root mode)..."
|
echo "==> Fixing ownership on writable directories (root mode)..."
|
||||||
|
|
|
||||||
|
|
@ -27,8 +27,10 @@ services:
|
||||||
- .env
|
- .env
|
||||||
volumes:
|
volumes:
|
||||||
- ./.output:/app/.output:ro
|
- ./.output:/app/.output:ro
|
||||||
# Projects shared by link (one JSON file each). Survives redeploys.
|
# Share links and accounts (JSON files). Survives redeploys.
|
||||||
- ./data:/app/data
|
- ./data:/app/data
|
||||||
|
# Daily backups of data/, see scripts/restore-backup.mjs.
|
||||||
|
- ./backups:/app/backups
|
||||||
labels:
|
labels:
|
||||||
traefik.enable: "true"
|
traefik.enable: "true"
|
||||||
traefik.docker.network: "web"
|
traefik.docker.network: "web"
|
||||||
|
|
|
||||||
|
|
@ -28,15 +28,8 @@ export default defineNuxtConfig({
|
||||||
clientBundle: { scan: true },
|
clientBundle: { scan: true },
|
||||||
},
|
},
|
||||||
|
|
||||||
// Share links and accounts live as JSON files in data/ (a volume in production).
|
// Share links and accounts live as JSON files in data/ (a volume in
|
||||||
nitro: {
|
// production), written crash-safe; see server/plugins/storage.ts.
|
||||||
storage: {
|
|
||||||
shares: { driver: 'fs', base: './data/shares' },
|
|
||||||
users: { driver: 'fs', base: './data/users' },
|
|
||||||
sessions: { driver: 'fs', base: './data/sessions' },
|
|
||||||
accounts: { driver: 'fs', base: './data/accounts' },
|
|
||||||
},
|
|
||||||
},
|
|
||||||
|
|
||||||
runtimeConfig: {
|
runtimeConfig: {
|
||||||
public: {
|
public: {
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,47 @@
|
||||||
|
#!/usr/bin/env node
|
||||||
|
// Restores data/ from a backup made by server/utils/backup.ts.
|
||||||
|
//
|
||||||
|
// node scripts/restore-backup.mjs backups/logigate-2026-10-20.ndjson.gz [target-dir]
|
||||||
|
//
|
||||||
|
// The target (default: data) is moved aside to data.before-restore-<time>
|
||||||
|
// first, so a restore can itself be undone. Stop the container before
|
||||||
|
// restoring and start it again afterwards.
|
||||||
|
import { createReadStream, existsSync } from 'node:fs'
|
||||||
|
import { mkdir, open, rename } from 'node:fs/promises'
|
||||||
|
import { dirname, join, resolve } from 'node:path'
|
||||||
|
import { createInterface } from 'node:readline'
|
||||||
|
import { createGunzip } from 'node:zlib'
|
||||||
|
|
||||||
|
const [file, target = 'data'] = process.argv.slice(2)
|
||||||
|
if (!file) {
|
||||||
|
console.error('Usage: node scripts/restore-backup.mjs <backup.ndjson.gz> [target-dir]')
|
||||||
|
process.exit(1)
|
||||||
|
}
|
||||||
|
const dir = resolve(target)
|
||||||
|
if (existsSync(dir)) {
|
||||||
|
const aside = `${dir}.before-restore-${new Date().toISOString().replace(/[:.]/g, '-')}`
|
||||||
|
await rename(dir, aside)
|
||||||
|
console.log(`Moved the current ${target} to ${aside}`)
|
||||||
|
}
|
||||||
|
|
||||||
|
const lines = createInterface({ input: createReadStream(file).pipe(createGunzip()), crlfDelay: Infinity })
|
||||||
|
let count = 0
|
||||||
|
let header = null
|
||||||
|
for await (const line of lines) {
|
||||||
|
if (!line.trim()) continue
|
||||||
|
const entry = JSON.parse(line)
|
||||||
|
if (!header) {
|
||||||
|
header = entry
|
||||||
|
if (entry.logigate !== 'backup') throw new Error('Not a Logigate backup')
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if (typeof entry.path !== 'string' || entry.path.split('/').includes('..')) throw new Error(`Bad path in backup: ${entry.path}`)
|
||||||
|
const out = join(dir, entry.path)
|
||||||
|
await mkdir(dirname(out), { recursive: true })
|
||||||
|
const fh = await open(out, 'w')
|
||||||
|
await fh.writeFile(entry.data, 'utf8')
|
||||||
|
await fh.sync()
|
||||||
|
await fh.close()
|
||||||
|
count++
|
||||||
|
}
|
||||||
|
console.log(`Restored ${count} files from ${file} (made ${header?.created})`)
|
||||||
|
|
@ -4,12 +4,17 @@ export default defineEventHandler(async event => {
|
||||||
if (!isProjectId(id)) throw createError({ statusCode: 404, message: 'Project not found' })
|
if (!isProjectId(id)) throw createError({ statusCode: 404, message: 'Project not found' })
|
||||||
return withLock(user.id, async () => {
|
return withLock(user.id, async () => {
|
||||||
const index = await readIndex(user.id)
|
const index = await readIndex(user.id)
|
||||||
|
// The last state goes to the trash (kept as a version for 30 days).
|
||||||
|
const meta = index.projects.find(p => p.id === id)
|
||||||
|
const last = meta && (await readProject(user.id, id))
|
||||||
|
if (meta && last) await keepVersion(user.id, id, { project: last, rev: meta.rev, at: meta.updated, name: meta.name }, true)
|
||||||
index.projects = index.projects.filter(p => p.id !== id)
|
index.projects = index.projects.filter(p => p.id !== id)
|
||||||
// Remember the deletion for a year, so other devices drop their copy too.
|
// Remember the deletion for a year, so other devices drop their copy too.
|
||||||
const yearAgo = Date.now() - 365 * 86_400_000
|
const yearAgo = Date.now() - 365 * 86_400_000
|
||||||
index.deleted = [{ id, at: Date.now() }, ...index.deleted.filter(d => d.id !== id && d.at > yearAgo)].slice(0, 2000)
|
index.deleted = [{ id, at: Date.now() }, ...index.deleted.filter(d => d.id !== id && d.at > yearAgo)].slice(0, 2000)
|
||||||
await accounts().removeItem(projectKey(user.id, id))
|
await accounts().removeItem(projectKey(user.id, id))
|
||||||
await writeIndex(user.id, index)
|
await writeIndex(user.id, index)
|
||||||
|
await emptyOldTrash(user.id, index)
|
||||||
return { ok: true }
|
return { ok: true }
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
import { toProject } from '../../../../app/lib/logic/project'
|
import { toProject } from '../../../../../app/lib/logic/project'
|
||||||
import type { CloudMeta } from '../../../utils/accounts'
|
import type { CloudMeta } from '../../../../utils/accounts'
|
||||||
// Save a project to the account. `baseRev` is the revision the device last
|
// Save a project to the account. `baseRev` is the revision the device last
|
||||||
// had; if another device saved in between, this answers 409 with the newer
|
// had; if another device saved in between, this answers 409 with the newer
|
||||||
// version's meta, unless `force` is set.
|
// version's meta, unless `force` is set.
|
||||||
|
|
@ -10,7 +10,7 @@ export default defineEventHandler(async event => {
|
||||||
if (!isProjectId(id)) throw createError({ statusCode: 400, message: 'Bad project id' })
|
if (!isProjectId(id)) throw createError({ statusCode: 400, message: 'Bad project id' })
|
||||||
const raw = await readRawBody(event, 'utf8')
|
const raw = await readRawBody(event, 'utf8')
|
||||||
if (!raw || raw.length > 2_100_000) throw createError({ statusCode: 413, message: 'Project too large' })
|
if (!raw || raw.length > 2_100_000) throw createError({ statusCode: 413, message: 'Project too large' })
|
||||||
let body: { project?: unknown; baseRev?: unknown; force?: unknown; link?: unknown }
|
let body: { project?: unknown; baseRev?: unknown; force?: unknown; link?: unknown; snapshot?: unknown }
|
||||||
try {
|
try {
|
||||||
body = JSON.parse(raw)
|
body = JSON.parse(raw)
|
||||||
} catch {
|
} catch {
|
||||||
|
|
@ -39,6 +39,11 @@ export default defineEventHandler(async event => {
|
||||||
const meta: CloudMeta = { id, name: project.name, updated: Date.now(), rev: (existing?.rev ?? 0) + 1, size }
|
const meta: CloudMeta = { id, name: project.name, updated: Date.now(), rev: (existing?.rev ?? 0) + 1, size }
|
||||||
if (cleanLink) meta.link = cleanLink
|
if (cleanLink) meta.link = cleanLink
|
||||||
else if (existing?.link && body.link === undefined) meta.link = existing.link
|
else if (existing?.link && body.link === undefined) meta.link = existing.link
|
||||||
|
// Savegame: keep the state this save replaces.
|
||||||
|
const previous = existing && (await readProject(user.id, id))
|
||||||
|
if (existing && previous) {
|
||||||
|
await keepVersion(user.id, id, { project: previous, rev: existing.rev, at: existing.updated, name: existing.name }, body.snapshot === true)
|
||||||
|
}
|
||||||
await accounts().setItem(projectKey(user.id, id), project)
|
await accounts().setItem(projectKey(user.id, id), project)
|
||||||
index.projects = [meta, ...index.projects.filter(p => p.id !== id)]
|
index.projects = [meta, ...index.projects.filter(p => p.id !== id)]
|
||||||
index.deleted = index.deleted.filter(d => d.id !== id)
|
index.deleted = index.deleted.filter(d => d.id !== id)
|
||||||
|
|
@ -0,0 +1,13 @@
|
||||||
|
// Savegames of one project, newest first (also works for a deleted project).
|
||||||
|
export default defineEventHandler(async event => {
|
||||||
|
const user = await requireUser(event)
|
||||||
|
const id = getRouterParam(event, 'id')
|
||||||
|
if (!isProjectId(id)) throw createError({ statusCode: 404, message: 'Project not found' })
|
||||||
|
const versions = []
|
||||||
|
for (const t of await versionTimes(user.id, id)) {
|
||||||
|
const v = await readVersion(user.id, id, t)
|
||||||
|
if (v) versions.push({ v: t, at: v.at, rev: v.rev, name: v.name, size: JSON.stringify(v.project).length })
|
||||||
|
}
|
||||||
|
setHeader(event, 'cache-control', 'no-store')
|
||||||
|
return { versions }
|
||||||
|
})
|
||||||
|
|
@ -0,0 +1,10 @@
|
||||||
|
export default defineEventHandler(async event => {
|
||||||
|
const user = await requireUser(event)
|
||||||
|
const id = getRouterParam(event, 'id')
|
||||||
|
const t = Number(getRouterParam(event, 'v'))
|
||||||
|
if (!isProjectId(id) || !Number.isFinite(t)) throw createError({ statusCode: 404, message: 'Version not found' })
|
||||||
|
const version = await readVersion(user.id, id, t)
|
||||||
|
if (!version) throw createError({ statusCode: 404, message: 'Version not found' })
|
||||||
|
setHeader(event, 'cache-control', 'no-store')
|
||||||
|
return { version }
|
||||||
|
})
|
||||||
|
|
@ -0,0 +1,15 @@
|
||||||
|
// Projects deleted in the last 30 days that can still be restored.
|
||||||
|
export default defineEventHandler(async event => {
|
||||||
|
const user = await requireUser(event)
|
||||||
|
const index = await readIndex(user.id)
|
||||||
|
const cutoff = Date.now() - TRASH_DAYS * 86_400_000
|
||||||
|
const projects = []
|
||||||
|
for (const d of index.deleted) {
|
||||||
|
if (d.at < cutoff) continue
|
||||||
|
const [latest] = await versionTimes(user.id, d.id)
|
||||||
|
const v = latest && (await readVersion(user.id, d.id, latest))
|
||||||
|
if (v) projects.push({ id: d.id, name: v.name, deleted: d.at, v: latest })
|
||||||
|
}
|
||||||
|
setHeader(event, 'cache-control', 'no-store')
|
||||||
|
return { projects }
|
||||||
|
})
|
||||||
|
|
@ -0,0 +1,10 @@
|
||||||
|
// Production only: a backup of data/ once a day (checked hourly, and a
|
||||||
|
// minute after start so a restart never skips a day).
|
||||||
|
import { resolve } from 'node:path'
|
||||||
|
|
||||||
|
export default defineNitroPlugin(() => {
|
||||||
|
if (process.env.NODE_ENV !== 'production') return
|
||||||
|
const run = () => backupIfDue(resolve('data'), resolve('backups')).catch(e => console.error('[backup] failed', e))
|
||||||
|
setTimeout(run, 60_000)
|
||||||
|
setInterval(run, 60 * 60_000)
|
||||||
|
})
|
||||||
|
|
@ -0,0 +1,11 @@
|
||||||
|
// Mounts the crash-safe file storage for share links and accounts.
|
||||||
|
import { join, resolve } from 'node:path'
|
||||||
|
|
||||||
|
export default defineNitroPlugin(() => {
|
||||||
|
const root = resolve(process.cwd(), 'data')
|
||||||
|
const storage = useStorage()
|
||||||
|
for (const name of ['shares', 'users', 'sessions', 'accounts']) {
|
||||||
|
storage.mount(name, safeFsDriver({ base: join(root, name) }))
|
||||||
|
}
|
||||||
|
void removeStaleTemp(root)
|
||||||
|
})
|
||||||
|
|
@ -0,0 +1,75 @@
|
||||||
|
// Nightly backup of the whole data/ folder into backups/, one gzip file per
|
||||||
|
// day (one JSON line per stored file). Restore with scripts/restore-backup.mjs.
|
||||||
|
|
||||||
|
import { createWriteStream, existsSync } from 'node:fs'
|
||||||
|
import { mkdir, readdir, readFile, rename, unlink } from 'node:fs/promises'
|
||||||
|
import { join, relative, sep } from 'node:path'
|
||||||
|
import { createGzip } from 'node:zlib'
|
||||||
|
|
||||||
|
export const backupName = (date: Date) => `logigate-${date.toISOString().slice(0, 10)}.ndjson.gz`
|
||||||
|
|
||||||
|
async function files(dir: string, out: string[] = []): Promise<string[]> {
|
||||||
|
for (const e of await readdir(dir, { withFileTypes: true }).catch(() => [])) {
|
||||||
|
const p = join(dir, e.name)
|
||||||
|
if (e.isDirectory()) await files(p, out)
|
||||||
|
else out.push(p)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Writes today's backup (if missing). Returns its path and how many files it holds. */
|
||||||
|
export async function writeBackup(dataDir: string, backupDir: string, date = new Date()): Promise<{ file: string; count: number }> {
|
||||||
|
await mkdir(backupDir, { recursive: true })
|
||||||
|
const file = join(backupDir, backupName(date))
|
||||||
|
const tmp = `${file}.tmp`
|
||||||
|
const gzip = createGzip()
|
||||||
|
const out = createWriteStream(tmp)
|
||||||
|
const done = new Promise<void>((resolve, reject) => {
|
||||||
|
out.on('finish', resolve)
|
||||||
|
out.on('error', reject)
|
||||||
|
gzip.on('error', reject)
|
||||||
|
})
|
||||||
|
gzip.pipe(out)
|
||||||
|
const write = (line: unknown) =>
|
||||||
|
new Promise<void>(resolve => (gzip.write(`${JSON.stringify(line)}\n`) ? resolve() : gzip.once('drain', resolve)))
|
||||||
|
|
||||||
|
await write({ logigate: 'backup', version: 1, created: date.toISOString() })
|
||||||
|
let count = 0
|
||||||
|
const all = await files(dataDir)
|
||||||
|
const present = new Set(all)
|
||||||
|
for (const f of all) {
|
||||||
|
if (f.includes('.tmp-')) continue
|
||||||
|
// A .bak only matters when its file is gone (a crash between two renames).
|
||||||
|
if (f.endsWith('.bak') && present.has(f.slice(0, -4))) continue
|
||||||
|
const real = f.endsWith('.bak') ? f.slice(0, -4) : f
|
||||||
|
const data = await readFile(f, 'utf8')
|
||||||
|
try {
|
||||||
|
JSON.parse(data)
|
||||||
|
} catch {
|
||||||
|
continue // Damaged file: leave it out rather than preserve the damage.
|
||||||
|
}
|
||||||
|
await write({ path: relative(dataDir, real).split(sep).join('/'), data })
|
||||||
|
count++
|
||||||
|
}
|
||||||
|
gzip.end()
|
||||||
|
await done
|
||||||
|
await rename(tmp, file)
|
||||||
|
return { file, count }
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Keeps the newest 14 backups plus the first backup of each of the last 12 months. */
|
||||||
|
export async function pruneBackups(backupDir: string) {
|
||||||
|
const names = (await readdir(backupDir).catch(() => [])).filter(n => /^logigate-\d{4}-\d{2}-\d{2}\.ndjson\.gz$/.test(n)).sort().reverse()
|
||||||
|
const keep = new Set(names.slice(0, 14))
|
||||||
|
const months = new Map<string, string>()
|
||||||
|
for (const n of names) months.set(n.slice(9, 16), n) // ends on the earliest of each month
|
||||||
|
for (const n of [...months.values()].slice(0, 12)) keep.add(n)
|
||||||
|
for (const n of names) if (!keep.has(n)) await unlink(join(backupDir, n)).catch(() => {})
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function backupIfDue(dataDir: string, backupDir: string) {
|
||||||
|
if (!existsSync(dataDir) || existsSync(join(backupDir, backupName(new Date())))) return
|
||||||
|
const { file, count } = await writeBackup(dataDir, backupDir)
|
||||||
|
await pruneBackups(backupDir)
|
||||||
|
console.log(`[backup] ${count} files to ${file}`)
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,71 @@
|
||||||
|
import type { AccountIndex } from './accounts'
|
||||||
|
// Savegames: earlier versions of each account project.
|
||||||
|
//
|
||||||
|
// Before a save replaces a project, its current state is kept as a version,
|
||||||
|
// at most one every 15 minutes (always before a restore and on deletion).
|
||||||
|
// Kept: the newest 30, plus the last version of each day for 90 days.
|
||||||
|
// Versions of a deleted project stay 30 days, so it can be restored.
|
||||||
|
|
||||||
|
import type { Project } from '../../app/lib/logic/types'
|
||||||
|
|
||||||
|
export interface Version {
|
||||||
|
project: Project
|
||||||
|
/** Revision and save time of the state that was kept. */
|
||||||
|
rev: number
|
||||||
|
at: number
|
||||||
|
name: string
|
||||||
|
}
|
||||||
|
|
||||||
|
const DAY = 86_400_000
|
||||||
|
export const SNAPSHOT_EVERY = 15 * 60_000
|
||||||
|
export const KEEP_NEWEST = 30
|
||||||
|
export const KEEP_DAILY_DAYS = 90
|
||||||
|
export const TRASH_DAYS = 30
|
||||||
|
|
||||||
|
const prefix = (uid: string, pid: string) => `${uid}:h:${pid}:`
|
||||||
|
|
||||||
|
/** Which versions (by time) to drop: all but the newest 30 and the last of each day for 90 days. */
|
||||||
|
export function versionsToDrop(times: number[], now: number): number[] {
|
||||||
|
const sorted = [...times].sort((a, b) => b - a)
|
||||||
|
const keep = new Set(sorted.slice(0, KEEP_NEWEST))
|
||||||
|
const days = new Set<string>()
|
||||||
|
for (const t of sorted) {
|
||||||
|
if (now - t > KEEP_DAILY_DAYS * DAY) continue
|
||||||
|
const day = new Date(t).toISOString().slice(0, 10)
|
||||||
|
if (!days.has(day)) {
|
||||||
|
days.add(day)
|
||||||
|
keep.add(t)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return sorted.filter(t => !keep.has(t))
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function versionTimes(uid: string, pid: string): Promise<number[]> {
|
||||||
|
const keys = await accounts().getKeys(prefix(uid, pid))
|
||||||
|
return keys
|
||||||
|
.filter(k => k.startsWith(prefix(uid, pid)))
|
||||||
|
.map(k => Number(k.slice(prefix(uid, pid).length)))
|
||||||
|
.filter(Number.isFinite)
|
||||||
|
.sort((a, b) => b - a)
|
||||||
|
}
|
||||||
|
|
||||||
|
export const readVersion = (uid: string, pid: string, t: number) => accounts().getItem<Version>(`${prefix(uid, pid)}${t}`)
|
||||||
|
|
||||||
|
/** Keeps the project's current state as a version, unless one was kept less than 15 minutes ago. */
|
||||||
|
export async function keepVersion(uid: string, pid: string, current: Version, force = false) {
|
||||||
|
const now = Date.now()
|
||||||
|
const times = await versionTimes(uid, pid)
|
||||||
|
if (!force && times[0] !== undefined && now - times[0] < SNAPSHOT_EVERY) return
|
||||||
|
await accounts().setItem(`${prefix(uid, pid)}${now}`, current)
|
||||||
|
for (const t of versionsToDrop([now, ...times], now)) await accounts().removeItem(`${prefix(uid, pid)}${t}`)
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function dropVersions(uid: string, pid: string) {
|
||||||
|
for (const t of await versionTimes(uid, pid)) await accounts().removeItem(`${prefix(uid, pid)}${t}`)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Deleted projects older than 30 days lose their versions for good. */
|
||||||
|
export async function emptyOldTrash(uid: string, index: AccountIndex) {
|
||||||
|
const cutoff = Date.now() - TRASH_DAYS * DAY
|
||||||
|
for (const d of index.deleted) if (d.at < cutoff) await dropVersions(uid, d.id)
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,121 @@
|
||||||
|
// File storage that survives power cuts. Every write goes to a temporary
|
||||||
|
// file, is flushed to disk, and only then replaces the real file; the last
|
||||||
|
// good version stays next to it as "<file>.bak". Reading falls back to the
|
||||||
|
// .bak when the file is missing or not valid JSON. Same file format as
|
||||||
|
// Nitro's built-in "fs" driver, so existing data reads unchanged.
|
||||||
|
|
||||||
|
import { existsSync } from 'node:fs'
|
||||||
|
import { mkdir, open, readdir, readFile, rename, rm, unlink } from 'node:fs/promises'
|
||||||
|
import { dirname, join, relative, resolve, sep } from 'node:path'
|
||||||
|
|
||||||
|
const TMP = '.tmp-'
|
||||||
|
|
||||||
|
function validJson(text: string): boolean {
|
||||||
|
try {
|
||||||
|
JSON.parse(text)
|
||||||
|
return true
|
||||||
|
} catch {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function syncDir(dir: string) {
|
||||||
|
try {
|
||||||
|
const fh = await open(dir, 'r')
|
||||||
|
try {
|
||||||
|
await fh.sync()
|
||||||
|
} finally {
|
||||||
|
await fh.close()
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
// Not every file system can sync a directory; the file itself is synced.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Writes `data` to `file` so that a crash at any moment leaves either the old or the new content. */
|
||||||
|
export async function atomicWrite(file: string, data: string) {
|
||||||
|
const dir = dirname(file)
|
||||||
|
await mkdir(dir, { recursive: true })
|
||||||
|
const tmp = `${file}${TMP}${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`
|
||||||
|
const fh = await open(tmp, 'w')
|
||||||
|
try {
|
||||||
|
await fh.writeFile(data, 'utf8')
|
||||||
|
await fh.sync()
|
||||||
|
} finally {
|
||||||
|
await fh.close()
|
||||||
|
}
|
||||||
|
// Keep the current version as the backup, but only if it is intact.
|
||||||
|
const current = await readFile(file, 'utf8').catch(() => null)
|
||||||
|
if (current !== null && validJson(current)) await rename(file, `${file}.bak`)
|
||||||
|
await rename(tmp, file)
|
||||||
|
await syncDir(dir)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Reads a JSON file, falling back to its .bak when it is missing or damaged. */
|
||||||
|
export async function safeRead(file: string): Promise<string | null> {
|
||||||
|
for (const f of [file, `${file}.bak`]) {
|
||||||
|
const text = await readFile(f, 'utf8').catch(() => null)
|
||||||
|
if (text !== null && validJson(text)) return text
|
||||||
|
}
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
async function walk(dir: string, out: string[]) {
|
||||||
|
const entries = await readdir(dir, { withFileTypes: true }).catch(() => [])
|
||||||
|
for (const e of entries) {
|
||||||
|
const p = join(dir, e.name)
|
||||||
|
if (e.isDirectory()) await walk(p, out)
|
||||||
|
else out.push(p)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Removes temporary files left behind by writes that a crash interrupted. */
|
||||||
|
export async function removeStaleTemp(base: string) {
|
||||||
|
const files: string[] = []
|
||||||
|
await walk(base, files)
|
||||||
|
for (const f of files) if (f.includes(TMP)) await unlink(f).catch(() => {})
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Storage driver for Nitro's useStorage(), mounted in server/plugins/storage.ts. */
|
||||||
|
export function safeFsDriver(opts: { base: string }) {
|
||||||
|
const base = resolve(opts.base)
|
||||||
|
const path = (key: string) => {
|
||||||
|
if (/(^|:)\.\.(:|$)/.test(key)) throw new Error(`Invalid storage key ${JSON.stringify(key)}`)
|
||||||
|
return join(base, key.replace(/:/g, '/'))
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
name: 'safe-fs',
|
||||||
|
options: opts,
|
||||||
|
async hasItem(key: string) {
|
||||||
|
return (await safeRead(path(key))) !== null
|
||||||
|
},
|
||||||
|
getItem(key: string) {
|
||||||
|
return safeRead(path(key))
|
||||||
|
},
|
||||||
|
setItem(key: string, value: string) {
|
||||||
|
return atomicWrite(path(key), value)
|
||||||
|
},
|
||||||
|
async removeItem(key: string) {
|
||||||
|
const p = path(key)
|
||||||
|
await unlink(p).catch(() => {})
|
||||||
|
await unlink(`${p}.bak`).catch(() => {})
|
||||||
|
await syncDir(dirname(p))
|
||||||
|
},
|
||||||
|
async getKeys(prefix?: string) {
|
||||||
|
// Only walk the folder the prefix points at, not the whole store.
|
||||||
|
const dir = prefix ? join(base, prefix.replace(/:$/, '').replace(/:/g, '/')) : base
|
||||||
|
const files: string[] = []
|
||||||
|
await walk(dir, files)
|
||||||
|
const keys = new Set<string>()
|
||||||
|
for (const f of files) {
|
||||||
|
if (f.includes(TMP)) continue
|
||||||
|
const real = f.endsWith('.bak') ? f.slice(0, -4) : f
|
||||||
|
keys.add(relative(base, real).split(sep).join(':'))
|
||||||
|
}
|
||||||
|
return [...keys]
|
||||||
|
},
|
||||||
|
async clear() {
|
||||||
|
await rm(base, { recursive: true, force: true })
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,149 @@
|
||||||
|
import { execFileSync } from 'node:child_process'
|
||||||
|
import { existsSync, mkdtempSync, readFileSync, readdirSync, writeFileSync } from 'node:fs'
|
||||||
|
import { mkdir, readFile, rm, unlink, writeFile } from 'node:fs/promises'
|
||||||
|
import { tmpdir } from 'node:os'
|
||||||
|
import { join } from 'node:path'
|
||||||
|
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||||
|
import { backupName, pruneBackups, writeBackup } from '../server/utils/backup'
|
||||||
|
import { versionsToDrop } from '../server/utils/history'
|
||||||
|
import { atomicWrite, removeStaleTemp, safeFsDriver, safeRead } from '../server/utils/safe-fs'
|
||||||
|
|
||||||
|
let dir = ''
|
||||||
|
beforeEach(() => {
|
||||||
|
dir = mkdtempSync(join(tmpdir(), 'logigate-'))
|
||||||
|
})
|
||||||
|
afterEach(async () => {
|
||||||
|
await rm(dir, { recursive: true, force: true })
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('crash-safe files', () => {
|
||||||
|
it('keeps the previous good version as .bak', async () => {
|
||||||
|
const f = join(dir, 'a', 'p.json')
|
||||||
|
await atomicWrite(f, '{"v":1}')
|
||||||
|
await atomicWrite(f, '{"v":2}')
|
||||||
|
expect(await safeRead(f)).toBe('{"v":2}')
|
||||||
|
expect(await readFile(`${f}.bak`, 'utf8')).toBe('{"v":1}')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reads the .bak when the file was cut off by a power failure', async () => {
|
||||||
|
const f = join(dir, 'p.json')
|
||||||
|
await atomicWrite(f, '{"v":1}')
|
||||||
|
await atomicWrite(f, '{"v":2}')
|
||||||
|
await writeFile(f, '{"v":3, "nodes": [{"id": "n1", "ki') // truncated
|
||||||
|
expect(await safeRead(f)).toBe('{"v":1}')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reads the .bak when the power went out between the two renames', async () => {
|
||||||
|
const f = join(dir, 'p.json')
|
||||||
|
await atomicWrite(f, '{"v":1}')
|
||||||
|
await atomicWrite(f, '{"v":2}')
|
||||||
|
await unlink(f)
|
||||||
|
expect(await safeRead(f)).toBe('{"v":1}')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('ignores a half-written temporary file and cleans it up', async () => {
|
||||||
|
const f = join(dir, 'p.json')
|
||||||
|
await atomicWrite(f, '{"v":1}')
|
||||||
|
await writeFile(`${f}.tmp-123-456-abc`, '{"v":2, "trunc')
|
||||||
|
expect(await safeRead(f)).toBe('{"v":1}')
|
||||||
|
await removeStaleTemp(dir)
|
||||||
|
expect(readdirSync(dir).sort()).toEqual(['p.json'])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('does not overwrite a good .bak with a damaged file', async () => {
|
||||||
|
const f = join(dir, 'p.json')
|
||||||
|
await atomicWrite(f, '{"v":1}')
|
||||||
|
await atomicWrite(f, '{"v":2}')
|
||||||
|
await writeFile(f, 'garbage')
|
||||||
|
await atomicWrite(f, '{"v":3}')
|
||||||
|
expect(await readFile(`${f}.bak`, 'utf8')).toBe('{"v":1}')
|
||||||
|
expect(await safeRead(f)).toBe('{"v":3}')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('works as a storage driver: keys by prefix, without .bak and temp files', async () => {
|
||||||
|
const d = safeFsDriver({ base: dir })
|
||||||
|
await d.setItem('u1:index', '{"projects":[]}')
|
||||||
|
await d.setItem('u1:p:abc', '{"name":"A"}')
|
||||||
|
await d.setItem('u1:p:abc', '{"name":"B"}')
|
||||||
|
await d.setItem('u2:index', '{}')
|
||||||
|
await writeFile(join(dir, 'u1', 'p', 'abc.tmp-1-2-x'), 'x')
|
||||||
|
expect((await d.getKeys('u1:')).sort()).toEqual(['u1:index', 'u1:p:abc'])
|
||||||
|
expect(await d.getItem('u1:p:abc')).toBe('{"name":"B"}')
|
||||||
|
expect(await d.hasItem('u1:p:nope')).toBe(false)
|
||||||
|
await d.removeItem('u1:p:abc')
|
||||||
|
expect(await d.hasItem('u1:p:abc')).toBe(false)
|
||||||
|
expect(() => d.getItem('u1:..:x')).toThrow()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reads files written by the old storage the same way', async () => {
|
||||||
|
await mkdir(join(dir, 'user'), { recursive: true })
|
||||||
|
writeFileSync(join(dir, 'user', 'abc'), '{"id":"abc","email":"a@b.c"}')
|
||||||
|
const d = safeFsDriver({ base: dir })
|
||||||
|
expect(await d.getItem('user:abc')).toBe('{"id":"abc","email":"a@b.c"}')
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('savegames', () => {
|
||||||
|
const DAY = 86_400_000
|
||||||
|
const now = Date.UTC(2026, 9, 20, 12)
|
||||||
|
|
||||||
|
it('keeps the newest 30 versions', () => {
|
||||||
|
const times = Array.from({ length: 40 }, (_, i) => now - i * 60_000)
|
||||||
|
const drop = versionsToDrop(times, now)
|
||||||
|
// All on one day, whose last version is among the newest anyway.
|
||||||
|
expect(drop).toEqual(times.slice(30))
|
||||||
|
})
|
||||||
|
|
||||||
|
it('keeps the last version of each day for 90 days, nothing older', () => {
|
||||||
|
const times = [now, ...Array.from({ length: 100 }, (_, i) => now - (i + 1) * DAY - 3_600_000), ...Array.from({ length: 100 }, (_, i) => now - (i + 1) * DAY - 7_200_000)]
|
||||||
|
const drop = new Set(versionsToDrop(times, now))
|
||||||
|
const sorted = [...times].sort((a, b) => b - a)
|
||||||
|
const older = sorted.slice(30).filter(t => !drop.has(t))
|
||||||
|
// Beyond the newest 30: one per day, none older than 90 days.
|
||||||
|
expect(new Set(older.map(t => new Date(t).toISOString().slice(0, 10))).size).toBe(older.length)
|
||||||
|
expect(older.every(t => now - t <= 90 * DAY)).toBe(true)
|
||||||
|
expect(older.length).toBeGreaterThan(70)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('daily backup', () => {
|
||||||
|
it('backs up every file and restores it with the script', async () => {
|
||||||
|
const data = join(dir, 'data')
|
||||||
|
const backups = join(dir, 'backups')
|
||||||
|
await atomicWrite(join(data, 'users', 'user', 'u1'), '{"id":"u1"}')
|
||||||
|
await atomicWrite(join(data, 'accounts', 'u1', 'index'), '{"projects":[]}')
|
||||||
|
await atomicWrite(join(data, 'accounts', 'u1', 'p', 'abc'), '{"name":"v1"}')
|
||||||
|
await atomicWrite(join(data, 'accounts', 'u1', 'p', 'abc'), '{"name":"v2"}')
|
||||||
|
await writeFile(join(data, 'accounts', 'u1', 'broken'), '{"half')
|
||||||
|
await atomicWrite(join(data, 'shares', 'X'), '{"project":0}')
|
||||||
|
await atomicWrite(join(data, 'shares', 'X'), '{"project":1}')
|
||||||
|
await unlink(join(data, 'shares', 'X')) // crash between renames: only X.bak (the previous version) left
|
||||||
|
|
||||||
|
const { file, count } = await writeBackup(data, backups, new Date(Date.UTC(2026, 9, 20)))
|
||||||
|
expect(file.endsWith(backupName(new Date(Date.UTC(2026, 9, 20))))).toBe(true)
|
||||||
|
expect(count).toBe(4)
|
||||||
|
|
||||||
|
const target = join(dir, 'restored')
|
||||||
|
execFileSync('node', ['scripts/restore-backup.mjs', file, target])
|
||||||
|
const read = (p: string) => readFileSync(join(target, p), 'utf8')
|
||||||
|
expect(read('users/user/u1')).toBe('{"id":"u1"}')
|
||||||
|
expect(read('accounts/u1/p/abc')).toBe('{"name":"v2"}')
|
||||||
|
expect(read('shares/X')).toBe('{"project":0}')
|
||||||
|
expect(existsSync(join(target, 'accounts', 'u1', 'broken'))).toBe(false)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('keeps 14 daily backups and one per month', async () => {
|
||||||
|
await mkdir(dir, { recursive: true })
|
||||||
|
for (let i = 0; i < 60; i++) {
|
||||||
|
const d = new Date(Date.UTC(2026, 9, 20) - i * 86_400_000)
|
||||||
|
writeFileSync(join(dir, backupName(d)), '')
|
||||||
|
}
|
||||||
|
await pruneBackups(dir)
|
||||||
|
const left = readdirSync(dir).sort()
|
||||||
|
expect(left).toContain(backupName(new Date(Date.UTC(2026, 9, 20))))
|
||||||
|
expect(left).toContain(backupName(new Date(Date.UTC(2026, 7, 22))))
|
||||||
|
expect(left).toContain(backupName(new Date(Date.UTC(2026, 8, 1))))
|
||||||
|
expect(left).toContain(backupName(new Date(Date.UTC(2026, 9, 1))))
|
||||||
|
expect(left.length).toBe(14 + 3) // 14 newest + first of October, September, August
|
||||||
|
})
|
||||||
|
})
|
||||||
Loading…
Reference in New Issue