Skip to content

Engineering Log

This is the maintainers' technical log for the current BillTracker architecture. It is intentionally different from the user-facing release notes:

  • Release notes explain visible product changes.
  • HISTORY.md is the full chronological application changelog.
  • This log records architecture state, design reasoning, verification, operational impact, and follow-up risk.

Current Baseline

Field Value
Application version 0.37.0
Baseline reviewed June 7, 2026
Runtime Node.js 22 Alpine container
Frontend React 19 SPA built by Vite
Backend Express JSON API and static SPA host
Database SQLite through better-sqlite3, WAL enabled
Persistent app volume /data
Live database /data/db/bills.db
Managed backups /data/backups
Docs source /portainer/hosting/bill-tracker/mkdocs
Docs container squidfunk/mkdocs-material:latest

System Ledger

Subsystem Primary files Persistent state Important failure boundary
HTTP server server.js None Middleware ordering and SPA fallback
Local auth services/authService.js, routes/auth.js users, sessions, user_login_history Cookie flags, inactive users, session invalidation
OIDC services/oidcService.js, routes/authOidc.js OIDC settings, local users, sessions Provider discovery, callback validation, admin group mapping
CSRF middleware/csrf.js, client/api.js httpOnly cookie + SPA memory cache SPA must fetch GET /api/auth/csrf-token on startup and send the token in x-csrf-token; server validates header == cookie
Tracker services/trackerService.js, services/statusService.js Bills, payments, monthly state Cycle-aware due dates and grace periods
Bills and payments routes/bills.js, routes/payments.js, services/paymentValidation.js Bills and payments Ownership, real date validation, positive amounts
SimpleFIN services/simplefinService.js, services/bankSyncService.js, services/bankSyncWorker.js Sources, accounts, transactions, encrypted secret Remote errors, access revocation, dedupe, pacing
Matching services/transactionMatchService.js, services/matchSuggestionService.js, services/billMerchantRuleService.js Transactions, payments, rejections, merchant rules Avoid duplicate payment creation and undo balance changes
Imports Spreadsheet, CSV, and user DB import services Preview sessions and history Preview/apply separation and ownership
Exports routes/export.js Temporary files only Delete temp files and exclude server-wide secrets
Backups services/backupService.js, services/backupScheduler.js Full SQLite snapshots Validation, atomic replacement, retention
Cleanup services/cleanupService.js, workers/dailyWorker.js Cleanup settings and summary Preserve recovery window and avoid deleting active files
Status routes/status.js, services/statusRuntime.js Runtime memory plus selected settings Report useful signals without leaking secrets or paths

Technical Decisions

TD-001: Same-Origin SPA And API

Decision: Express serves both the API and Vite production build.

Reasoning: Normal deployment needs no CORS configuration, cookies remain same-origin, and the container stays simple.

Operational impact: Leave CORS_ORIGIN unset unless the frontend is deliberately hosted separately.

TD-002: SQLite With WAL

Decision: Keep a single SQLite database with WAL enabled.

Reasoning: BillTracker is a self-hosted household-scale application. SQLite keeps deployment and backup handling understandable while providing transactional writes.

Operational impact: Run one app service against a database file. Preserve the live file and sidecars before manual recovery work.

TD-003: Server-Side Sessions

Decision: Store UUID sessions in SQLite and send only bt_session to the browser.

Reasoning: Server-side revocation is needed for logout-all, password reset, role changes, deactivation, and administrator safety.

Operational impact: Session cleanup runs at startup, periodically, and in the daily worker.

TD-004: Double-Submit CSRF With In-Memory Token Fetch

Decision: The server sets bt_csrf_token as httpOnly by default (since v0.35). The SPA fetches the token from GET /api/auth/csrf-token on startup, stores it in a module-level memory cache, and sends it in the x-csrf-token header on mutations. The server validates that the header and cookie values match.

Reasoning: The previous readable-cookie design exposed the CSRF token to any XSS vulnerability. The in-memory fetch + httpOnly cookie flow moves the token out of the XSS-accessible cookie surface without breaking the double-submit guarantee.

Operational impact: CSRF_HTTP_ONLY=true is the default and is the correct setting for the current SPA. CSRF_HTTP_ONLY=false remains available for custom clients that need to read the cookie directly, but is not needed by the bundled SPA.

TD-005: Protected Bootstrap Admin

Decision: Separate the default bootstrap administrator from user tracker access.

Reasoning: Administration should not imply routine visibility into household financial data.

Operational impact: Create a regular user for normal tracking. Host and backup access remain privileged and must be protected separately.

TD-006: Preview Before Import Apply

Decision: XLSX, CSV, and user SQLite imports use staged preview and apply workflows.

Reasoning: Financial import data is messy. Explicit review prevents silent corruption and gives users control over ambiguous mappings.

Operational impact: import_sessions require cleanup. Import history is user-scoped and retained unless optional cleanup is enabled.

TD-007: User Export Is Not Admin Backup

Decision: User exports contain only user-owned records. Full SQLite backups remain an Admin feature.

Reasoning: Users need portability without receiving passwords, sessions, server settings, or other users' records.

Operational impact: Document and test both paths independently.

TD-008: Zero-Configuration Secret Encryption

Decision: Encrypt SimpleFIN access URLs with AES-256-GCM. Generate and store a stable key in SQLite settings unless TOKEN_ENCRYPTION_KEY is supplied.

Reasoning: Bank sync should work without a separate secret bootstrap step while still supporting operator-managed key material.

Operational impact: A database backup may include the generated key. An operator-supplied override must remain stable and be protected separately.

TD-009: Full Backup Validation Before Publish Or Restore

Decision: Write backups to temporary files, run SQLite integrity checks, and atomically rename validated files.

Reasoning: Partial or invalid snapshots should never appear as restorable managed backups.

Operational impact: Cleanup removes stale .partial and .upload files after a 2-hour age threshold.

TD-010: Bounded SimpleFIN Worker

Decision: Auto-sync uses configurable intervals, a 1-hour minimum source age, and a 3-second stagger between sources.

Reasoning: The worker should not hammer SimpleFIN or repeat a recent manual sync unnecessarily.

Operational impact: A manual sync may intentionally delay the next worker sync for that source.

Latest Engineering Entries

2026-06-07: v0.37.0 Documentation Refresh

Status: Complete

Context: The application shipped four minor releases (v0.34.0, v0.35.0, v0.36.0, v0.37.0) since the docs were last updated. The most visible documentation gaps were the autopay trust indicator, tracker sorting, sparkline, and the private calendar subscription feed. The release notes page and engineering baseline still pointed at the old v0.33.8.7 build.

Changes:

  • Updated the release notes page to v0.37.0 with new feature, security, and operations entries.
  • Bumped the engineering baseline (application version, baseline date, frontend row) to v0.37.0.
  • Refreshed the tracker-planning page with sorting, sparkline, Changed badge, and autopay trust sections.
  • Refreshed the payments page with autopay trust and bank-overrides sections.
  • Refreshed the data-sync page with auto-match review, auto-learn rules, and ambiguous-protection sections.
  • Refreshed the bills page with the cancellation-reason dropdown.
  • Refreshed the security, privacy, and authentication pages to describe session-token hashing and the geolocation opt-in toggle.
  • Added a Service Catalog page under User Guide.
  • Updated the database reference and migration history with the v0.70, v0.78, v0.82, v0.84-v0.94, v0.95-v0.99 families.
  • Updated the API reference with the new verify-autopay, recent-auto, undo-auto, unmatch-bulk, attribute-to-month, webauthn, calendar feed, and spending endpoints.

Verification:

mkdocs build --strict
zero warnings
no broken internal links
expanded source now includes Service Catalog page
release notes index reflects v0.37.0

Operational note: No new deployment, backup, or schema concerns arise from the doc refresh itself. The notes describe behaviors that are already live in the running app.

2026-05-30: MkDocs Repair And Detailed Reference Expansion

Status: Complete

Context: The docs container was mounted correctly, but its source had an extra docs/docs/ directory and site_url pointed to https://dream.scheller.ltd/null/BillTracker. mkdocs serve redirected / to /null/BillTracker/, where every page and asset failed.

Changes:

  • Flattened MkDocs source to one docs/ directory.
  • Replaced stale Forgejo-derived URL metadata with a root-served config.
  • Removed unavailable plugins from the runtime configuration.
  • Added complete navigation with no phantom pages.
  • Rewrote user, admin, configuration, technical, security, privacy, and release documentation from current source.
  • Added an operator runbook, verification guide, data-flow reference, and this engineering log.
  • Captured a live v0.33.8.7 login screenshot without authenticated financial data.

Verification:

mkdocs build --strict
28-page baseline served successfully before expansion
root / changed from redirect to HTTP 200
old /null/BillTracker/ path returns HTTP 404
logo, favicon, and search index return HTTP 200
expanded source: 36 Markdown pages and 12,293 words
expanded generated site: 36 HTML index pages
new visual tour, runbook, testing, data-flow, and engineering-log URLs return HTTP 200
live login screenshot and expanded search index return HTTP 200

Operational note: The MkDocs development server required a container restart after config replacement because it retained the original startup base URL.

2026-05-29: Tracker Layout Revert, Subscription Badge Retained

Version: 0.33.8.7

Context: The side-by-side compact tracker experiment widened layout containers and reduced column space. The result was reverted after evaluation.

Kept:

  • Compact S subscription badge
  • Bucket remaining amount
  • Bucket Done label

Reverted:

  • Side-by-side buckets
  • Compact tracker props and narrow table override
  • 2000px maximum layout width

Decision: Preserve readability and predictable table layout over denser wide-screen packing.

2026-05-29: SimpleFIN Seed, Backfill, And Status Accuracy

Versions: 0.33.8.3-0.33.8.4

Changes:

  • Started dailyWorker during application boot.
  • Added SimpleFIN status card and worker metrics.
  • Added first-connect 89-day seed sync.
  • Added explicit 89-day backfill action.
  • Kept routine sync at 30 days.
  • Corrected source error reporting to require status = 'error'.
  • Corrected tracker overdue count query.

Risk addressed: Active SimpleFIN sources can carry partial-warning text in last_error; status must not present those sources as fully failed.

2026-05-29: Advisory Filters And Subscription Catalog V2

Versions: 0.33.8.0-0.33.8.2

Changes:

  • Seeded more than 5,000 advisory non-bill transaction patterns.
  • Added bill-like override terms.
  • Expanded subscription catalog with 90 additional services.
  • Corrected selected subscription categories.
  • Scoped Georgia digit rendering to numeric and currency glyphs.

Decision: Transaction classification remains advisory. Users retain the ability to create a recurring bill when a filter guess is wrong.

2026-05-29: Merchant Rules And Transaction Payment Backfill

Version: 0.33.7.2

Changes:

  • Persisted merchant-to-bill rules.
  • Applied merchant rules to newly synced transactions.
  • Added bill-side SimpleFIN payment backfill.
  • Created payment records during recommendation-to-bill flows.

Risk addressed: Matching a transaction must not create duplicate payments or lose reversibility during unmatch.

2026-05-28: Shared Transaction Foundation

Migration family: v0.59-v0.64

Changes:

  • Added payment source and linked transaction metadata.
  • Added shared data_sources, financial_accounts, and transactions.
  • Added provider transaction dedupe index.
  • Added persisted suggestion rejections.
  • Added subscription metadata and account monitoring.

Decision: Manual CSV import and provider sync share one transaction model so matching and payments behave consistently.

2026-05-11: Spreadsheet Import Parser Hardening

Version: 0.26.1

Changes:

  • Reworked dual-column header detection around repeated fields.
  • Prevented column leakage between header groups.
  • Added header_set_index.
  • Filtered summary rows, total rows, blank rows, negative leftover rows, and dashed separators.

Verification: Real spreadsheet fixture produced separate left and right bill groups without leaked amounts.

Current Risks And Follow-Ups

Area Current state Follow-up
Reverse proxy CSRF cookie CSRF Secure flag uses Express req.secure; server does not configure trust proxy Verify Set-Cookie under TLS termination and decide whether to configure trusted proxy hops
Backup confidentiality Backups and exports are not encrypted by the app Use encrypted storage and protected transfer channels
SQLite scaling Intended for one app replica Re-evaluate before multi-replica deployment
Screenshot freshness Authenticated screens intentionally omitted from production docs Capture synthetic demo-user screenshots after major UI milestones
Documentation source MkDocs tree is intentionally ignored by app Git Keep remote sync and strict-build verification in the docs maintenance workflow

Entry Template

Add new entries latest-first:

### YYYY-MM-DD: Short Technical Title

**Version:** `x.y.z`

**Status:** Planned | In progress | Complete | Reverted

**Context:** Why the work was needed.

**Changes:**

- Concrete implementation details

**Files:**

- `path/to/file`

**Persistent state impact:** Tables, settings, files, or none.

**Verification:**

```text
Commands and observed result
```

**Operational impact:** Deploy, rollback, monitoring, or support notes.

**Remaining risk:** Known limitations and follow-up work.

Maintenance Rules

  1. Record architecture-affecting work here in the same release window.
  2. Keep user-facing product language in release notes.
  3. Name persistent-state changes explicitly.
  4. Include verification commands and observed results.
  5. Record reversions with the same care as feature additions.
  6. Never include secrets, production transaction data, session IDs, or private infrastructure credentials.

See also

Next steps