Storage Management

Storage limits and management in TMA Cloud.

Storage Driver

• S3: Files stored in S3-compatible object storage. Object keys stored in database.
• Set the bucket endpoint, name, and credentials (see Environment Variables). Missing or incomplete configuration stops backend startup.
• Uploads stream directly to the bucket with encryption. Downloads, copies, deletion, and sharing also use bucket storage. Bulk deletion uses the S3 multi-object API in batches of up to 1,000 keys.
• The driver uses operations supported by AWS S3 and Cloudflare R2: PutObject, GetObject, HeadObject, ListObjectsV2, DeleteObject, DeleteObjects, CopyObject, and multipart upload and copy operations.
• A single upload or copy request is limited to 5,000,000,000 bytes. Larger objects use multipart operations, which is the lower portable boundary across AWS S3 and R2.

Storage Limits

Per-User Limits

• Configurable storage limits per user
• S3: No disk; default is unlimited when no limit set. Display: "X used of Unlimited" or "X used of Y" (when limit set). Only per-user limit enforced.
• Set by administrators; real-time usage tracking

Storage Calculation

• Files: The account owner's users.storage_used counter tracks the sum of all file sizes
• Folders: Counted as 0 bytes
• Trash: Counted until permanently deleted

Storage Usage

Tracking

• Reads are constant-time from users.storage_used; statement-level database triggers update the counter after file inserts, updates, and deletes
• Upload and replacement transactions lock the account row before checking quota. Long object-store copies reserve their bytes first and release the reservation in the metadata transaction, so concurrent API instances cannot approve the same free space twice.
• S3: Total = per-user limit or null (Unlimited). Free = limit − used or null. No disk.
• Visual charts and indicators; per-user usage statistics

Monitoring

• Usage warnings at thresholds
• Limit enforcement on upload (per-user limit only)
• Storage quota exceeded errors

Per-User Access (Strict DB Permissions)

File access is enforced with strict database permissions so that User A cannot download or access User B's files, even if they guess a file ID.

• Main file API: All file operations (download, rename, move, etc.) use getFile(id, userId) or equivalent queries that require id AND user_id and deleted_at IS NULL. Only the owning user sees their files.
• OnlyOffice: The file-serving and callback endpoints use a per-user encryption context: the JWT and document key include userId. The server only serves or saves a file when the database row matches both id and user_id from the token/key.
• Share links: Shared-item download requires a valid share token and returns the file only if it is in that share (join with share_link_files), so access is scoped to the share.

This ensures that guessing or enumerating file IDs does not grant access to another user's data.

File Encryption

Files are automatically encrypted. Encryption uses AES-256-GCM in Google Tink's AES-GCM-HKDF-STREAMING format (AES256_GCM_HKDF_1MB), which splits each object into 1 MB segments, each with its own authentication tag. Each newly encrypted object version receives a random data key (DEK); FILE_ENCRYPTION_KEY is the key-encryption key (KEK) that wraps the DEK, and the wrapped DEK is stored on the file's database row.

Behavior

• Scope: File contents in the S3-compatible bucket are encrypted
• Transparent: Encryption and decryption happen automatically
• Streaming: Large files processed in streams to avoid memory issues
• Seekable: Because segments are independent, a single-file download can serve an HTTP Range request by decrypting only the overlapping segments

File Operations

• Read/Write: All file operations use streaming (S3 object keys)
• Upload: Files stream from the client through encryption to storage. A known object over 5,000,000,000 bytes uses multipart upload. For an unknown-length encrypted stream, the part size is calculated from the configured maximum upload size.
• Download: Files streamed from storage to client (S3: GetObject stream); supports HTTP Range for partial reads
• Copy: Copy runs on the background worker. The object store copies ciphertext directly. Objects up to 5,000,000,000 bytes use one copy request; larger objects use multipart copy. Part size grows with object size so the copy stays within 10,000 parts. A logical copy reuses the source object's wrapped DEK because its ciphertext is unchanged.
• Deep copies: The recursive plan is kept in a PostgreSQL temporary table. The application reads one cursor page at a time and copies at most four objects concurrently.
• Share: Share link downloads use the same bucket object keys and streaming decryption
• Folder ZIP: Authenticated and public-share folder archives read database cursor pages and append one decrypted file stream at a time.
• OnlyOffice save: A callback writes to a new object key, rechecks replacement quota under row locks, and commits the metadata swap. The old object is then queued for deletion. A rejected or failed save leaves the old version intact and queues any unused new object for deletion.

Key Configuration

• Set FILE_ENCRYPTION_KEY — the key-encryption key (KEK) that wraps each file's data key
• Set FILE_KEK_VERSION — the version number of the current KEK (default 1)
• Generate a key: openssl rand -base64 32 (a 32-byte base64 or hex key is recommended)
• Key can be base64, hex, or a passphrase (stretched with PBKDF2)
• Development fallback key used if not set (not secure for production)

Key Rotation

• Rotating the KEK only rewraps each file's stored DEK; the encrypted objects are never read or rewritten
• Set a new FILE_ENCRYPTION_KEY, bump FILE_KEK_VERSION, keep the previous key as FILE_ENCRYPTION_KEY_V<oldVersion>, then run rotate-kek.js. Remove the old key once it reports Remaining=0
• A deployment created before envelope encryption runs backfill-envelope-encryption.js once to give existing files a DEK. See CLI Commands

Storage Operations

Upload Limits

• Max upload size: Per-file size limit, configurable by admin in Settings → Storage (default 10 GB). Stored in app_settings.max_upload_size_bytes. Enforced on frontend (before upload) and backend (during upload).
• Multipart upload sizing uses the maximum encrypted size derived from this setting. The default multipart part size is 16 MiB, parts are equal-sized except for the last, and the plan may not exceed 10,000 parts.
• Pre-upload validation (Content-Length check)
• Final safeguard check (actual file size)
• Prevents file upload if limit exceeded
• Clear error messages with usage details
• Files cleaned up if validation fails

Cleanup

• Trash cleanup frees space
• Automatic background processes handle trash, expired share links, audit logs, old sessions, expired quota reservations, and expired file-operation results
• Failed uploads, replacements, and OnlyOffice saves queue unused storage objects for retryable deletion
• Orphaned files are not removed automatically; an admin reviews and deletes them from Settings → Administration → Review orphans
• S3: Upload validation (e.g. parentId) runs after stream upload. Failed bulk-upload and abandoned stream objects are queued for deletion. A process crash can still leave an orphan; review them periodically.

Orphans

An orphan is either an object in storage that no files row points at, or a files row whose stored object is missing. Both are reported by an on-demand scan.

• Scanning is read-only and makes a single paged pass over the bucket. Each storage page is staged in a temporary PostgreSQL table, so application memory is bounded to one storage page plus the capped report.
• Scan and delete work runs on the standalone worker. The API queues the job and the admin client polls its status.
• A grace window (1 hour minimum, 24 hours default, 1 year maximum) hides anything younger on either side, so an upload or paste still in progress is never reported.
• Row age comes from files.created_at, not files.modified, because uploads and copies preserve the client's original mtime.
• Deletion is itemised. Every entry is re-verified against the database and the storage timestamp at the moment of deletion, so a stale scan cannot remove a live file.
• Trashed rows still own their objects and count as referenced.

See Orphan Review for the admin workflow.

S3 Bucket Protection

Backend scripts apply bucket settings using the project S3 config. Run from backend directory.

Apply all protections

npm run s3:protect-all

Applies in one run: block public access; bucket policy that denies HTTP (HTTPS only); versioning; default SSE if supported; lifecycle rules.

Lifecycle rules

• Abort incomplete multipart uploads after 1 day (no effect on completed objects).
• Delete noncurrent versions after 7 days (versioning cleanup).
• Remove expired delete markers (tombstones from deletes on versioned bucket).

From backend: npm run s3:lifecycle applies both lifecycle rules. See CLI Commands.

Individual scripts

PutBucketPolicy replaces the entire bucket policy. If you add other policy statements in the storage UI, re-running s3:policy-https or s3:protect-all overwrites them.

Related Topics

• File System - How files are stored
• Admin Guide: Storage Limits - Configure limits
• CLI Commands - S3 scripts reference