ai newsMi., 28. Okt.
EilmeldungGemini 4 Carbon: Benchmark Reality, Token Economics, and Developer Tradeoffs

Auditable Receipt Storage for AI Images — A 5-Step Upload Design

TL;DR: For AI-generated receipt images in a healthtech system, use multipart upload for the original object, keep the object private, and make the application authorize every upload and retrieval. The client can upload directly to object storage for simpler delivery, but the authorization decision

dev.toMittwoch 20:43 · 28.10.8 Min. Lesezeit
Bild: dev.to

TL;DR: For AI-generated receipt images in a healthtech system, use multipart upload for the original object, keep the object private, and make the application authorize every upload and retrieval. The client can upload directly to object storage for simpler delivery, but the authorization decision must remain in the application. Record an immutable receipt ID, content type, byte count, checksum, and storage key before marking processing complete. A successful HTTP upload is not the same thing as an auditable receipt.

The page that wakes the on-call engineer should not say only “upload failures increased.” It should show that receipt originals are missing from the audit workflow, the affected tenant or partition, the last successful completion time, and whether the failure is happening before or after the object is committed. That distinction determines the first action: inspect authorization and upload-session creation when delivery is failing, or inspect object verification and event processing when delivery succeeded but the record did not advance.

The design below works backward from that page. It keeps access control as the primary decision axis, while treating delivery simplicity, on-call load, and capacity as constraints rather than reasons to weaken the audit trail.

Start with a state machine, not a storage SDK. A receipt can be created, uploading, uploaded, verified, processed, or quarantined. The original file should remain available for audit even when image interpretation fails. A failed transformation is a processing failure; it is not permission to replace or discard the source object.

The useful SLO is also a workflow SLO: the percentage of accepted receipts whose original object is verified and linked to an audit record within the target window. Measure upload latency separately. Otherwise a fast endpoint that accepts zero-byte or unauthorized objects can look healthy.

For capacity planning, begin with the number of receipts per day, the p95 original size, the retention period, and a replication or versioning factor. A rough planning expression is:

daily receipts × p95 bytes × retention days × safety factor

Use p95 rather than an average when a single generated image can be much larger than its neighbors. Recalculate it from observed distributions; do not quietly turn a 30-day estimate into a five-year retention promise. At 10,000 receipts per day, a 2 MB p95 image is about 20 GB of new logical data per day before metadata, replication, and retained versions are counted. The number is not a price quote. It is a signal that retention and deletion policy belong in the architecture review.

Multipart upload separates a large object into parts and lets the storage service assemble the original after the parts are committed. That gives a client a recoverable transfer boundary when a network connection drops, and it avoids routing every image byte through the application process. The trade is operational state: incomplete uploads, part limits, expiry, retries, and finalization all need metrics and cleanup.

For a Node.js service, the application should create a short-lived upload session after checking the caller's tenant, receipt ID, allowed media type, and maximum size. The browser or worker then sends parts using the storage protocol. Node.js does not need to buffer the complete image in memory. It does need to receive the completion result, verify the object metadata, and write the audit linkage transactionally enough that a later retry is harmless.

The word “multipart” can also mean an HTTP multipart/form-data request. That is a different layer. Use it when a client sends fields and a file to an application endpoint; use object-storage multipart upload when the transfer itself needs independently retriable parts. Mixing those meanings in an API contract makes debugging needlessly expensive.

Here is the control-plane shape in Go. It deliberately uses an interface rather than a vendor SDK; the important behavior is the authorization boundary and the state transition.

package receipts import ( "context" "fmt" ) type UploadPlan struct { ReceiptID string ObjectKey string MaxBytes int64 MediaType string } type Store interface { CreateMultipart ( ctx context. Context, key, mediaType string, maxBytes int64 ) ( string, error ) } func StartUpload ( ctx context. Context, store Store, tenant, receiptID string, allowed bool ) ( UploadPlan, string, error ) { if! allowed || tenant == "" || receiptID == "" { return UploadPlan {}, "", fmt.

Errorf ( "upload is not authorized" ) } plan:= UploadPlan { ReceiptID: receiptID, ObjectKey: "receipts/" + tenant + "/" + receiptID + "/original", MaxBytes: 10 * 1024 * 1024, MediaType: "image/png", } uploadID, err:= store. CreateMultipart ( ctx, plan. ObjectKey, plan.

MediaType, plan. MaxBytes ) if err!= nil { return UploadPlan {}, "", err } return plan, uploadID, nil } Enter fullscreen mode Exit fullscreen mode The 10 MB limit is an example policy, not a universal fact. Set it from the image-generation contract and observed data, then enforce it again when the upload is completed.

A client-provided content type is a hint, not proof. The verifier should inspect the resulting object's size and allowed media type before changing state to verified.

The application should authorize the intent, and the storage policy should constrain the resulting operation. A private object is the safer default for receipts because the original can contain identifiers that are irrelevant to image generation but material to an audit. A direct upload reduces delivery work, yet it does not mean the client gets a reusable read credential or a key chosen by the client.

Keep object keys opaque enough that a leaked identifier does not become a directory browser. Derive them from a server-owned tenant and receipt identifier, and store the mapping in the database. Do not accept an arbitrary path, bucket, or retention setting from the browser. On retrieval, authorize the audit actor against the receipt record before issuing a short-lived read operation or proxying the bytes.

The access-control decision has a cost in latency and code. A proxy gives the application a clear policy point and consistent logging, but it consumes bandwidth and worker capacity. A direct, short-lived upload is easier to scale for image bytes, but its issuance endpoint, expiration behavior, completion callback, and audit logging become part of the security boundary. The right choice depends on which boundary the team can observe and operate reliably.

This is a buy-versus-build decision, not a security shortcut. A managed service can reduce the number of storage daemons a platform team owns; it cannot decide whether a user is entitled to see a receipt. Self-hosting can make placement and protocol choices more explicit; it does not remove the need for authorization, verification, or recovery testing.

The dangerous gap is between “all parts arrived” and “the audit record is complete.” Finalization can be retried, so the completion handler should be idempotent. It should load the receipt, compare the expected object key and tenant, verify size and media type, record a checksum when available, and then advance the state. Replaying the same completion event must not create a second receipt or overwrite a different original.

Keep an expiry path for abandoned multipart sessions. Count sessions created, parts rejected, completions attempted, completions verified, verification failures, and cleanup failures. Alert on a rising ratio of stale sessions and on the age of the oldest unverified receipt. A threshold that is too low creates pages during ordinary client retries; a threshold that is too high hides an audit gap. Both errors have a cost, so tune it against the workflow SLO and a known recovery window.

One limitation is that direct multipart upload is not suitable when policy requires byte inspection before storage. The opposite trade-off applies to a proxy: it is not suitable when image traffic would consume the application's available streaming capacity. Neither path removes the need to test restore, credential expiry, or authorization after a tenant change. Those are design constraints, not details to postpone until the first audit request.

Retries need a boundary. Retry a transient part transfer with bounded backoff. Do not retry a permission denial as if it were a network failure. Do not mark the database record complete because a request timed out after the storage service may have accepted it; query the upload state or reconcile asynchronously. The reconciler is not optional bookkeeping. It is how the system resolves the ambiguous interval created by distributed commits.

Choose direct multipart upload when the original images are large enough that proxying their bytes would consume a meaningful share of application capacity, and when the team can operate session expiry, completion verification, and reconciliation. Choose a proxy when the files are small or when the application must inspect every byte before storage. In either case, preserve the original under a private, server-derived key and make the audit record the source of authorization.

There is no universally correct upload path.

Test the ugly paths before testing throughput: an expired upload authorization, a part replay, a completion retry, a mismatched tenant, an object larger than the declared limit, a missing finalization event, and a storage outage after the database enters uploading. Then test restore and audit retrieval with a retained original. The result should be observable as a workflow, not as a collection of successful HTTP responses.

The alert is useful only when it points to that distinction. If it tells the on-call engineer which state is stuck and which evidence is missing, the system can recover without weakening access control. If it reports only request volume, the team will eventually trade an audit guarantee for a green dashboard.

Quelle: dev.to (dev.to). Original öffnen