Understand

Storage providers

Object storage is an implementation layer beneath the ThimbleDB protocol. The browser cache, envelope format, trie/snapshot layouts, key scopes, and write semantics do not depend on one cloud.

Provider contract

Every provider implements four operations:

interface ObjectStore {
  get(key: string): Promise<StoredObject | null>
  put(
    key: string,
    bytes: Uint8Array,
    conditions?: {
      ifMatch?: string
      ifNoneMatch?: boolean
    }
  ): Promise<{ etag: string }>
  delete(key: string): Promise<void>
  list(prefix: string): Promise<string[]>
}

The two conditional-write modes are load-bearing:

  • create an immutable object only when its key does not already exist
  • replace HEAD only when its current ETag matches

A provider that cannot enforce both operations atomically is not a safe multi-writer backend.

Supported providers

ProviderPositionWrite integrationBrowser read integration
Cloudflare R2Preferred reference providerNative Worker R2 bindingAuthenticated Worker broker
Local filesystemDevelopment and single-process useIn-process file adapterSame-origin authenticated broker
Azure Blob StorageSupported secondary providerAzure SDK and conditional blob writesAuthenticated authority broker
Amazon S3Supported secondary providerAWS SDK and IAM roleAuthenticated authority broker
S3-compatible storageRequires conformance testingS3 endpoint adapterProvider-specific

The base thimbledb install does not include cloud provider SDKs. Install only the adapter required by a Node deployment:

# Azure Blob
npm install thimbledb @azure/storage-blob

# Amazon S3 or R2 through the S3 API
npm install thimbledb @aws-sdk/client-s3

The Cloudflare authority uses its native R2 binding and needs neither package. The Node authority loads Azure and S3 adapters only when the corresponding THIMBLE_PROVIDER value is selected.

Cloudflare R2

R2 is the preferred provider because:

  • Workers receive a native bucket binding without long-lived API credentials
  • reads and writes are strongly consistent
  • conditional R2 operations map directly to the protocol
  • private buckets remain behind the Worker broker
  • egress is free
  • the free tier covers many small evaluation applications

Cloudflare-specific authentication and deployment do not change stored bytes.

Local filesystem

The local provider stores object keys below .thimble-data. It is useful for:

  • local browser development
  • protocol tests
  • benchmarks
  • one Node process on one machine
  • offline demonstrations

Its key locks exist only inside one Node process. Two independent processes can race and violate compare-and-swap semantics. Do not use the local adapter for a multi-process or shared-network-filesystem deployment.

Use SQLite, OS file locks, or another transactional embedded store when a durable multi-process local provider is required.

Azure Blob Storage

Azure maps protocol conditions to If-None-Match and If-Match. Browser reads use the authority broker.

The supplied Node authority uses a connection string. Prefer managed identity for a long-lived production deployment.

Amazon S3

S3 maps protocol conditions to conditional PutObject. The authority can use an IAM role through the normal AWS credential chain.

The Lambda authority brokers browser reads while its IAM role accesses private S3 buckets.

Adding a provider

Before describing a provider as supported:

  1. Implement the ObjectStore contract.
  2. Prove atomic create-if-absent.
  3. Prove stale ETag replacement is rejected.
  4. Verify strong read-after-write behaviour.
  5. Verify ETag formatting through the authenticated object broker.
  6. Run the engine contract tests.
  7. Run concurrent-writer and maintenance benchmarks.
  8. Document authority authentication, trusted proxy handling, and secret delivery.

API compatibility with S3 is not enough. Conditional and consistency semantics must be tested.

This page is built from the repository source.

View or improve this page on GitHub