Uploaded images, generated reports, and backups don’t belong in a database. They
belong in an object store: a bucket of named blobs, priced by the gigabyte, served
straight to clients. Every cloud has one, and they don’t agree on an API.
ObjectStorage is one interface over every one of them. The provider is chosen by a
property, so the same code moves from Amazon to Google to Azure, or onto a local
emulator, by changing one line.
Each provider is implemented over the backend’s own HTTP client, with no cloud
SDK. The build links only the provider the configuration names. The scheme
decides: r2://${CF_ACCOUNT_ID} links R2 alone. When the URL isn’t in the
application’s properties at all, because the deployment supplies it, the build
links every provider; cn1.storage.providers=gs names the set instead.
Choosing a provider
cn1.storage.url names the provider:
| URL | Provider |
|---|---|
| An in-process store for tests and development. This is the default when no URL is set. |
| Amazon S3 in that region. |
| An S3-compatible service such as MinIO, Ceph, Backblaze B2 or Wasabi, addressed path-style. |
| Cloudflare R2. Set |
| Google Cloud Storage. |
| Azure Blob Storage. A bucket here is a container. |
Credentials
Each provider takes its credentials from the configuration first and then from the environment variables its own tools use, so a server deployed on a cloud usually needs no secret at all.
# Amazon S3: explicit keys, else the standard AWS chain (environment,
# web identity, ECS, EC2 instance role)
cn1.storage.url=s3://eu-west-1
# Cloudflare R2: an R2 API token's S3 keys
cn1.storage.url=r2://${CF_ACCOUNT_ID}
cn1.storage.r2.access-key-id=${R2_KEY_ID}
cn1.storage.r2.secret-access-key=${R2_SECRET}
# Google Cloud Storage: a service account key file
# (GOOGLE_APPLICATION_CREDENTIALS), else the metadata server on GCE,
# Cloud Run and GKE
cn1.storage.url=gs://my-project
# Azure: a connection string or an account key, else the managed identity
cn1.storage.url=azblob://myaccount
cn1.storage.azure.account-key=${AZURE_STORAGE_KEY}| Property | Meaning |
|---|---|
| S3 keys. The same keys under |
| The base URL of an S3-compatible service. |
| A service account key file. Falls back to |
| The project new buckets are created in. Used when the URL names none. Falls back to |
| The location new buckets are created in. |
| A fake-gcs-server URL. Falls back to |
| An Azure connection string. Falls back to |
| The account key. Falls back to |
| A user-assigned managed identity. Falls back to |
| Another endpoint: a sovereign cloud, a private endpoint, or an emulator. |
A managed identity on Azure needs a data-plane role such as Storage Blob Data Contributor. On Google Cloud without a key file, the service account needs the Service Account Token Creator role on itself to sign URLs.
Uploads straight from the app
The object bytes of a mobile upload shouldn’t pass through the server. The server hands the app a signed URL, and the app sends the file to the provider directly:
@Component
public class Avatars {
private final ObjectStorage storage;
public Avatars(ObjectStorage storage) {
this.storage = storage;
}
/// A URL the app PUTs the image to, valid for ten minutes.
public String uploadUrl(long userId) throws IOException {
return storage.presignPut("avatars", userId + ".png", 600);
}
/// A URL any client can show the image from, valid for an hour.
public String viewUrl(long userId) throws IOException {
return storage.presignGet("avatars", userId + ".png", 3600);
}
}The URL carries its own short-lived authorization, so the app needs no
credentials. The app sends the bytes with a plain HTTP PUT. Azure requires one
extra header on that PUT, x-ms-blob-type: BlockBlob. S3 and Google Cloud
Storage ignore it, so an app that targets several providers can always send it.
How each provider signs:
S3 and R2 use AWS Signature Version 4, computed locally.
Google Cloud Storage uses a V4 signed URL. It’s signed with the service account key, or, on the metadata server, through the IAM
signBlobAPI.Azure uses a service SAS with an account key, or a user delegation SAS with a managed identity. The user delegation SAS costs one call a day to fetch the delegation key.
The in-memory store refuses to sign a URL, because no server would answer the URL.
The rest of the interface
putObject, getObject, headObject, deleteObject and listObjects act on
objects from the server itself. Deleting a missing object isn’t an error, and
headObject answers null for one. createBucket succeeds when the account
already owns the bucket, so a first run against an emulator can call it
unconditionally.
Outside a bean, ObjectStorages.current() returns the same instance. In a server
wired by hand, ObjectStorages.register adds a provider.
Local development
All three providers have emulators, and the backend talks to each through configuration alone:
# MinIO
cn1.storage.url=s3://us-east-1
cn1.storage.s3.endpoint=http://localhost:9000
# fake-gcs-server
cn1.storage.url=gs://demo
cn1.storage.gcs.emulator-host=http://localhost:4443
# Azurite
cn1.storage.url=azblob://
cn1.storage.azure.connection-string=UseDevelopmentStorage=true