Walrus Console API: Programmatic Clients

For CI jobs, agents and scripts that upload and delete with nobody watching. It covers three things a script needs and a person in the web app does not: checking a folder before encrypting for it, waiting on a state instead of sleeping, and knowing when storage has been released.

The Quickstart covers keys, folder creation and Seal. Start there if you have not made a first upload yet. A folder in Console is a bucket in the API.

1. Check the folder before you upload

Files in a private folder are encrypted on your side, under the folder's Seal policy. Console stores the bytes it is given and cannot tell whether they will decrypt, so it checks the folder's policy instead and refuses the upload when the policy cannot be resolved. Ask for the same check before you encrypt:

GET /api/v1/buckets/{bucketId}/policy
Authorization: Bearer <api key>

{ "data": { "state": "resolvable", "seal_policy_id": "0x8f96..." } }

reason is present only with missing:

resolvable is about the policy itself. Decrypting also needs your key to hold a role in the folder's group, and the group not to be paused. This route checks neither.

The upload route applies the same rule, and refuses an upload into a missing folder before anything is stored. With policy_missing it answers 404, as every other route does. With any other reason it answers:

{
  "error": "This bucket's encryption policy cannot be resolved, so files uploaded to it could not be decrypted. Nothing was uploaded.",
  "code": "bucket_policy_unresolvable"
}

That is a 409, and retrying it will not help. Uploads are not refused on unknown.

A folder that is being deleted answers 409 bucket_deleting here, then bucket_gone, as an upload into it would.

A new API key gets 403 mirror_missing_grant on this route for a few seconds, as on every folder route. The Quickstart explains the retry. If it has not cleared after a minute, report the folder id.

This route needs a read grant on the folder, and an upload needs a write grant. A key that can only read the folder gets resolvable here and 403 mirror_missing_grant on the upload.

2. Wait on a state, not on a timer

Uploads and deletes both finish in the background. Each has a route that reports where it is.

upload   POST   .../files                     -> 202, data.id
         GET    .../files/{fileId}/status     queued -> active -> completed
                                                               -> failed

delete   DELETE .../files/{fileId}            -> 204
         GET    .../files/{fileId}/deletion   deleting -> deleted
                                                       -> delete_failed

Upload

GET /api/v1/buckets/{bucketId}/files/{fileId}/status

1. POST .../files returns 202 with data.id.
2. Poll the status every 1 to 2 seconds until completed or failed.

The status of an old upload eventually answers 404, because only the most recent finished jobs are kept. The file record is permanent: GET .../files/{fileId} reports status as pending, uploading, active or failed, and active is the same fact as completed.

Delete

GET /api/v1/buckets/{bucketId}/files/{fileId}/deletion

{ "data": { "state": "deleting" } }

1. DELETE .../files/{fileId} returns 204.
2. Poll the deletion until deleted or delete_failed.

A delete usually takes a few seconds. Wait for an upload to reach completed before deleting the file.

While the file is deleting, and once it is deleted, it is out of listings and both GET .../files/{fileId} and its /status answer 404. Only /deletion still knows it. After delete_failed the file record answers again, with status failed: that is a failed delete, not a failed upload, and /deletion is what tells the two apart. For a file nobody asked to delete, /deletion answers 404.

When the worker hits a transient upstream error it tries again, five times in all over about 30 seconds. Give your wait a timeout: a file still deleting after 5 minutes has stalled and will not finish on its own. Report its id instead of sending a second DELETE, which answers 404.

Deleting a whole folder is reported on the folder: status on GET /api/v1/buckets/{bucketId} reads deleting, then deleted. Meanwhile reads of the files under it answer 409 bucket_deleting, then bucket_gone.

3. Know when storage is released

storage_used on GET /api/v1/buckets/{bucketId} is the folder's counter. The space quota that uploads are checked against moves with it, in the same transaction.

So there is no second delay to wait out: once a file's deletion reads deleted, its storage is released, and an upload that was refused with quota_exceeded can be sent again.