cf previews deploy cannot find Durable Object namespaces past the first page of 1000 #406

Summary

cf previews deploy for a Worker with a Container failed with:

Could not find a Preview Durable Object namespace for Container class "Sandbox".

The account has 1056 Durable Object namespaces. cf's fetchPagedListResult follows only result_info.cursor. The Durable Object namespaces endpoint pages by page number and returns no cursor, so cf reads only the first 1000 namespaces.

Observed with cf 1.0.0-beta.12. Links below are pinned to cf@1.0.0-beta.21 and the matching workers-sdk release commit.

Cause

When the Preview deployment's bindings carry no namespace_id for the Container class, cf lists all namespaces and searches them for the class and Preview ID (packages/cli/src/commands/previews/containers.ts#L44-L72):

namespaces ??= await listDurableObjects(config, accountId);
namespaceId = namespaces.find(
	(namespace) =>
		namespace.class === container.class_name &&
		namespace.preview?.id === deployment.preview_id
)?.id;

listDurableObjects comes from @cloudflare/containers-shared and requests per_page=1000 through the injected fetchPagedListResult (packages/containers-shared/src/deploy.ts#L233-L243):

return await fetchPagedListResult<DurableObjectNamespace>(
	complianceConfig,
	`/accounts/${accountId}/workers/durable_objects/namespaces`,
	{},
	new URLSearchParams({ per_page: "1000" })
);

cf previews deploy injects the deploy context's fetchPagedListResult (packages/cli/src/commands/previews/deploy.ts#L132-L142). In the deploy context, fetchListResult and fetchPagedListResult both call workers-utils' fetchListResultBase (packages/cli/src/lib/deploy-context.ts#L75-L98):

// Same implementation as fetchListResult — workers-utils has no
// separate paged-list base function; both fetch all pages.
fetchPagedListResult: (complianceConfig, resource, init, queryParams) =>
	// TODO: actually move fetchPagedListREsult into workers-utils...
	fetchListResultBase(

fetchListResultBase requests another page only when result_info.cursor is set (packages/workers-utils/src/cfetch/index.ts#L233-L276, hasCursor at #L586-L591):

if (hasCursor(json.result_info)) {
	cursor = json.result_info?.cursor;
} else {
	getMoreResults = false;
}

The namespaces endpoint returns page-number result_info and no cursor:

curl -s -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
  "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/workers/durable_objects/namespaces?per_page=1000&page=1" | jq -c .result_info
# {"page":1,"per_page":1000,"count":1000,"total_count":1056}

curl -s -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
  "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/workers/durable_objects/namespaces?per_page=1000&page=2" | jq -c .result_info
# {"page":2,"per_page":1000,"count":56,"total_count":1056}

Wrangler passes containers-shared a different fetchPagedListResult (packages/wrangler/src/preview/containers.ts#L104-L108), which sets page=1, 2, … until hasMorePages returns false (packages/wrangler/src/cfetch/index.ts#L99-L134, hasMorePages at #L340-L353).

The same code is in 1.0.0-beta.12. It pins @cloudflare/workers-utils 0.46.0 and @cloudflare/containers-shared 0.21.3, where fetchListResultBase, hasCursor and listDurableObjects match the versions linked above. Between cf@1.0.0-beta.12 and cf@1.0.0-beta.21, deploy-context.ts changed only its logger.

Other unfiltered calls through the same fetchPagedListResult:

• deployContainers, which deploy-helpers' deploy calls, falls back to listDurableObjects to resolve a namespace ID (containers-shared/src/deploy.ts#L145). deploy-helpers also calls it in durable-object-container-applications.ts#L335.
• Queue provisioning lists /accounts/{account_id}/queues with no name filter (deploy-helpers/src/deploy/helpers/provision-bindings.ts#L852-L856). That endpoint also pages by number: {"page":1,"per_page":100,"count":65,"total_count":65,"total_pages":1}.

Reproduction

Without an account that has more than 1000 namespaces, call fetchListResultBase against a local server that returns page-number result_info:

npm i @cloudflare/workers-utils@0.48.0
node repro.mjs

// repro.mjs
import { once } from "node:events";
import { createServer } from "node:http";
import { fetchListResultBase } from "@cloudflare/workers-utils";

// One namespace per page, paged by page number like the real endpoint.
const namespaces = [{ class: "Other" }, { class: "Sandbox" }];
const requests = [];

const server = createServer((req, res) => {
	const url = new URL(req.url, "http://localhost");
	requests.push(url.search);
	const page = Number(url.searchParams.get("page") ?? "1");
	res.end(JSON.stringify({
		success: true,
		result: [namespaces[page - 1]],
		result_info: { page, per_page: 1, count: 1, total_count: namespaces.length },
	}));
});
server.listen(0);
await once(server, "listening");
process.env.CLOUDFLARE_API_BASE_URL = `http://localhost:${server.address().port}/client/v4`;

const logger = { debug() {}, log() {}, info() {}, warn() {}, error() {} };
const listed = await fetchListResultBase(
	{}, "/accounts/acc/workers/durable_objects/namespaces", {}, "repro", logger,
	new URLSearchParams({ per_page: "1" }), { apiToken: "unused" }
);
server.close();

console.log({ requests, listed });

Output: one request, and the Sandbox namespace on page 2 is missing:

{ requests: [ '?per_page=1' ], listed: [ { class: 'Other' } ] }

On an account with more than 1000 namespaces, cf previews deploy fails with the error above when the Preview namespace is on page 2 of the list and the deployment's bindings carry no namespace_id for the class.

Expected behaviour

fetchPagedListResult reads every page when the endpoint pages by number, as Wrangler's fetchPagedListResult does.

Why cf lists namespaces here

The list is the fallback for Container classes that have no Durable Object binding, for example a class declared only in exports and reached through ctx.exports. The Preview deployment's env then has no durable_object_namespace entry to carry a namespace_id, and its exports entry is only {"type":"durable-object","storage":"sqlite"}. Wrangler has the same fallback, with a comment saying such classes "fall back to the namespaces list API" (packages/wrangler/src/preview/containers.ts#L119-L149).

No other lookup returns the id today:

• The namespaces list documents only page and per_page (durable-objects-namespace-list-namespaces). script, class, name, preview_id and search are ignored: the response is the same unfiltered list.
• The Preview, its deployments, the Worker's versions and its script settings do not include the namespace id for an unbound class.

So reading every page is the fix available in cf now. A script or preview_id filter on the list, or namespace ids in the deployment's exports, would remove the need to list at all.

Suggested fix

Add a fetchPagedListResultBase to workers-utils with Wrangler's page loop, which the TODO in deploy-context.ts names, and use it for fetchPagedListResult in deploy-context.ts. Until then, cf can build the loop from workers-utils exports (fetchInternalBase, hasMorePages, throwFetchError):

fetchPagedListResult: async (complianceConfig, resource, init = {}, queryParams) => {
	const results = [];
	for (let page = 1; ; page++) {
		queryParams = new URLSearchParams(queryParams);
		queryParams.set("page", String(page));
		const { response: json, status, retryAfterMs } = await fetchInternalBase(
			complianceConfig, resource, init, userAgent, logger, queryParams, undefined, credentials
		);
		if (!json.success) {
			throwFetchError(resource, json, status, retryAfterMs);
		}
		results.push(...json.result);
		if (!hasMorePages(json.result_info)) {
			return results;
		}
	}
},

We patched this logic into the 1.0.0-beta.12 dist as a workaround. With it, the namespace lookup lists all namespaces, finds the missing Preview namespaces, and the Preview deploys succeed. fetchListResult keeps the cursor loop.

Related: #20 (list commands return only the first page), #120 (warn when list responses have more pages), #97 / #98 (--zone lookup past the first page of zones).