|
| 1 | +# crypto.DigestStream |
| 2 | + |
| 3 | +An informal specification of the Cloudflare-specific DigestStream (a |
| 4 | +WritableStream subclass computing a hash digest) as implemented in workerd, |
| 5 | +derived from — and kept in lockstep with — the test suite in this |
| 6 | +directory. **The tests are the normative artifact.** Both the C++ |
| 7 | +implementation (`src/workerd/api/crypto/crypto.{h,c++}`) and the TypeScript |
| 8 | +implementation (`src/per_isolate/crypto/digest-stream.ts`, behind |
| 9 | +`typescript_implemented_streams`) are covered. Both drive the SAME native |
| 10 | +digest context (`utils.createDigestContext` → CRC/OpenSSL contexts and the |
| 11 | +WTF-8/toWellFormed string encoder), so hashing, string encoding, and byte |
| 12 | +counting are parity by construction; divergences live in the stream |
| 13 | +wrapper and promise plumbing. DigestStream is non-standard: no WPT exists. |
| 14 | + |
| 15 | +## Interface |
| 16 | + |
| 17 | +```webidl |
| 18 | +[Exposed=Worker] |
| 19 | +interface DigestStream : WritableStream { |
| 20 | + constructor((DOMString or HashAlgorithm) algorithm, |
| 21 | + optional DigestStreamOptions options = {}); |
| 22 | + readonly attribute Promise<ArrayBuffer> digest; // memoized identity |
| 23 | + readonly attribute bigint bytesWritten; |
| 24 | +}; |
| 25 | +dictionary DigestStreamOptions { boolean toWellFormed = false; }; |
| 26 | +``` |
| 27 | + |
| 28 | +- Algorithms (case-insensitive over the whole name, string or `{name}` |
| 29 | + object): `md5`, `SHA-1`, `SHA-256`, `SHA-384`, `SHA-512`, `crc32`, |
| 30 | + `crc32c`, `crc64nvme`. Unknown names throw `NotSupportedError` |
| 31 | + synchronously; the options bag is type-checked BEFORE the algorithm |
| 32 | + lookup (primitive bags → TypeError "constructor parameter 2 is not of |
| 33 | + type 'Options'."); `toWellFormed` is ToBoolean-coerced. |
| 34 | +- **Inheritance divergence (ledger #1):** instances are |
| 35 | + `instanceof WritableStream` and the instance prototype chain is wired in |
| 36 | + both, but only TypeScript's genuine `class extends` links the |
| 37 | + CONSTRUCTOR's prototype to the WritableStream function; the C++ jsg |
| 38 | + static chain does not. Subclassing via `class X extends |
| 39 | + crypto.DigestStream` works in both. |
| 40 | +- `digest` and `bytesWritten` are brand-checked prototype accessors |
| 41 | + (`digest` moves to a per-instance own property in the unflagged legacy |
| 42 | + era); `Symbol.dispose` is brand-checked; `[object DigestStream]` |
| 43 | + branding under `set_tostring_tag`. |
| 44 | + |
| 45 | +## Core semantics |
| 46 | + |
| 47 | +- **Write settlement:** the hash update runs inside write() (chunks are |
| 48 | + consumed before the write settles — post-write mutation or detach cannot |
| 49 | + change the digest; metadata comes from internal slots). Chunks: |
| 50 | + ArrayBuffer, any view (offsets honored), strings; bare |
| 51 | + SharedArrayBuffers reject (views onto them are accepted); everything |
| 52 | + else rejects TypeError "DigestStream is a byte stream but received an |
| 53 | + object of non-ArrayBuffer/ArrayBufferView/string type on its writable |
| 54 | + side." Zero-length writes are no-ops. |
| 55 | +- **String encoding:** WTF-8 per chunk by default (lone surrogate = ED A0 |
| 56 | + 80 — deliberately NOT TextEncoder's substitution; pinned for existing |
| 57 | + digests). `toWellFormed: true` substitutes U+FFFD with |
| 58 | + TextEncoderStream-style stateful pairing across chunks and a close-time |
| 59 | + flush of a dangling lead. `bytesWritten` counts UTF-8 bytes (bigint in |
| 60 | + every era). |
| 61 | +- **Digest promise:** memoized identity (same object every access, before |
| 62 | + and after settling); resolves on close() with the digest ArrayBuffer; |
| 63 | + rejects on abort(reason) with the reason; never settled by abandonment |
| 64 | + or GC. close() is single-use (second close rejects, digest unaffected); |
| 65 | + writes after close reject without disturbing the digest. |
| 66 | +- **`Symbol.dispose`:** errors the digest ("The DigestStream was |
| 67 | + disposed.") without touching stream state — the stream stays unlocked, |
| 68 | + zero-length writes still resolve, non-empty writes reject; idempotent; |
| 69 | + a no-op after close. **Rejection reporting diverges (ledger #2):** the |
| 70 | + TypeScript implementation marks the digest promise handled, so an |
| 71 | + abandoned rejected digest produces NO unhandledrejection report; C++ |
| 72 | + reports it. A derived promise (`.then()` with no catch) still reports in |
| 73 | + both. |
| 74 | +- **Pipes:** a valid pipeTo destination from user streams, transforms, and |
| 75 | + Response bodies (brand checks pass in both implementations). |
| 76 | + |
| 77 | +## Compatibility flags |
| 78 | + |
| 79 | +| Flag (enable date) | Selects | Unflagged behavior tested by | |
| 80 | +| --- | --- | --- | |
| 81 | +| `workers_api_getters_setters_on_prototype` (2022-01-31) | `digest` as prototype accessor | `legacyDigestIsOwnInstanceProperty` | |
| 82 | +| `set_tostring_tag` (2024-09-26) | `[object DigestStream]` branding | `legacyToStringTag` | |
| 83 | +| `capture_async_api_throws` (2022-10-31) | pinned; the pre-flag invalid-chunk behavior (rejected promise ALSO reported as an uncaught exception even when handled) cannot be pinned in the harness — see legacy-shape.js | — | |
| 84 | +| `streams_enable_constructors` + `transformstream_enable_standard_constructor` (2022-11-30) | pipe tests build standard sources | — | |
| 85 | + |
| 86 | +`digest-cpp-pedantic.wd-test` runs the full module set with the dateless |
| 87 | +opt-in `pedantic_wpt` added, pinning the ABSENCE of pedantic effects |
| 88 | +(the implementation consults the flag nowhere; the reachable |
| 89 | +standard-streams machinery changes nothing the suite pins). |
| 90 | + |
| 91 | +## Divergence ledger (C++ vs TypeScript) |
| 92 | + |
| 93 | +| # | Area | C++ | TypeScript | Pinned in | |
| 94 | +| --- | --- | --- | --- | --- | |
| 95 | +| 1 | Constructor static inheritance | `getPrototypeOf(crypto.DigestStream) !== WritableStream` | `=== WritableStream` | `isRealWritableStreamSubclass` | |
| 96 | +| 2 | Abandoned digest rejection reporting | reported | marked handled, not reported | `abandonedDigestReporting` | |
| 97 | + |
| 98 | +## Assertion catalogue |
| 99 | + |
| 100 | +| Module | Asserts | |
| 101 | +| --- | --- | |
| 102 | +| `api-surface.js` | WritableStream subclassing incl. ledger #1; branding; brand-checked accessors + dispose; subclassable | |
| 103 | +| `construction.js` | case-insensitive algorithm matching (digest-compared); CRC exact spelling; unknown/missing algorithms throw synchronously; `{name}` object form; all 8 algorithms with sizes; option-bag Web IDL rules with exact TypeError; type-check-before-lookup ordering | |
| 104 | +| `digest-vectors.js` | pinned md5/SHA-256/crc32 outputs for bytes, strings, non-Uint8 views, offset subarrays; AWS SDK checksum vectors; mixed-type write accumulation with bytesWritten; shared digestOf helper | |
| 105 | +| `string-encoding.js` | WTF-8-not-TextEncoder default; toWellFormed matches TextEncoder for every lone-surrogate shape; inert for valid input and byte chunks; identical bytesWritten; every falsy spelling defaults off; ToBoolean coercion; stateful pair-joining across chunks; dangling-lead flush; per-chunk default encoding | |
| 106 | +| `chunk-types.js` | non-byte chunk rejection with exact message; bare-SAB rejection + SAB-view acceptance; DataView offset honored | |
| 107 | +| `digest-promise.js` | promise identity stability across settling; bytesWritten bigint semantics incl. zero-length and UTF-8 counting | |
| 108 | +| `lifecycle.js` | close resolves; abort rejects (incl. after writes); write-after-close rejects without disturbing the digest; double close safe; abandoned stream/writer safe; unused stream safe | |
| 109 | +| `dispose.js` | dispose errors digest + non-empty writes while stream state untouched; zero-length writes still resolve; idempotent; no-op after close | |
| 110 | +| `unhandled-rejection.js` | ledger #2 reporting matrix + derived-promise reporting | |
| 111 | +| `buffer-lifecycle.js` | consume-at-write: post-write mutation/detach invisible; lying metadata getters never consulted | |
| 112 | +| `pipe-integration.js` | pipeTo from user streams; TransformStream chain; Response body | |
| 113 | +| `large-payload.js` | 1MB+ chunk digesting | |
| 114 | +| `gc-interplay.js` | GC never settles an abandoned digest; writer keeps a collected wrapper operable | |
| 115 | +| `legacy-shape.js` | unflagged era: `[object Object]`; digest as own instance property (no prototype accessor); bytesWritten stays a bigint prototype accessor; flow unchanged | |
| 116 | +| `which-impl.js` | implementation detection | |
0 commit comments