Skip to content

feat(google-api-core): add support for resumable uploads - #18352

Merged
parthea merged 81 commits into
mainfrom
feat/resumable-transfer-api-core
Sep 23, 2026
Merged

feat(google-api-core): add support for resumable uploads#18352
parthea merged 81 commits into
mainfrom
feat/resumable-transfer-api-core

Conversation

@parthea

@parthea parthea commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Towards b/457416314, b/556259599
gemini-code-assist[bot]
gemini-code-assist Bot reviewed Sep 11, 2026
View reviewed changes

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new resumable transfer library for Google APIs, implementing both synchronous (using requests) and asynchronous (using aiohttp) resumable upload sessions, supported by a sans-I/O protocol state machine and comprehensive tests. The feedback highlights several key areas for improvement: ensuring backward compatibility with Python 3.7/3.8 by replacing asyncio.to_thread with loop.run_in_executor, handling byte-type header keys in the state machine, rejecting unsupported str and dict stream types early, retrying timeouts globally in the synchronous session, and removing redundant deadline checks.
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_state.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Resolved
@parthea parthea changed the title [DRAFT] feat: add support for resumable uploads Sep 11, 2026
parthea and others added 9 commits September 11, 2026 21:58
@parthea
lint
326ae78
@parthea
lint
9936e0d
@parthea
mypy
9e689b8
@parthea
lint
9c49a81
@parthea
address review feedback
b0de580
@parthea
address review feedback
ae11c27
@parthea @gemini-code-assist
Update packages/google-api-core/google/api_core/resumable_transfer/up…
f57c598 
…load_state.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@parthea @gemini-code-assist
Update packages/google-api-core/google/api_core/resumable_transfer/up…
efbc623 
…load_async.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@parthea @gemini-code-assist
Update packages/google-api-core/google/api_core/resumable_transfer/up…
41a3759 
…load.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@parthea parthea changed the title [DRAFT] feat(google-api-core): add support for resumable uploads Sep 14, 2026
@parthea
parthea marked this pull request as ready for review September 14, 2026 19:39
@parthea
parthea requested a review from a team as a code owner September 14, 2026 19:39
daniel-sanche
daniel-sanche requested changes Sep 14, 2026
View reviewed changes

@daniel-sanche daniel-sanche left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still digesting this, but giving a quick first round of comments
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Resolved
daniel-sanche
daniel-sanche reviewed Sep 21, 2026
View reviewed changes

@daniel-sanche daniel-sanche left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its looking really close, but a couple more comments
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_state.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_state.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Resolved
daniel-sanche
daniel-sanche approved these changes Sep 22, 2026
View reviewed changes

@daniel-sanche daniel-sanche left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall LGTM, but a few final comments to consider
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_state.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Resolved
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Resolved
@parthea

parthea commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

All review comments have been addressed. I'm going to hold off on merging this until #18354 is green
@parthea

parthea commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

Merging. #18354 is close to green
@parthea
parthea merged commit 17d6c53 into main Sep 23, 2026
111 checks passed
@parthea
parthea deleted the feat/resumable-transfer-api-core branch September 23, 2026 20:19
@release-please release-please Bot mentioned this pull request Sep 23, 2026
Merged
sofisl pushed a commit that referenced this pull request Sep 24, 2026
@release-please
chore: release main (#18421)
c0208c6 
🤖 I have created a release *beep* *boop*
---


<details><summary>gapic-generator: 1.40.0</summary>

##
[1.40.0](gapic-generator-v1.39.0...gapic-generator-v1.40.0)
(2026-09-24)


### Features

* add 3.15 support to gapic library classifiers
([#18450](#18450))
([5fe1ef3](5fe1ef3))


### Bug Fixes

* **bazel:** allow CustomProtoInfo in gapic_compat_proto_library
([#18417](#18417))
([11bdf92](11bdf92))
</details>

<details><summary>google-ads-marketingplatform-admin: 0.6.1</summary>

##
[0.6.1](google-ads-marketingplatform-admin-v0.6.0...google-ads-marketingplatform-admin-v0.6.1)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-analytics-data: 0.23.2</summary>

##
[0.23.2](google-analytics-data-v0.23.1...google-analytics-data-v0.23.2)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-api-core: 2.39.0</summary>

##
[2.39.0](google-api-core-v2.38.0...google-api-core-v2.39.0)
(2026-09-24)


### Features

* **google-api-core:** add support for resumable uploads
([#18352](#18352))
([17d6c53](17d6c53))
</details>

<details><summary>google-apps-chat: 0.10.6</summary>

##
[0.10.6](google-apps-chat-v0.10.5...google-apps-chat-v0.10.6)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-agentregistry: 0.1.1</summary>

##
[0.1.1](google-cloud-agentregistry-v0.1.0...google-cloud-agentregistry-v0.1.1)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-bigtable: 2.47.0</summary>

##
[2.47.0](google-cloud-bigtable-v2.46.0...google-cloud-bigtable-v2.47.0)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-firestore: 2.32.0</summary>

##
[2.32.0](google-cloud-firestore-v2.31.0...google-cloud-firestore-v2.32.0)
(2026-09-24)


### Features

* **firestore:** add BSONBinary support
([#18389](#18389))
([85556ad](85556ad))
* **firestore:** add BSONDecimal128 support
([#18395](#18395))
([a04c831](a04c831))
* **firestore:** add BSONRegex support
([#18394](#18394))
([322c9bc](322c9bc))
* **firestore:** add BSONTimestamp support
([#18390](#18390))
([a1b2d63](a1b2d63))
</details>

<details><summary>google-cloud-geminidataanalytics: 0.13.4</summary>

##
[0.13.4](google-cloud-geminidataanalytics-v0.13.3...google-cloud-geminidataanalytics-v0.13.4)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-lustre: 0.4.2</summary>

##
[0.4.2](google-cloud-lustre-v0.4.1...google-cloud-lustre-v0.4.2)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-monitoring-dashboards: 3.1.0</summary>

##
[3.1.0](google-cloud-monitoring-dashboards-v3.0.0...google-cloud-monitoring-dashboards-v3.1.0)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-ndb: 2.6.1</summary>

##
[2.6.1](google-cloud-ndb-v2.6.0...google-cloud-ndb-v2.6.1)
(2026-09-24)


### Bug Fixes

* **google-cloud-ndb:** allow redis 8.x
([#18438](#18438))
([9630fe4](9630fe4))
</details>

<details><summary>google-cloud-parametermanager: 0.4.2</summary>

##
[0.4.2](google-cloud-parametermanager-v0.4.1...google-cloud-parametermanager-v0.4.2)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-storage: 3.15.0</summary>

##
[3.15.0](google-cloud-storage-v3.14.1...google-cloud-storage-v3.15.0)
(2026-09-24)


### Features

* **storage:** support storage_class in AsyncAppendableObjectWriter
([#18289](#18289))
([dc30df2](dc30df2))
* **storage:** support storage_class via Blob in
AsyncAppendableObjectWriter
([#18318](#18318))
([b0872f2](b0872f2))
</details>

<details><summary>google-cloud-tasks: 2.25.0</summary>

##
[2.25.0](google-cloud-tasks-v2.24.0...google-cloud-tasks-v2.25.0)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-vectorsearch: 0.11.4</summary>

##
[0.11.4](google-cloud-vectorsearch-v0.11.3...google-cloud-vectorsearch-v0.11.4)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>googleapis-common-protos: 1.75.4</summary>

##
[1.75.4](googleapis-common-protos-v1.75.3...googleapis-common-protos-v1.75.4)
(2026-09-24)


### Bug Fixes

* regenerate gencode with protobuf 6.33.5
([#18461](#18461))
([9cb21d0](9cb21d0))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants

@parthea @daniel-sanche