fix(storage): decouple details context, invalidate cache on refresh, and add singleflight - #2950
Open
Vurliy wants to merge 2 commits into
Open
fix(storage): decouple details context, invalidate cache on refresh, and add singleflight#2950Vurliy wants to merge 2 commits into
Vurliy wants to merge 2 commits into
Conversation
added 2 commits
August 20, 2026 06:12
16 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary / 摘要
1. Problem & Symptoms / 现状与问题现象
Symptom 1 (Multi-drive quota missing on home/manage pages) / 现象 1(多网盘容量在首页和管理页显示为
-):When multiple cloud storages (such as OneDrive, Google Drive, Baidu Netdisk, or high-latency remote storage) are mounted, opening the home page (
/) or the storage management list often displays-instead of the storage capacity/quota progress bar. Users have to manually navigate into each specific folder and refresh to temporarily trigger a quota query (as reported in OpenList无法显示各个网盘的容量大小了 #1815, [BUG] 别名容量显示错误 #2633, [BUG] 别名容量显示错误 #2635).挂载多个云盘(如 OneDrive、Google Drive、百度网盘或跨国高延迟存储)时,进入首页或管理后台存储列表,大部分网盘在“大小”列中均显示为
-而非容量进度条。用户必须手动点进各个子文件夹并刷新才能临时触发容量获取(参见 OpenList无法显示各个网盘的容量大小了 #1815、[BUG] 别名容量显示错误 #2633、[BUG] 别名容量显示错误 #2635)。Symptom 2 (Force refresh abortion & missing background caching) / 现象 2(强刷时后台任务被强行中断且无法落入缓存):
When clicking the bottom-right refresh button on the root directory (
refresh: true), drives that take longer than 1 second to respond return-. More importantly, waiting does not result in the cache being populated because the background tasks are terminated immediately after the frontend request finishes.在根目录点击右下角强刷时,响应超过 1 秒的网盘返回
-。但更严重的是,等待后再次刷新依然无法获得数据,因为后台请求在前端响应结束的瞬间就被直接中断了。Symptom 3 (Stale cache confusion on force refresh & immediate F5) / 现象 3(强刷后立即刷新页面导致旧缓存混淆与数据不一致):
When clicking the bottom-right force refresh button, unfinished drives initially return
-. If the user immediately right-clicks / refreshes the page (F5) while background requests are still in-flight, the backend (lacking explicit cache invalidation) serves the old stale cache from before the refresh. The user cannot distinguish whether the displayed quota is fresh or old cached data, violating data consistency expectations.用户点击右下角强刷按钮后,尚未完成探测的网盘会先返回
-。若用户在后台请求仍在进行时立即右键刷新(F5),由于缺乏显式的缓存失效机制,后端会直接返回刷新前的旧缓存数据。用户根本无法分辨当前看到的容量是最新探测到的结果还是之前的历史旧缓存,违背了数据一致性预期。2. Root Cause Analysis / 核心根因深度分析
Premature Context Cancellation / 请求 Context 过早取消:
In
internal/op/storage.go(GetStorageVirtualFilesWithDetailsByPath), the background goroutines executingGetStorageDetails(ctx, dri, refresh)directly inherit the incoming HTTP request'sc.Request.Context(). To ensure fast initial page load, the frontend only waits up to 1 second (time.After(time.Second)). As soon as the 1-second deadline elapses and the HTTP response completes, the web framework automatically cancelsc.Request.Context(). Any in-flight network requests (e.g., to Microsoft Graph API or remote storage) are immediately aborted withcontext canceled, preventing the background goroutines from completing and writing the fresh quota intodetailCache.在
internal/op/storage.go的GetStorageVirtualFilesWithDetailsByPath中,并发后台 Goroutine 调用GetStorageDetails时直接继承了 HTTP 请求的c.Request.Context()。为了保证首屏秒开,前台最多等待 1 秒。一旦 1 秒超时到达,HTTP 响应结束,框架立即自动发出cancel()信号,导致后台正在进行的跨国网络请求被强制以context canceled中断,未能跑完并写入detailCache。OneDrive Token Refresh Blocked by
noRetry: true/ OneDrive 驱动禁止自动刷新令牌:In
drivers/onedrive/util.go(getDrive) anddrivers/onedrive_app/util.go(getDrive),getDrive()callsd.Request(api, http.MethodGet, ..., &resp, true). The 4th argumentnoRetry = trueexplicitly disables automatic token renewal. When an account has been idle for more than 1 hour (Microsoft OAuth AccessToken expired), Microsoft Graph API responds with401 InvalidAuthenticationToken. BecausenoRetryistrue, the driver immediately fails and refuses to invokerefreshToken(), causing all quota queries for idle OneDrive accounts to fail permanently.在
drivers/onedrive/util.go的getDrive与drivers/onedrive_app/util.go的getDrive中,请求调用传递了noRetry = true。当账号闲置超过 1 小时(微软 AccessToken 过期)时,微软返回401 InvalidAuthenticationToken,驱动因禁止重试而直接报错放弃,根本不会去调用refreshToken()换取新令牌,导致闲置账号的容量探测全部失败。Lack of Explicit Cache Invalidation on Force Refresh / 强刷时缺少显式缓存失效机制:
When a force refresh (
refresh: true) is triggered, the existingdetailCachewas not explicitly purged. If an immediate follow-up non-force request (e.g., F5) arrived while the background probing was still in progress,GetStorageDetailswould hit the staledetailCacheand return outdated numbers instead of indicating that probing was underway.当触发强刷(
refresh: true)时,系统未显式清除原有的detailCache。若用户在后台探测期间发起常规请求(如 F5 刷新),GetStorageDetails会直接命中未失效的旧缓存,导致返回陈旧数据,而非反映当前正在探测的状态。3. What Was Fixed / 修复与改造内容
Context Decoupling with Bounded Timeout / 上下文解耦与独立超时保护:
Decoupled the context in
GetStorageVirtualFilesWithDetailsByPathusingbgCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), timeoutSec)(storage_details_timeout_seconds, default 15s). The foreground retains the 1-second fast response deadline (zero UI blocking), while the background goroutine is granted an independent 15-second lifetime to finish fetching data and write intodetailCache.使用
context.WithoutCancel配合独立超时保护(默认 15 秒)解绑上下文。前台保持 1 秒快速响应(保证页面不卡顿),后台 Goroutine 拥有独立生命周期平稳跑完并落入 30 分钟缓存。Automatic OAuth Token Refresh in OneDrive / 修复 OneDrive 驱动自动换 Token:
Removed
noRetry = trueinonedrive.getDrive()andonedrive_app.getDrive(), allowingd.Request()to automatically captureInvalidAuthenticationToken, invokerefreshToken(), and retry the quota query seamlessly.移除了
getDrive()中的noRetry: true参数,使底层自动捕获 401 错误并自动换取新 Token 重试。Strict Invalidate-On-Refresh & Singleflight Cooldown / 强刷显式失效与 Singleflight 保护:
Explicitly invalidate stale
detailCacheon force refresh (Cache.InvalidateStorageDetails) to ensure data consistency, combined withdetailsG singleflight.Groupto coalesce concurrent requests, and added a configurable settingstorage_details_cooldown_seconds(default 0s, preserving native behavior).强刷时主动失效旧缓存(
Cache.InvalidateStorageDetails)以保证数据强一致性,配合 Singleflight 并发去重,并增加了可配置项storage_details_cooldown_seconds(默认 0 秒,完全兼容官方原生预期)。Unit Testing / 完整单元测试:
Added
internal/op/storage_details_test.gocovering Singleflight coalescing, Invalidate-On-Refresh, and Cooldown logic.添加了完整的 Go 单元测试。
4. Behavior After Fix / 修复后完整表现行为
Opening the home page or storage list responds in under 1 second. Fast storage quota is displayed instantly; slower cloud storages continue probing in the background without cancellation and write into the 30-minute cache upon completion.
首屏与列表请求在 1 秒内迅速返回,零页面卡顿;慢速网盘在后台平稳完成探测并写入 30 分钟缓存。
Idle OneDrive accounts automatically refresh expired tokens on demand, eliminating permanent
-displays.长期闲置的 OneDrive 账号在探测时自动静默刷新令牌,彻底消除永久
-现象。Clicking the bottom-right refresh button purges the old cache immediately. In-flight drives consistently return
-(even during rapid F5 page refreshes). Once the background probe finishes (typically 2-3s), subsequent page visits immediately display 100% fresh and accurate storage quota.点击右下角强刷立即清除旧缓存;在后台探测完成前,任何常规刷新均一致性返回
-,绝不回退返回旧缓存;探测完成后,后续刷新立即秒级呈现 100% 真实最新的容量数据。High-frequency concurrent requests are merged into a single upstream call, protecting external cloud storage APIs.
高频并发请求自动合并为单个底层探测,有效保护外部网盘 API。
Related repository PRs / 关联仓库 PR:
Related Issues / 关联 Issue
Testing / 测试
go test ./internal/op -run TestGetStorageDetailsChecklist / 检查清单
gofmt/go fmt.AI Disclosure / AI 使用声明
Tools used / 使用工具:
Usage scope / 使用范围:
Code generation / 代码生成
Refactoring / 重构
Tests / 测试
Review assistance / 审查辅助
I have reviewed and validated all AI-assisted content included in this PR. / 我已审核并验证此 PR 中的所有 AI 辅助内容。
I have ensured that all AI-assisted commits include
Co-Authored-Byattribution. / 我已确保所有 AI 辅助提交都包含Co-Authored-By归属信息。I can reproduce all AI-assisted content included in this PR without any AI tools. / 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。