Skip to content

fix: use JSON serialization for transfer results and progress - #840

Draft
pppanghu77 wants to merge 37 commits into
linuxdeepin:masterfrom
pppanghu77:agent/agent/97b0f79af349
Draft

pppanghu77 wants to merge 37 commits into
linuxdeepin:masterfrom
pppanghu77:agent/agent/97b0f79af349

Conversation

@pppanghu77

Copy link
Copy Markdown
Contributor

Root Cause Analysis

The data transfer tool serializes transfer results between Windows and UOS using a space-delimited format (name success reason;). However, name and reason are translated strings that may contain spaces. The receiver splits by ; then by space and requires exactly 3 fields, silently discarding entries with more than 3 fields. The same defect exists in transfer_content progress notifications: the sender uses space-delimited type content progress estimated; and the receiver requires exactly 4 fields.

Key evidence: resultdisplay.cpp:110 joins with spaces; transferworker.cpp:356 splits and discards when field count ≠ 3. Reproduction showed 100% loss for entries with spaces (before.log), and 0% loss with JSON (rehearsal.log).

Fix Approach

Replace the space-delimited custom serialization with JSON format. The sender (addResult, updateProcess) uses QJsonObject/QJsonArray + QJsonDocument::toJson(Compact); the receiver (handleMiscMessage) uses QJsonDocument::fromJson to parse. Both sides are modified synchronously; function signatures remain unchanged. This matches the analysis report's recommended fix (section 10).

Change Safety Assessment

Code Safety

  • Risk Level: Low
  • Only internal serialization/deserialization logic changed; no function signatures, public members, or class interfaces modified. All callers (TransferHelper signal-slot connections) are unaffected.
  • Blame shows target code introduced by ffeb1c6b (IPC refactor, 2023-10-25); this change does not revert any historical fix. Associated PMS bugs (246087, 228367, 234177) are retransfer/UI/network issues unrelated to serialization format.

Business Impact Scope

  • Data transfer result display — post-transfer success/failure entry list (resultdisplay.cpp addResult, transferworker.cpp handleMiscMessage add_result parsing)
  • Transfer progress notification — real-time progress updates to peer (transferringwidget.cpp updateProcess, transferworker.cpp handleMiscMessage transfer_content parsing)

Verification Suggestion

Test Windows→UOS and UOS→Windows data transfer with items whose names contain spaces; verify result list consistency and progress notification display on both sides.


根因分析

数据迁移工具在 Windows 与 UOS 之间传输完成结果时,使用空格分隔的 name success reason; 格式序列化,但 name 和 reason 是翻译字符串可能包含空格。接收方按 ; 再按空格拆分并要求恰好 3 个字段,超过 3 个字段的条目被静默丢弃。transfer_content 进度通知存在相同缺陷。

关键证据:resultdisplay.cpp:110 用空格拼接;transferworker.cpp:356 拆分后字段数 ≠ 3 则丢弃。复现实验含空格条目 100% 丢失,JSON 方案 0% 丢失。

修复方案

将空格分隔的自定义序列化替换为 JSON 格式。发送方使用 QJsonObject/QJsonArray + QJsonDocument::toJson(Compact),接收方使用 QJsonDocument::fromJson 解析。两端同步修改,函数签名不变。与分析报告第十节方案一致。

改动安全评估

代码安全评估

  • 风险等级: 低风险
  • 仅修改函数内部序列化/反序列化逻辑,不改变函数签名、公开成员变量或类接口,调用方不受影响
  • blame 显示目标代码由 ffeb1c6b(IPC 重构)引入,本次改动不撤销任何历史修复;关联 PMS bug(246087、228367、234177)均为重传/UI/网络问题,与序列化格式无关

业务影响范围

  • 数据迁移结果展示 — 迁移完成后成功/失败条目列表(resultdisplay.cpp addResult、transferworker.cpp handleMiscMessage add_result 解析)
  • 迁移进度通知 — 迁移过程中实时通知对端进度(transferringwidget.cpp updateProcess、transferworker.cpp handleMiscMessage transfer_content 解析)

验证建议

测试 Windows→UOS 和 UOS→Windows 数据迁移(含名称带空格的条目),验证结果列表一致性和进度通知显示。

pppanghu77 and others added 30 commits June 25, 2024 14:03
 [build] fix build

Log: [build] fix build
Task: https://pms.uniontech.com/task-view-354339.html
new tag v0.8.5, fix frontend launch issue for win.

Log: v0.8.5 override other incompatible version.
fix Multi user startup test

Log: fix Multi user startup test
Bug: https://pms.uniontech.com/bug-view-256723.html
[start] Change the startup sequence of the front-end and back-end

Log: [start] Change the startup sequence of the front-end and back-end
Bug: https://pms.uniontech.com/bug-view-261983.html
new tag v0.8.6

Log: new tag v0.8.6
[translation] add bo ug translation

Log: [translation] add bo ug translation
Fix title icon issue which caused by DTK updated.

Log: Fix title icon issue.
Bug: https://pms.uniontech.com/bug-view-264627.html
v0.8.7 release.

Log: v0.8.7 release.
…offline

Avoiding the impact of network fluctuations

Log: Added the option to manually disconnect the other party offline
Task: https://pms.uniontech.com/task-view-357177.html
Fix List Status and move the log location to/etc

Log: Fix List Status and move the log location to/et
Task: https://pms.uniontech.com/task-view-357177.html
v0.8.8 release.

Log: v0.8.8 release.
Modify the storage location of log level files

Log: Modify the storage location of log level files
Task: https://pms.uniontech.com/story-view-36529.html
Fix translation and UI issues

Log: Fix translation and UI issues
Bug: https://pms.uniontech.com/bug-view-250549.html
libcoost creates persistent static resources for each thread, which can cause memory leaks when threads are frequently created and released
using a thread pool to avoid frequent creation and destruction of threads
Log: Fix memory leaks
Bug: https://pms.uniontech.com/bug-view-266935.html
The daemon is renamed dde operation daemon

Log: The daemon is renamed dde operation daemon
Bug: https://pms.uniontech.com/bug-view-266935.html
v0.8.9 release.

Log: v0.8.9 release.
The default number of thread pools is too low, resulting in communication failure

Log: [thread] Increase the initial number of threads in threadpool
Bug: https://pms.uniontech.com/bug-view-273695.html
…nnection

Fix the issue of disconnection after network disconnection

Log: Fix the issue of disconnection after network disconnection
Bug: https://pms.uniontech.com/bug-view-273833.html
Record the searched devices and provide the option to minimize pallets

Log: [optimization] Optimize user experience
Bug: https://pms.uniontech.com/bug-view-250143.html
fix the password for the connection page is not complete in tibetan

Log: fix the password display is incomplete
Bug: https://pms.uniontech.com/bug-view-250623.html
Probability crash upon awakening

Log: Probability crash upon awakening
Bug: no
Unable to open from launcher after closing

Log: fix some ui bug
Bug: https://pms.uniontech.com/bug-view-282501.html
Implement a check in the setFile function to ensure that file paths do not exceed two directory levels. This change modifies the filename to retain only the last two directories and the file name itself, enhancing path management.

Log: Filter file paths to limit depth to two directories.
…P retrieval

Replace direct usage of QNetworkInterface::allAddresses() with
deepin_cross::CommonUitls::getFirstIp() to get the first available IP
address.
Log: fix: (data-transfer) replace QNetworkInterface with CommonUtils for IP retrieval
Bug: https://pms.uniontech.com/bug-view-341249.html
…rm transfer mechanism

- Add security checks in `getSaveFullpath` to prevent path traversal attacks by validating
  filenames and ensuring they are within allowed directories using realpath resolution.
- Introduce transfer confirmation mechanism in `JobManager` to require user approval before
  accepting incoming file transfers.
- Add methods to confirm, remove, and check transfer confirmation status for apps.
- Update `SendRpcService` to handle transfer confirmation and refusal messages.
- Include cleanup of confirmation status when jobs finish, cancel, or fail.

Log: fix(security): add path traversal protection in TransferJob and confirm transfer mechanism
Bug: https://pms.uniontech.com/bug-view-343709.html
Replace the library's dependencies with a method that displays the calling dynamic library.

Log: Remove displayjack kvm depend.
- Add file transfer confirmation status when password authentication succeeds
- Allow subsequent file transfers by calling JobManager::confirmTransfer
- This enables file operations after remote login is authenticated

Log: fix(rpc): add file transfer confirmation after successful authentication
Bug: https://pms.uniontech.com/bug-view-354299.html
pppanghu77 and others added 6 commits April 7, 2026 12:38
- Remove the transfer confirmation cleanup on job finished/canceled/failed
- Keep the confirmation state alive for the entire session to allow
  subsequent transfers without re-authentication
- This fixes the network error when transferring files again after
  returning from a completed transfer

Log: fix(transfer): 传输完成后点击返回再次传输提示网络错误
Bug: https://pms.uniontech.com/bug-view-355819.html
…emove transfer.json

- Fix storage directory for data-transfer: place target subdirectory under ~/Downloads
  instead of directly under home directory
- Change cleanup behavior to only remove transfer.json instead of the entire directory

Log: fix(data-transfer): save files to ~/Downloads subdirectory and only remove transfer.json
Bug: https://pms.uniontech.com/bug-view-356089.html
sync by github workflow

Log: none
Influence: none
… prompt on app installation

- Add initSecurityLoaderReport() in main.cpp: when started by deepin-security-loader (--fd1/--fd2 passed), report the system bus unique name and the com.deepin.lastore service list to the loader in JSON so it registers the lastore allow-caller whitelist on our behalf, making InstallPackage free of polkit authentication
- Call the report before any early-exit branch (including single-instance failure), otherwise the loader times out and kills the process, which shows up as a crash on startup
- Add launcher script /usr/bin/deepin-data-transfer: start the real binary /usr/libexec/deepin/deepin-data-transfer through deepin-security-loader with the deepin-daemon group when available (dbus >= 1.12.20.17-deepin1), otherwise fall back to direct launch with the historical polkit behavior
- Enable Qt DBus component/linking and install the real binary to libexec/deepin with the launcher taking over /usr/bin
- Update debian packaging: install the libexec binary and add Recommends: deepin-security-loader

修复(data-transfer): 经 deepin-security-loader 启动以避免安装应用时弹出 polkit 鉴权框

- main.cpp 新增 initSecurityLoaderReport():经 deepin-security-loader 启动(带 --fd1/--fd2 参数)时,将 system bus 唯一连接名和 com.deepin.lastore 服务列表以 JSON 回报给 loader,由其代为注册 lastore 的 allow-caller 白名单,使 InstallPackage 等接口免 polkit 鉴权
- 回报调用置于任何提前退出分支(含单例失败)之前,否则 loader 会超时结束子进程,表现为启动闪退
- 新增启动脚本 /usr/bin/deepin-data-transfer:环境满足(dbus >= 1.12.20.17-deepin1)时经 deepin-security-loader 附加 deepin-daemon 组启动真实二进制 /usr/libexec/deepin/deepin-data-transfer,否则回退直接启动,鉴权行为与历史一致
- 启用 Qt DBus 组件与链接,真实二进制安装至 libexec/deepin,/usr/bin 由启动脚本接管
- 更新 debian 打包:安装 libexec 二进制并增加 Recommends: deepin-security-loader

Log: 数据迁移工具经 deepin-security-loader 附加 deepin-daemon 组启动并回报 bus 连接名,实现 lastore 安装接口免鉴权,修复迁移三方应用时弹出密码认证框的问题
Bug: https://pms.uniontech.com/bug-view-375325.html
…n different filesystems

- Move directories with QDir::rename instead of QFile::rename, which fails with EISDIR on directories
- Fall back to recursive copy plus source removal when rename fails with EXDEV across filesystems: copyDirectory() for directories and QFile::copy for single files
- Fix rename-collision handling: dst.remove(fileName) deleted every occurrence of the name and corrupted paths with repeated segments; only the trailing segment is cut now
- Ensure the destination parent directory exists before moving, for targets such as ~/D:/ derived from Windows drive letters

修复(数据迁移): 修复源和目标位于不同文件系统时 moveFile 搬运失败的问题

- 目录改用 QDir::rename 移动,避免 QFile::rename 移动目录时报 EISDIR 错误
- rename 跨文件系统失败(EXDEV)时退化为复制后删除源:目录走 copyDirectory() 递归复制,单文件走 QFile::copy
- 修复重名避让逻辑: dst.remove(fileName) 会误删路径中所有同名段导致路径破坏,改为只截取结尾段
- 移动前确保目标父目录存在,兼容 ~/D:/ 这类源自 Windows 盘符的目标路径

Log: 修复数据迁移搬运阶段接收目录与主目录不在同一文件系统时 rename 跨设备失败(EXDEV)导致搬运失败的问题
Bug: https://pms.uniontech.com/bug-view-375881.html
- In domain-account environments the loginctl session username may differ
  from the home directory name (e.g. "@xxx"), and the name comparison
  failed, making the daemon exit right after startup (inactive desktop session)
- isActiveUser() now compares the active desktop session UID against getuid(),
  decoupled from the user naming scheme; when multiple desktop sessions are
  active, only the one belonging to the current user is taken as the active session
- KEY_CURRENT_ACTIVE_USER now stores the active session UID and the
  ServiceManager runtime check compares UIDs accordingly
- Guard the loginctl/getuid() based session check with _WIN32 so the daemon
  still compiles on Windows, where the check is not applicable
- Update SPDX copyright year to 2023-2026 for the modified files

修复(daemon): 改用 UID 而非用户名判定活动桌面会话

- 域账号场景下 loginctl 会话用户名与家目录名可能不一致(如 "@xxx"),
  名字比对失败导致 daemon 启动即退出(exit, inactive desktop session)
- isActiveUser 改为比对活动桌面会话 UID 与 getuid(), 与命名体系解耦;
  存在多个活跃桌面会话时, 以属于当前用户的会话为准
- KEY_CURRENT_ACTIVE_USER 改存活动会话 UID, ServiceManager 运行时检查同步按 UID 判定
- 活动会话检查增加 _WIN32 平台守卫, 修复 Windows 构建下 getuid() 编译失败的问题
- 更新所改文件的 SPDX 版权年份为 2023-2026

Log: 修复域账号环境下跨端协同 daemon 无法启动的问题

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, we are unable to review this pull request

The GitHub API does not allow us to fetch diffs exceeding 20000 lines

@pppanghu77
pppanghu77 marked this pull request as draft October 29, 2026 19:33
@deepin-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: pppanghu77

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@github-actions

Copy link
Copy Markdown

TAG Bot

TAG: 0.8.9
EXISTED: no
DISTRIBUTION: unstable

1. Root cause: sender serializes transfer results as "name success
   reason;" with space delimiter, but name and reason are translated
   strings that may contain spaces; receiver splits by ";" then " "
   and requires exactly 3 fields, discarding entries with >3 fields
2. Same defect in transfer_content: sender uses space-delimited
   "type content progress estimated;", receiver requires 4 fields
3. Fix: replace space-delimited serialization with JSON format in
   resultdisplay.cpp (addResult), transferringwidget.cpp (updateProcess),
   and transferworker.cpp (handleMiscMessage parsing for both paths)
4. Add QJsonParseError check and logging for JSON parsing in
   transferworker.cpp to aid debugging in mixed-version deployment
5. Impact: all transfer result entries now correctly survive
   serialization regardless of spaces in names or reasons

Log: fix transfer result list inconsistency between Windows and UOS

Influence:
1. Test data transfer from Windows to UOS, verify result list matches
2. Test data transfer from UOS to Windows, verify result list matches
3. Test transfer with items whose names contain spaces
4. Verify transfer progress notification displays correctly on both sides

fix: 使用JSON序列化传输结果和进度信息

1. 根因:发送方将传输结果序列化为"名称 成功 原因;"格式,以空格
   分隔,但名称和原因是翻译字符串可能包含空格;接收方按";"再按
   空格拆分并要求恰好3个字段,超过3个字段的条目被静默丢弃
2. transfer_content存在相同缺陷:发送方使用空格分隔的
   "类型 内容 进度 预估时间;",接收方要求4个字段
3. 方案:将空格分隔的序列化替换为JSON格式,涉及
   resultdisplay.cpp(addResult)、transferringwidget.cpp(updateProcess)、
   transferworker.cpp(handleMiscMessage两处解析路径)
4. 在transferworker.cpp的JSON解析路径增加QJsonParseError检查
   与日志输出,便于新旧版本混合部署时排查静默失败
5. 影响:所有传输结果条目无论名称或原因是否含空格均能正确
   完成序列化和反序列化

Log: 修复Windows与UOS间数据迁移后完成列表不一致的问题

Influence:
1. 测试从Windows到UOS的数据迁移,验证结果列表一致
2. 测试从UOS到Windows的数据迁移,验证结果列表一致
3. 测试名称包含空格的迁移条目
4. 验证迁移进度通知在两端均正常显示

Bug: https://pms.uniontech.com/bug-view-246975.html
@pppanghu77
pppanghu77 force-pushed the agent/agent/97b0f79af349 branch from 49f0a7d to 1311398 Compare October 29, 2026 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants