跳到内容
当前位置:首页>Excel>【拾光帖·智能体】第10讲|Excel 脏数据别急着算:去重/单位统一/分类映射,一步干净|WorkBuddy

【拾光帖·智能体】第10讲|Excel 脏数据别急着算:去重/单位统一/分类映射,一步干净|WorkBuddy

  • 2026-11-08 13:47:46
【拾光帖·智能体】第10讲|Excel 脏数据别急着算:去重/单位统一/分类映射,一步干净|WorkBuddy

你好,我是师兄。你有没有被脏数据气到过?下面的表格是否经常遇到?

性别列里男/M/1 混着写;同一批病人,住院号和姓名都一样,却有两行记录(重复入院被录了两次);血红蛋白有人写 g/L、有人写 g/dL;日期一半是 2024/3/5、一半是 2024-03-05。是不是看着就头大?

今天第 10 讲,师兄就教你怎么把「脏 Excel」先交给 WorkBuddy 洗干净,再去算——这一步做对,后面 结果才站得住。

脏数据清洗的底层链路——原始脏表 → 去重 / 单位统一 / 分类映射 → 干净表

先讲底层:AI 清洗不是「魔法」,是三步规则

你喂给WorkBuddy 一张表,它不会凭空「懂」哪行是重复、哪个单位该统一。它做的是你给的规则——所以喂它之前,你要先想清楚「脏在哪」。师兄把脏数据归成三类,对应三步:

  • · ① 重复行:同一病人被录了多次 → 按「住院号 + 入院日期」去重,保留一条。
  • · ② 单位不统一:g/L 和 g/dL 混写 → 你定一个标准(比如全文用 g/L),让它批量换算。
  • · ③ 分类乱写:男/M/1 都指男性 → 建一张「映射表」,统一成「男 / 女」。

记住一句话:清洗规则是你定的,AI 只是按你定的规则执行。规则不清楚,它就会「自作主张」。

一句话让 AI 不跑偏

上传「case_raw.xlsx」。这是一份病例资料原始表,存在脏数据,请按下面规则清洗,另存为「case_clean.xlsx」,不要覆盖原件: 1) 去重:按「住院号 + 入院日期」删除完全重复的行,报告删了几行; 2) 单位统一:血红蛋白全部换算为 g/L,计算公式g/L=(g/dL×10),并在新列「Hb_gL」给出,原列保留; 3) 分类映射:性别列「男/M/1」→「男」、「女/F/2」→「女」,无法识别的标「待核」; 4) 报告每列缺失值个数,缺失不要补数、不要删除整行,只标注。 先打印你识别到的「异常值清单」让我确认,再执行清洗。5 )日期→YYYY-MM-DD

3分钟之后,你就得到了上面这个清晰,指标统一的新表格了。如果你用传统方法自己一个个去核对查找,手动替换,修改,3小时能否做完?这就是AI时代下,会用工具为我们带来的便利。

师兄踩坑清单(脏数据的坑)

坑
表现
怎么避开
盲目去重
把「真的两次入院」当重复删了,n 虚低
去重键用「住院号+入院日期」而非只按姓名
单位硬乘
它把 g/dL 直接×100 当 g/L,差 10 倍
明确「g/dL×10 = g/L」,并要求先列换算公式
映射一刀切
「未知」被强行归到男/女
无法识别的单独标「待核」,交回你判断
覆盖原件
清洗后原表被改,没法回溯
强制「另存为新文件」,绝不覆盖

本讲实操案例(可复现)

场景:你有一份 186 例的原始表 case_raw.xlsx,含 4 行重复记录、性别混写(男/M/1)、血红蛋白 g/L 与 g/dL 混用。

环节
内容
输入文件
case_raw.xlsx(186 例,腹腔镜组 98 / 开腹组 88,含脏数据)
你的提示词
上面的提示词(含「先打印异常值清单」)
AI 产出
case_clean.xlsx:删 4 行重复→182 例待核;Hb 统一为 g/L;性别映射为男/女;各列缺失数报告
你核对
去重后 n、单位换算公式、性别映射表——三项你签字确认
交付
干净表再进第 7 讲的 Table 1 流程

一条不能松的底线

清洗规则是你定的,AI 只是执行。去重后 n 由你核对、映射表由你确认——写进论文的每一个数字,最终签字的是你,不是工具。

最后顺带让workbuddy帮忙做个基线表1,不也就是顺手的事情?

本讲小结

你学到了
记住这句
脏数据三类
重复行 / 单位乱 / 分类混
清洗三步
去重 → 统一单位 → 映射分类
防呆关键
先打印异常清单,确认再跑;另存不覆盖
适合谁
所有要处理原始 Excel 的科研人(无版本门槛)
好了,我是师兄 下课!练习表格已经同步到微信学习群,有需要的自取!!
基本 文件 流程 错误 SQL 调试
  1. 请求信息 : 2026-11-09 22:55:14 HTTP/1.1 GET : http://h.sjds.net/a/564694.html
  2. 运行时间 : 0.077976s [ 吞吐率:12.82req/s ] 内存消耗:4,217.20kb 文件加载:140
  3. 缓存信息 : 0 reads,0 writes
  4. 会话信息 : SESSION_ID=f533baaf7c97fe9c1389e8318d20a9d0
  1. /www/wwwroot/h.sjds.net/public/index.php ( 0.79 KB )
  2. /www/wwwroot/h.sjds.net/vendor/autoload.php ( 0.17 KB )
  3. /www/wwwroot/h.sjds.net/vendor/composer/autoload_real.php ( 2.49 KB )
  4. /www/wwwroot/h.sjds.net/vendor/composer/platform_check.php ( 0.90 KB )
  5. /www/wwwroot/h.sjds.net/vendor/composer/ClassLoader.php ( 14.03 KB )
  6. /www/wwwroot/h.sjds.net/vendor/composer/autoload_static.php ( 4.90 KB )
  7. /www/wwwroot/h.sjds.net/vendor/topthink/think-helper/src/helper.php ( 8.34 KB )
  8. /www/wwwroot/h.sjds.net/vendor/topthink/think-validate/src/helper.php ( 2.19 KB )
  9. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/helper.php ( 1.47 KB )
  10. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/stubs/load_stubs.php ( 0.16 KB )
  11. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Exception.php ( 1.69 KB )
  12. /www/wwwroot/h.sjds.net/vendor/topthink/think-container/src/Facade.php ( 2.71 KB )
  13. /www/wwwroot/h.sjds.net/vendor/symfony/deprecation-contracts/function.php ( 0.99 KB )
  14. /www/wwwroot/h.sjds.net/vendor/symfony/polyfill-mbstring/bootstrap.php ( 8.26 KB )
  15. /www/wwwroot/h.sjds.net/vendor/symfony/polyfill-mbstring/bootstrap80.php ( 9.78 KB )
  16. /www/wwwroot/h.sjds.net/vendor/symfony/var-dumper/Resources/functions/dump.php ( 1.49 KB )
  17. /www/wwwroot/h.sjds.net/vendor/topthink/think-dumper/src/helper.php ( 0.18 KB )
  18. /www/wwwroot/h.sjds.net/vendor/symfony/var-dumper/VarDumper.php ( 4.30 KB )
  19. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/App.php ( 15.30 KB )
  20. /www/wwwroot/h.sjds.net/vendor/topthink/think-container/src/Container.php ( 15.76 KB )
  21. /www/wwwroot/h.sjds.net/vendor/psr/container/src/ContainerInterface.php ( 1.02 KB )
  22. /www/wwwroot/h.sjds.net/app/provider.php ( 0.19 KB )
  23. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Http.php ( 6.04 KB )
  24. /www/wwwroot/h.sjds.net/vendor/topthink/think-helper/src/helper/Str.php ( 7.29 KB )
  25. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Env.php ( 4.68 KB )
  26. /www/wwwroot/h.sjds.net/app/common.php ( 0.03 KB )
  27. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/helper.php ( 18.78 KB )
  28. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Config.php ( 5.54 KB )
  29. /www/wwwroot/h.sjds.net/config/app.php ( 0.95 KB )
  30. /www/wwwroot/h.sjds.net/config/cache.php ( 0.78 KB )
  31. /www/wwwroot/h.sjds.net/config/console.php ( 0.23 KB )
  32. /www/wwwroot/h.sjds.net/config/cookie.php ( 0.56 KB )
  33. /www/wwwroot/h.sjds.net/config/database.php ( 2.48 KB )
  34. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/facade/Env.php ( 1.67 KB )
  35. /www/wwwroot/h.sjds.net/config/filesystem.php ( 0.61 KB )
  36. /www/wwwroot/h.sjds.net/config/lang.php ( 0.91 KB )
  37. /www/wwwroot/h.sjds.net/config/log.php ( 1.35 KB )
  38. /www/wwwroot/h.sjds.net/config/middleware.php ( 0.19 KB )
  39. /www/wwwroot/h.sjds.net/config/route.php ( 1.89 KB )
  40. /www/wwwroot/h.sjds.net/config/session.php ( 0.57 KB )
  41. /www/wwwroot/h.sjds.net/config/trace.php ( 0.34 KB )
  42. /www/wwwroot/h.sjds.net/config/view.php ( 0.82 KB )
  43. /www/wwwroot/h.sjds.net/app/event.php ( 0.25 KB )
  44. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Event.php ( 7.67 KB )
  45. /www/wwwroot/h.sjds.net/app/service.php ( 0.13 KB )
  46. /www/wwwroot/h.sjds.net/app/AppService.php ( 0.26 KB )
  47. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Service.php ( 1.64 KB )
  48. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Lang.php ( 7.35 KB )
  49. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/lang/zh-cn.php ( 13.70 KB )
  50. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/initializer/Error.php ( 3.31 KB )
  51. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/initializer/RegisterService.php ( 1.33 KB )
  52. /www/wwwroot/h.sjds.net/vendor/services.php ( 0.14 KB )
  53. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/service/PaginatorService.php ( 1.52 KB )
  54. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/service/ValidateService.php ( 0.99 KB )
  55. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/service/ModelService.php ( 2.04 KB )
  56. /www/wwwroot/h.sjds.net/vendor/topthink/think-trace/src/Service.php ( 0.77 KB )
  57. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Middleware.php ( 6.72 KB )
  58. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/initializer/BootService.php ( 0.77 KB )
  59. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/Paginator.php ( 11.86 KB )
  60. /www/wwwroot/h.sjds.net/vendor/topthink/think-validate/src/Validate.php ( 63.20 KB )
  61. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/Model.php ( 23.55 KB )
  62. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/model/concern/Attribute.php ( 21.05 KB )
  63. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/model/concern/AutoWriteData.php ( 4.21 KB )
  64. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/model/concern/Conversion.php ( 6.44 KB )
  65. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/model/concern/DbConnect.php ( 5.16 KB )
  66. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/model/concern/ModelEvent.php ( 2.33 KB )
  67. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/model/concern/RelationShip.php ( 28.29 KB )
  68. /www/wwwroot/h.sjds.net/vendor/topthink/think-helper/src/contract/Arrayable.php ( 0.09 KB )
  69. /www/wwwroot/h.sjds.net/vendor/topthink/think-helper/src/contract/Jsonable.php ( 0.13 KB )
  70. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/model/contract/Modelable.php ( 0.09 KB )
  71. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Db.php ( 2.88 KB )
  72. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/DbManager.php ( 8.52 KB )
  73. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Log.php ( 6.28 KB )
  74. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Manager.php ( 3.92 KB )
  75. /www/wwwroot/h.sjds.net/vendor/psr/log/src/LoggerTrait.php ( 2.69 KB )
  76. /www/wwwroot/h.sjds.net/vendor/psr/log/src/LoggerInterface.php ( 2.71 KB )
  77. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Cache.php ( 4.92 KB )
  78. /www/wwwroot/h.sjds.net/vendor/psr/simple-cache/src/CacheInterface.php ( 4.71 KB )
  79. /www/wwwroot/h.sjds.net/vendor/topthink/think-helper/src/helper/Arr.php ( 16.63 KB )
  80. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/cache/driver/File.php ( 7.84 KB )
  81. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/cache/Driver.php ( 9.03 KB )
  82. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/contract/CacheHandlerInterface.php ( 1.99 KB )
  83. /www/wwwroot/h.sjds.net/app/Request.php ( 0.09 KB )
  84. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Request.php ( 55.78 KB )
  85. /www/wwwroot/h.sjds.net/app/middleware.php ( 0.25 KB )
  86. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Pipeline.php ( 2.61 KB )
  87. /www/wwwroot/h.sjds.net/vendor/topthink/think-trace/src/TraceDebug.php ( 3.40 KB )
  88. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/middleware/SessionInit.php ( 1.94 KB )
  89. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Session.php ( 1.80 KB )
  90. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/session/driver/File.php ( 6.27 KB )
  91. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/contract/SessionHandlerInterface.php ( 0.87 KB )
  92. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/session/Store.php ( 7.12 KB )
  93. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Route.php ( 23.73 KB )
  94. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/route/RuleName.php ( 5.75 KB )
  95. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/route/Domain.php ( 2.53 KB )
  96. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/route/RuleGroup.php ( 22.43 KB )
  97. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/route/Rule.php ( 26.95 KB )
  98. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/route/RuleItem.php ( 9.78 KB )
  99. /www/wwwroot/h.sjds.net/route/app.php ( 1.72 KB )
  100. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/facade/Route.php ( 4.70 KB )
  101. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/route/dispatch/Controller.php ( 4.74 KB )
  102. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/route/Dispatch.php ( 10.44 KB )
  103. /www/wwwroot/h.sjds.net/app/controller/Index.php ( 4.81 KB )
  104. /www/wwwroot/h.sjds.net/app/BaseController.php ( 2.05 KB )
  105. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/facade/Db.php ( 0.93 KB )
  106. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/connector/Mysql.php ( 5.44 KB )
  107. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/PDOConnection.php ( 52.47 KB )
  108. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/Connection.php ( 8.39 KB )
  109. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/ConnectionInterface.php ( 4.57 KB )
  110. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/builder/Mysql.php ( 16.58 KB )
  111. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/Builder.php ( 24.06 KB )
  112. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/BaseBuilder.php ( 27.50 KB )
  113. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/Query.php ( 15.71 KB )
  114. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/BaseQuery.php ( 45.13 KB )
  115. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/concern/TimeFieldQuery.php ( 7.43 KB )
  116. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/concern/AggregateQuery.php ( 3.26 KB )
  117. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/concern/ModelRelationQuery.php ( 20.07 KB )
  118. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/concern/ParamsBind.php ( 3.66 KB )
  119. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/concern/ResultOperation.php ( 7.01 KB )
  120. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/concern/WhereQuery.php ( 19.37 KB )
  121. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/concern/JoinAndViewQuery.php ( 7.11 KB )
  122. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/concern/TableFieldInfo.php ( 2.63 KB )
  123. /www/wwwroot/h.sjds.net/vendor/topthink/think-orm/src/db/concern/Transaction.php ( 2.77 KB )
  124. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/log/driver/File.php ( 5.96 KB )
  125. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/contract/LogHandlerInterface.php ( 0.86 KB )
  126. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/log/Channel.php ( 3.89 KB )
  127. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/event/LogRecord.php ( 1.02 KB )
  128. /www/wwwroot/h.sjds.net/vendor/topthink/think-helper/src/Collection.php ( 16.47 KB )
  129. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/facade/View.php ( 1.70 KB )
  130. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/View.php ( 4.39 KB )
  131. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Response.php ( 8.81 KB )
  132. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/response/View.php ( 3.29 KB )
  133. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/Cookie.php ( 6.06 KB )
  134. /www/wwwroot/h.sjds.net/vendor/topthink/think-view/src/Think.php ( 8.38 KB )
  135. /www/wwwroot/h.sjds.net/vendor/topthink/framework/src/think/contract/TemplateHandlerInterface.php ( 1.60 KB )
  136. /www/wwwroot/h.sjds.net/vendor/topthink/think-template/src/Template.php ( 46.61 KB )
  137. /www/wwwroot/h.sjds.net/vendor/topthink/think-template/src/template/driver/File.php ( 2.41 KB )
  138. /www/wwwroot/h.sjds.net/vendor/topthink/think-template/src/template/contract/DriverInterface.php ( 0.86 KB )
  139. /www/wwwroot/h.sjds.net/runtime/temp/c1d6e8c94b2c2fa10e259903674c804d.php ( 8.11 KB )
  140. /www/wwwroot/h.sjds.net/vendor/topthink/think-trace/src/Html.php ( 4.42 KB )
  1. CONNECT:[ UseTime:0.000993s ] mysql:host=172.18.0.4;port=3306;dbname=h_sjds;charset=utf8mb4
  2. SHOW FULL COLUMNS FROM `fenlei` [ RunTime:0.001184s ]
  3. SELECT * FROM `fenlei` WHERE `fid` = 0 [ RunTime:0.000381s ]
  4. SELECT * FROM `fenlei` WHERE `fid` = 63 [ RunTime:0.000505s ]
  5. SHOW FULL COLUMNS FROM `set` [ RunTime:0.000517s ]
  6. SELECT * FROM `set` [ RunTime:0.000305s ]
  7. SHOW FULL COLUMNS FROM `article` [ RunTime:0.000936s ]
  8. SELECT * FROM `article` WHERE `id` = 564694 LIMIT 1 [ RunTime:0.000376s ]
  9. UPDATE `article` SET `lasttime` = 1794236114 WHERE `id` = 564694 [ RunTime:0.001465s ]
  10. SELECT * FROM `fenlei` WHERE `id` = 65 LIMIT 1 [ RunTime:0.000381s ]
  11. SELECT * FROM `article` WHERE `id` < 564694 ORDER BY `id` DESC LIMIT 1 [ RunTime:0.000438s ]
  12. SELECT * FROM `article` WHERE `id` > 564694 ORDER BY `id` ASC LIMIT 1 [ RunTime:0.000381s ]
  13. SELECT * FROM `article` WHERE `id` < 564694 ORDER BY `id` DESC LIMIT 10 [ RunTime:0.000462s ]
  14. SELECT * FROM `article` WHERE `id` < 564694 ORDER BY `id` DESC LIMIT 10,10 [ RunTime:0.000485s ]
  15. SELECT * FROM `article` WHERE `id` < 564694 ORDER BY `id` DESC LIMIT 20,10 [ RunTime:0.000561s ]
0.088025s