electron 使用 web worker 上传批量的文件（7W+）报 Maximum call stack size exceeded 错误

V2EX = way to explore

V2EX 是一个关于分享和探索的地方

现在注册

已注册用户请登录

这是一个创建于 1656 天前的主题，其中的信息可能已经有所发展或是发生改变。

web worker 从主进程获取到上传文件的路径，在 onmessage 处理上传，通过 fs.promises.stat 获取文件的信息，通过 postMessage 回传至 worker 主进程中，在 worker 主进程中，将这些数据更新至 vuex 中，出现了下面的问题，小数量级的文件，不会出现这个问题

  (node:33176) UnhandledPromiseRejectionWarning: RangeError: Maximum call stack size exceeded
      at Array.mutator (D:\pc-win\SKYD-win\node_modules\vue\dist\vue.runtime.common.dev.js:870:27)
      at Store.addMultiTaskUL (webpack:///./src/renderer/store/modules/task.js?:164:72)
      at wrappedMutationHandler (D:\pc-win\SKYD-win\node_modules\vuex\dist\vuex.common.js:727:13)
      at commitIterator (D:\pc-win\SKYD-win\node_modules\vuex\dist\vuex.common.js:393:7)
      at Array.forEach (<anonymous>)
      at D:\pc-win\SKYD-win\node_modules\vuex\dist\vuex.common.js:392:11
      at Store._withCommit (D:\pc-win\SKYD-win\node_modules\vuex\dist\vuex.common.js:523:3)
      at Store.commit (D:\pc-win\SKYD-win\node_modules\vuex\dist\vuex.common.js:391:8)
      at Store.boundCommit [as commit] (D:\pc-win\SKYD-win\node_modules\vuex\dist\vuex.common.js:336:19)
      at local.commit (D:\pc-win\SKYD-win\node_modules\vuex\dist\vuex.common.js:681:13)
      at eval (webpack:///./src/renderer/store/modules/task.js?:476:7)
      at new Promise (<anonymous>)
      at new F (webpack:///./node_modules/core-js/library/modules/_export.js?:36:28)
      at Store.addMultiTaskUL (webpack:///./src/renderer/store/modules/task.js?:440:12)
      at Array.wrappedActionHandler (D:\pc-win\SKYD-win\node_modules\vuex\dist\vuex.common.js:734:23)
      at Store.dispatch (D:\pc-win\SKYD-win\node_modules\vuex\dist\vuex.common.js:439:15)
  (node:33176) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 10)

10 条回复 • 2020-04-29 15:11:08 +08:00

TomeWong

2020-04-29 11:10:42 +08:00

```
self.onmessage = async event => {
console.log(event)
const { diskType, files, dirpath, sid, usn, currentFileId, currentGroupId, currentCreatedByUsn, baseUrl, uploadLimit, uploadSpeed } = event.data
const queueData = [],
taskDataArr = []
let completeStatus
for (let elem of files) {
const filePath = elem,
// stats = await fs.promises.stat(filePath).catch(error => console.log(error)),
// size = stats.size,
chunkSize = 5 * 1024 * 1024, //每片分块的大小 5M
// pieces = Math.ceil(size / chunkSize),
uploadType = pieces == 1 ? 1 : 3,
num = filePath.lastIndexOf('\\') + 1,
name = filePath.substring(num),
uuid = uuidv4(),
basename = dirpath ? path.basename(dirpath) : '',
realPath = dirpath ? filePath.substring(filePath.indexOf(basename), filePath.lastIndexOf("\\")).replace(/\\/g, "/") : ''
const payload = {
appFileId: "",
diskType: diskType,
uploadType: uploadType,
creatorUsn: currentCreatedByUsn || usn,
parentid: currentFileId == -2 ? -1 : currentFileId || -1,
groupId: diskType == 1 ? '' : currentGroupId,
fileSize: 0,
fileName: name,
fileMd5: '',
fileRealPath: realPath,
comeFrom: 30,
model: 0,
discussContent: ""
}
const queueItemData = { sid, uuid, payload, name, size, currentFileId, uploadType, pieces, chunkSize, filePath, currentCreatedByUsn, baseUrl, uploadLimit, uploadSpeed }
// taskDataArr.push({ filePath, name, uuid, size, pieces, uploadType, diskType, parentid: currentFileId == -2 ? -1 : currentFileId || -1, fileRealPath: realPath })
taskDataArr.push({ filePath, name, uuid, size, pieces, uploadType, diskType, parentid: currentFileId == -2 ? -1 : currentFileId || -1, fileRealPath: realPath })
queueData.push(queueItemData)

}
completeStatus = {
status: 'updateStore'
}
self.postMessage({taskDataArr, completeStatus})
}
```

TomeWong

2020-04-29 11:23:57 +08:00

是否转换字符串来传输，而不是数组对象

wednesdayco

2020-04-29 13:59:28 +08:00

分……片……

TomeWong

2020-04-29 14:27:13 +08:00

@wednesdayco 现在都是小文件，可以正常上传，但将 web worker 中处理的信息回传至 worker 主进程，更新至 vuex 有问题，vuex 是不是存在存储大小限制的问题

Vegetable

2020-04-29 14:30:01 +08:00

显然这是递归层数过多
你是一个一个上传, 一个文件成功之后, 回调里传下一个吗?

Vegetable

2020-04-29 14:32:31 +08:00

let deep = 1

function c() {
deep++
c()
}

try {
c()
} catch (error) {
console.log(error)
//RangeError: Maximum call stack size exceeded
console.log(deep)
//15712
}

TomeWong

2020-04-29 14:39:58 +08:00

@Vegetable 是一个一个上传的，并发量为 3，上传是正常的，在上传前会先根据每条文件的路径获取一些信息，然后再将这些信息通过 postMessage 回传至 worker 主进程，在 worker 主进程将这些处理的信息更新至 vuex 中，这个更新过程出现了问题

VDimos

2020-04-29 14:40:39 +08:00 via Android

你这递归层级太多了吧，做下尾递归优化，不要保留递归栈。

MrYELiex

2020-04-29 14:41:00 +08:00

错误信息很明显调用栈过多

TomeWong

2020-04-29 15:11:08 +08:00

@VDimos 在 vuex 中使用 map 来处理，可能会有问题