解决阿里云大文件无法上传问题

解决阿里云大文件无法上传问题

最近开发上传oss功能时，考虑到不想走服务器流量，就打算使用临时密钥的方式。

具体的技术方案

https://cloud.tencent.com/document/product/436/14048

后端的核心代码如下

public CtYunOssTemporaryVo temporary(List<CtYunOssTemporaryDTO> dto) {
      String code = StrUtil.nullToDefault(AuthUtil.getCode(),"default");
      TreeMap<String, Object> config = new TreeMap<String, Object>();
      config.put("secretId", secretIdTx);
      config.put("secretKey", secretKeyTx);
      Policy policy = new Policy();
      config.put("durationSeconds", 1800);
      config.put("bucket", endpointTx);
      config.put("region", regionTx);
      Statement statement = new Statement();
      statement.setEffect("allow");
      statement.addActions(new String[]{
              "cos:PutObject",
              "cos:PostObject",
              "ci:CreateFileProcessJobs"
      });
      List<String> resources = new ArrayList<>();

      List<CtYunOssTemporaryFileVo> fileList = new ArrayList<>();
      for (CtYunOssTemporaryDTO temporaryDTO : dto) {
          String suffix = temporaryDTO.getSuffix();
          if (StrUtil.isEmpty(suffix)) {
              throw new ApiResultException("文件" + temporaryDTO.getFileName() + "后缀不能为空");
          }
          CtYunOssTemporaryFileVo fileVo = BeanUtil.toBean(temporaryDTO, CtYunOssTemporaryFileVo.class);
          String uid = IdUtil.fastSimpleUUID();
          String key = '生成唯一key';
          //qcs::cos:ap-beijing:uid/1250000000:examplebucket-1250000000/doc2/*
          String format = String.format("qcs::cos:%s:uid/%s:%s/%s", endpointTx, uidTx, bucketNameTx,key);
          resources.add(format);

          //日志数据
          SysFiles entity = new SysFiles();
          entity.setPermission(FilePermissionEnum.PUBLIC.name());
          entity.setId(IdUtil.getSnowflake(0, 0).nextId());
          entity.setGroupName("cos-temporary");
          entity.setName(temporaryDTO.getFileName());
          entity.setType(suffix);
          entity.setAbsolutePath(pathTx + key);
          entity.setRelativePath(key);
          dao.insert(entity);

          fileVo.setFileId(uid);
          fileVo.setKey(key);
          fileVo.setAbsolutePath(pathTx + key);
          fileList.add(fileVo);
      }
      statement.addResources(resources.toArray(new String[0]));

      policy.addStatement(statement);
      config.put("policy", Jackson.toJsonPrettyString(policy));
      try {
          Response response = CosStsClient.getCredential(config);
          if(response == null || response.credentials == null || response.credentials.tmpSecretKey == null){
            throw new ApiResultException("无法获取上传密钥");
          }
          return CtYunOssTemporaryVo.builder()
                  .tmpSecretId(response.credentials.tmpSecretId)
                  .tmpSecretKey(response.credentials.tmpSecretKey)
                  .sessionToken(response.credentials.sessionToken)
                  .startTime(response.startTime)
                  .expiredTime(response.expiredTime)
                  .bucket(bucketNameTx)
                  .region(regionTx)
                  .fileList(fileList)
                  .build();
      } catch (IOException e) {
        e.printStackTrace();
          throw new ApiResultException("上传权限获取异常");
      }
  }

因为从后端的思维，之前服务器上传的时候，我们正常采用的时put请求和post请求将数据发送到服务器端，所以将demo中的

文档提供的

statement.addActions(new String[]{
              "cos:PutObject",
              // 分块上传
              "cos:InitiateMultipartUpload",
              "cos:ListMultipartUploads",
              "cos:ListParts",
              "cos:UploadPart",
              "cos:CompleteMultipartUpload",
              // 表单上传、小程序上传
              "cos:PostObject",
              // 文件压缩
              "ci:CreateFileProcessJobs"
      });

//

我写的 考虑到最低权限赋值

statement.addActions(new String[]{
              "cos:PutObject",
              "cos:PostObject",
              "ci:CreateFileProcessJobs"
      });

问题的产生

在和前端的联调中发现并没有问题，直到测试上传了一个大文件，发现上传不上去

<Error>
	<Code>AccessDenied</Code>
	<Message>
	Access Denied.
	</Message>
	<ServerTime>
	2024-12-26T06:35:31Z
	</ServerTime>
	<Resource>/</Resource>
<RequestId>
	Njc2Y2Y5MzNfODg3MDgxMGJfMTkzZDdfMTAwNDM3MA==
</RequestId>
<TraceId>
OGVmYzZiMmQzYjA2OWNhODk0NTRkMTBiOWVmMDAxODc0OWRkZjk0ZDM1NmI1M2E2MTRlY2MzZDhmNmI5MWI1OTBjYzE2MjAxN2M1MzJiOTdkZjMxMDVlYTZjN2FiMmI0NTZkM2E5M2VhMDJhYmRiMjFmNGRlMGQ5YTgxNzQzYmM=
</TraceId>
</Error>

通过下载官方的sdk 和demo发现，在前端上传大文件时，请求不是PUT而是GET

通过查看代码发现，有个 SliceSize 属性

// 监听选文件
document.getElementById('file-selector').onchange = function () {

    var file = this.files[0];
    if (!file) return;

    // 上传文件
    cos.uploadFile({
        Bucket: Bucket,
        Region: Region,
        Key: file.name,
        Body: file,
        SliceSize: 10 * 1024 * 1024, // 大于1mb才进行分块上传
        onTaskReady: function (tid) {
          taskId = tid;
        },
        onProgress: function (progressData) {
            console.log('上传中', JSON.stringify(progressData));
        },
    }, function (err, data) {
        console.log(err, data);
    });

    // 可使用队列暂停、重启任务
    // cos.pauseTask(taskId);

};

所以发现，10M上传转换成了分片上传，所以需要将下列一并添加到权限中

// 分块上传
"cos:InitiateMultipartUpload",
"cos:ListMultipartUploads",
"cos:ListParts",
"cos:UploadPart",
"cos:CompleteMultipartUpload",