Skip to content

开发 Git 多文件上传接口并补齐 OAuth Git 凭据字段 - #35

Open
TechQuery with Copilot wants to merge 14 commits into
masterfrom
copilot/develop-git-multiple-file-upload
Open

开发 Git 多文件上传接口并补齐 OAuth Git 凭据字段#35
TechQuery with Copilot wants to merge 14 commits into
masterfrom
copilot/develop-git-multiple-file-upload

Conversation

Copilot AI commented May 11, 2026

Copy link
Copy Markdown
Contributor

PR-35 PR-35 PR-35 Powered by Pull Request Badge

为支持前端以 FormData 批量上传代码文件到 Git 仓库,新增了 /file/Git/*noProtocolURL 上传链路,并补齐了从上传 URL 自动匹配平台凭据所需的数据与映射能力。
同时补充容器运行环境依赖,确保服务内可执行 Git 上传命令。

  • 接口能力:新增 /file/Git/:noProtocolURL(.*) PUT 上传入口

    • FileController 增加 Git 上传接口,接收 multipart FormData。
    • 为支持“字段名即仓库路径”的上传形态,使用 multer.any() + @Ctx() 读取 ctx.req.files,并以 file.fieldname 作为仓库内目标路径。
    • 上传临时文件改为磁盘存储dest: os.tmpdir()),降低大批量上传时的内存占用。
    • 将路径校验与 Git 错误脱敏逻辑抽离到工具模块 source/gitUploadUtility.ts,控制器聚焦接口流程。
    • 将文件整理到工作目录后,通过 zx 调用 npx xgit upload 推送到目标仓库。
    • 支持 branch(默认 main)和可选 folder 参数。
    • 增加路径安全校验(防止 .. / 绝对路径穿越、控制字符、超长路径)与错误脱敏。
    • 错误处理统一改为后端框架封装的错误类型(如 BadRequestErrorUnprocessableEntityErrorForbiddenErrorInternalServerError)。
    • 增加临时文件回收逻辑:仅清理未成功移动的上传临时文件,并统一清理工作目录。
  • OAuth 凭据模型:补充第三方用户名

    • OAuthCredential 新增 username 字段,并改为必填,用于与 accessToken 组合构建 HTTP Git URL。
    • OAuth 登录同步逻辑更新为同时保存 token + username(GitHub 使用 login,CNB 使用 username)。
    • syncProfile 参数复用现有 profile 字段,语义调整为:profile.tokenaccessTokenprofile.name 传第三方用户名。
  • 平台映射:新增平台名与主域名映射能力

    • 在 OAuth 模型中新增 OAuthPlatformHostMapresolveOAuthPlatformByHost()
    • 上传时根据 noProtocolURL 的 host 自动解析平台,并查找当前用户对应 OAuth 凭据。
  • 运行环境与依赖

    • Dockerfile 在基础镜像中安装 git,满足 xgit 执行依赖。
    • 新增运行时依赖 zx(用于执行 npx xgit 与文件系统/路径操作)。
    • 新增开发依赖 @types/multer(补齐上传文件类型声明)。
  • 补充示例(上传调用形态)

    // PUT /file/Git/github.com/org/repo?branch=main&folder=src
    // FormData:
    //   formData.append('README.md', file1);
    //   formData.append('docs/guide.md', file2);
    //   formData.append('packages/a/index.ts', file3);

Copilot AI linked an issue May 11, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Add Git repository multi-file upload interface 开发 Git 多文件上传接口并补齐 OAuth Git 凭据字段 May 11, 2026
Copilot AI requested a review from TechQuery May 11, 2026 13:03
Comment thread source/controller/File.ts Outdated
Comment thread source/controller/File.ts Outdated
Comment thread source/controller/File.ts Outdated
Comment thread source/controller/OAuth.ts Outdated
Comment thread source/model/OAuth.ts Outdated
Copilot AI requested a review from TechQuery May 11, 2026 14:21
Comment thread source/controller/File.ts Outdated
Comment thread source/controller/File.ts Outdated
Comment on lines +33 to +76
const hasControlCharacter = (text: string) =>
[...text].some(character => {
const code = character.charCodeAt(0);

return code < 32 || code === 127;
});
const MAX_UPLOAD_FILE_PATH_LENGTH = 512;

const sanitizePathInput = (inputPath: string) => {
if (inputPath.length > MAX_UPLOAD_FILE_PATH_LENGTH)
throw new UnprocessableEntityError(`Invalid file path: ${inputPath}`);

const normalizedPath = path.normalize(inputPath).replace(/^[\\/]+/g, '');

if (
!normalizedPath ||
normalizedPath === '.' ||
normalizedPath.startsWith('..') ||
hasControlCharacter(normalizedPath) ||
path.isAbsolute(normalizedPath)
)
throw new UnprocessableEntityError(`Invalid file path: ${inputPath}`);

return normalizedPath;
};

const resolveSafePath = (basePath: string, inputPath: string) => {
const normalizedPath = sanitizePathInput(inputPath);
const targetPath = path.resolve(basePath, normalizedPath);

if (targetPath !== basePath && !targetPath.startsWith(`${basePath}${path.sep}`))
throw new UnprocessableEntityError(`Invalid file path: ${inputPath}`);

return targetPath;
};

const normalizeRelativePath = (path: string) => sanitizePathInput(path);

const sanitizeGitError = (message: string, accessToken: string) =>
message
.replaceAll(accessToken, '***')
.replaceAll(encodeURIComponent(accessToken), '***')
.replace(/\/\/[^/@\s]+@/g, '//***@')
.trim();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这些工具函数有哪些可以用流行的库来代替呢?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这块主要是路径规范化与越界校验,当前保持内置 path + 框架错误类型,避免再引入额外依赖;上传/文件操作已改用 multer + zx(fs/path/os)。相关调整见提交 8d22364

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这块主要是路径规范化与越界校验,当前保持内置 path + 框架错误类型,避免再引入额外依赖

那就放到工具函数的模块吧,类的模块尽量还是写类本身。

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已把路径校验和 Git 错误脱敏等工具函数移到独立模块 source/gitUploadUtility.ts,控制器里只保留接口流程。见提交 a933bce

Comment thread source/controller/File.ts Outdated
Comment thread source/controller/File.ts Outdated
Comment thread source/controller/File.ts Outdated
Comment thread source/controller/OAuth.ts Outdated
Comment thread test/OAuth.test.ts Outdated
Copilot AI requested a review from TechQuery May 11, 2026 17:54
Comment thread source/controller/File.ts Outdated
Comment on lines +30 to +39
const uploadMiddleware = multer({
storage: multer.diskStorage({
destination: os.tmpdir(),
filename: (_request, _file, callback) =>
callback(
null,
`hop-upload-${Date.now()}-${randomUUID()}`
)
})
});

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个库的默认存储行为不是每次上传都有相互隔离的临时文件夹吗?还是我记错了?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

你没记错,multer 在指定 dest 时会用默认磁盘存储并生成临时文件名;我这里已改成 multer({ dest: os.tmpdir() }),不再自定义文件名逻辑。见提交 a933bce

Comment thread source/controller/File.ts Outdated
Comment on lines +33 to +76
const hasControlCharacter = (text: string) =>
[...text].some(character => {
const code = character.charCodeAt(0);

return code < 32 || code === 127;
});
const MAX_UPLOAD_FILE_PATH_LENGTH = 512;

const sanitizePathInput = (inputPath: string) => {
if (inputPath.length > MAX_UPLOAD_FILE_PATH_LENGTH)
throw new UnprocessableEntityError(`Invalid file path: ${inputPath}`);

const normalizedPath = path.normalize(inputPath).replace(/^[\\/]+/g, '');

if (
!normalizedPath ||
normalizedPath === '.' ||
normalizedPath.startsWith('..') ||
hasControlCharacter(normalizedPath) ||
path.isAbsolute(normalizedPath)
)
throw new UnprocessableEntityError(`Invalid file path: ${inputPath}`);

return normalizedPath;
};

const resolveSafePath = (basePath: string, inputPath: string) => {
const normalizedPath = sanitizePathInput(inputPath);
const targetPath = path.resolve(basePath, normalizedPath);

if (targetPath !== basePath && !targetPath.startsWith(`${basePath}${path.sep}`))
throw new UnprocessableEntityError(`Invalid file path: ${inputPath}`);

return targetPath;
};

const normalizeRelativePath = (path: string) => sanitizePathInput(path);

const sanitizeGitError = (message: string, accessToken: string) =>
message
.replaceAll(accessToken, '***')
.replaceAll(encodeURIComponent(accessToken), '***')
.replace(/\/\/[^/@\s]+@/g, '//***@')
.trim();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这块主要是路径规范化与越界校验,当前保持内置 path + 框架错误类型,避免再引入额外依赖

那就放到工具函数的模块吧,类的模块尽量还是写类本身。

Comment thread source/controller/File.ts Outdated
Comment thread source/controller/File.ts Outdated
Copilot AI requested a review from TechQuery May 11, 2026 19:26
@TechQuery
TechQuery marked this pull request as ready for review May 11, 2026 19:38
Copilot AI review requested due to automatic review settings May 11, 2026 19:38
@TechQuery TechQuery added the feature New feature or request label May 11, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

该 PR 旨在支持前端通过 FormData 批量上传多文件并推送到 Git 仓库:新增 /file/Git/:noProtocolURL(.*) 上传入口;同时扩展 OAuth 凭据模型以保存用于 Git HTTP URL 的第三方 username,并新增按仓库 host 自动解析 OAuth 平台的能力;容器层面补齐 git 运行依赖并引入 zx 以执行上传命令。

Changes:

  • 新增 Git 多文件上传接口(multipart + 字段名作为仓库内路径),并在服务端整理文件后执行 xgit upload 推送
  • OAuthCredential 增加 username 字段,并在 OAuth 登录同步时写入(配合 host->platform 解析)
  • Docker 镜像安装 git;新增依赖 zx@types/multer

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
source/controller/File.ts 新增 /file/Git/:noProtocolURL(.*) 上传接口、解析文件并调用 xgit upload 推送
source/gitUploadUtility.ts 新增路径校验/安全拼接与 Git 错误脱敏工具函数
source/model/OAuth.ts 增加 OAuth 平台 host 映射与 OAuthCredential.username 字段
source/controller/OAuth.ts OAuth 登录同步逻辑调整:保存 accessToken + username,并更新 profile 处理
package.json 增加运行时依赖 zx 与开发依赖 @types/multer
pnpm-lock.yaml 锁文件更新以纳入新增依赖
Dockerfile 基础镜像安装 git 以支持运行期执行 Git 命令
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported
Comments suppressed due to low confidence (1)

source/controller/OAuth.ts:60

  • oldProfile includes name but newProfile intentionally omits it (it’s destructured out). This makes isDeepStrictEqual(oldProfile, newProfile) always false, so every OAuth login will trigger a User save + activityLogService.logUpdate, even when nothing changed. Either include name in newProfile (if you still want to sync it) or remove it from oldProfile/comparison so the equality check is meaningful.
        const { token: accessToken, name, ...newProfileData } = profile;
        if (!accessToken)
            throw new UnprocessableEntityError(
                `${platform} user info is missing required field (accessToken)`
            );
        if (!name)
            throw new UnprocessableEntityError(
                `${platform} user info is missing required field (username)`
            );

        const user =
            (await this.userStore.findOneBy({ email })) ||
            (await sessionService.signUp({ email, password: accessToken }));
        const { name: oldName, avatar, languages } = user;
        const oldProfile = { name: oldName, avatar, languages: languages?.length ? languages : [] };
        const newProfile = {
            ...newProfileData,
            languages: newProfileData.languages?.length ? newProfileData.languages : []
        };

        if (!isDeepStrictEqual(oldProfile, newProfile)) {
            await this.userStore.save(Object.assign(user, newProfile));

            await activityLogService.logUpdate(user, 'User', user.id);
        }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread source/model/OAuth.ts
Comment on lines +40 to +42
@IsString()
@Column()
username: string;
Comment thread source/controller/File.ts Outdated
Comment on lines +73 to +78
@Req() { files }: { files?: Express.Multer.File[] },
@CurrentUser() user: User,
@Param('noProtocolURL') noProtocolURL: string,
@QueryParam('branch') branch = 'main',
@QueryParam('folder') folder?: string
) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Koa, @koa/multer attaches parsed uploads to the underlying Node request (ctx.req.files), while @Req() in routing-controllers returns the Koa request object. Destructuring { files } from @Req() is likely to always be undefined, causing this endpoint to reject valid uploads. Prefer @Ctx() ctx and read ctx.req.files, or accept the full request object and access request.req.files explicitly.

@copilot 我们是不是应该用 Koa 的 @Ctx()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

是的,这里改用 Koa 的 @Ctx() 更稳妥。我已改为从 @Ctx() { req } 读取 req.files,避免 @Req() 取不到上传文件的问题。见提交 a09f609。此改动仅后端接口逻辑,无 UI 变更截图。

Comment thread source/controller/File.ts
Comment on lines +103 to +113
repositoryURL.username = credential.username;
repositoryURL.password = credential.accessToken;

const targetFolder = folder ? normalizeRelativePath(folder) : undefined;
const tempRoot = await fs.mkdtemp(path.resolve(os.tmpdir(), 'hop-git-upload-'));
const pendingUploadedPaths = new Set(files.map(({ path }) => path));
const originalVerbose = Boolean($.verbose);

try {
$.verbose = true;
for (const file of files) {
Comment thread source/controller/File.ts
Comment on lines +120 to +123
if (targetFolder)
await $`npx xgit upload ${tempRoot} ${repositoryURL} ${branch} ${targetFolder}`;
else
await $`npx xgit upload ${tempRoot} ${repositoryURL} ${branch}`;
Comment thread source/controller/File.ts
Comment on lines 34 to +72
@@ -44,4 +64,79 @@ export class FileController {

await s3Client.send(command);
}

@Put('/Git/:noProtocolURL(.*)')
@Authorized()
@HttpCode(201)
@UseBefore(uploadMiddleware.any())
async uploadFilesToGit(
Comment on lines +15 to +18
export const sanitizePathInput = (inputPath: string) => {
if (inputPath.length > MAX_UPLOAD_FILE_PATH_LENGTH)
throw new UnprocessableEntityError(`Invalid file path: ${inputPath}`);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

开发 Git 代码库多文件上传接口

3 participants