通过配置 Git Proxy 加速克隆 Github 项目
前言⌗
可能是之前无论是 NPM 还是 Composer 又或者是 Homebrew 在安装所需要的包时,我都会通过环境变量设置代理,来提升下载的速度。我潜意识里就觉得 ALL_PROXY
这个环境变量可能是所有协议都会用这个代理。
但是当我使用 git clone [email protected]:u-boot/u-boot.git
克隆项目时,我发现很长时间并没有任何反应,再一看 Surge 的流量监控,没有流量经过,那应该是没有走我设置的 ALL_PROXY
。
经过一番面向搜索引擎学习,发现 Git 支持的协议有下面三种:
- HTTP
- SSH
- Git Protocol
误区⌗
看到网上很多的教程都说的是在 ~/.gitconfig
文件中配置 http 和 https 的代理:
[http "https://github.com/"]
proxy = http://<host>:<port>/
[https "https://github.com/"]
proxy = https://<host>:<port>/
又或者是通过下面的命令设置代理:
git config --global http.proxy http://username:[email protected]:8080
git config --global https.proxy http://username:[email protected]:8080
或者是像我之前使用的使用临时环境变量:
export https_proxy=http://127.0.0.1:8234;export http_proxy=http://127.0.0.1:8234;export all_proxy=socks5://127.0.0.1:8235
注意:这种方式只针对使用 HTTP 协议的才有效!如果你在 clone 的时候使用的是 SSH 协议,这些设置并不能起到什么作用。
配置 SSH 代理⌗
与在之前的《使用 ClashX 为 SSH 加速》文中一样,我们只需要在 ~/.ssh/config
文件中添加如下配置:
Host github.com
User git
ProxyCommand nc -x 127.0.0.1:8235 -X 5 %h %p
设置好以后,就可以测试使用 SSH 的方式从 Github 上克隆项目了。
对于使用 gh
1 的开发者来说,如果使用的是 SSH 协议,那么也是一样的。
gh repo clone u-boot/u-boot
Git Protocol⌗
本来没打算写关于 Git Protocol 的部分,因为很少用单,但是在学习操作系统的时候需要安装 riscv-tools
,于是照着 6.S081: Operating System Engineering 的步骤安装,但是发现在克隆 glibc
的源码时因为网络原因,卡主不动了。看了下日志,这个项目仓库的地址是 git://sourceware.org/git/glibc.git
。
期初没认出来这个是 Git Protocol,我按照 SSH 的方法在配置文件中设置了 sourceware.org
的代理,但是发现并没有生效,然后琢磨了半天,发现这个不是 SSH 协议。
既然找到问题了,那就好解决了,首先需要创建一个 git-proxy
脚本:
#! /bin/bash
# connect to the Git repository through a SOCKS proxy
# default setting is to use port 1080 on the local host
proxy="localhost:1080"
from="default"
# check if there is a value in the git configuration
if git config --get socks.proxy >& /dev/null; then
proxy=`git config --get socks.proxy`
from="git's socks.proxy"
fi
# check if a generic proxy has been defined in the environment
if [ -n "$ALL_PROXY" ]; then
proxy="$ALL_PROXY"
from="\$ALL_PROXY"
fi
if [ -n "$all_proxy" ]; then
proxy="$all_proxy"
from="\$all_proxy"
fi
# check if a SOCKS proxy has been defined in the environment
if [ -n "$SOCKS_PROXY" ]; then
proxy="$SOCKS_PROXY"
from="\$SOCKS_PROXY"
fi
if [ -n "$socks_proxy" ]; then
proxy="$socks_proxy"
from="\$socks_proxy"
fi
if [ -n "$SOCKS5_PROXY" ]; then
proxy="$SOCKS5_PROXY"
from="\$SOCKS5_PROXY"
fi
if [ -n "$socks5_proxy" ]; then
proxy="$socks5_proxy"
from="\$socks5_proxy"
fi
# check if a git specific SOCKS proxy has been defined in the environment
if [ -n "$GIT_SOCKS_PROXY" ]; then
proxy="$GIT_SOCKS_PROXY"
from="\$GIT_SOCKS_PROXY"
fi
function usage() {
cat << @EOF
Usage:
`basename $0` HOST PORT
Helper script to connect to a Git repository over the git:// protocol at host HOST and port PORT through a SOCKS proxy at $proxy ($from).
To use the proxy for all git:// traffic, set the core.gitproxy option to "git-proxy":
git config core.gitproxy "git-proxy"
To use the proxy only for some reporitories, use the syntax explained in git-config(1).
To configure which proxy to use, set an appropriate environment variable (see below) or socks.proxy option to the proxy address, for example "localhost:1080":
git config socks.proxy "localhost:1080"
The address of the proxy is read from (in order of priority):
- the GIT_SOCKS_PROXY environment variable;
- the SOCKS_PROXY or SOCKS5_PROXY environment variable;
- the ALL_PROXY environment variable (see curl(1));
- the socks.proxy git option;
- the default value: localhost:1080 .
@EOF
}
if [ -z "$1" ] || [ -z "$2" ] || [ -n "$3" ]; then
usage
exit 1
fi
# connect through the specifid proxy
nc -x "$proxy" "$1" "$2"
将上面的代码保存为脚本,然后保存到你的任意 PATH 目录中,例如我这里是放在了 /usr/local/bin/
的目录中,命名为:git-proxy
,然后执行如下命令,验证是否生效:
git-proxy --help
Usage:
git-proxy HOST PORT
Helper script to connect to a Git repository over the git:// protocol at host HOST and port PORT through a SOCKS proxy at 127.0.0.1:8235 (git's socks.proxy).
To use the proxy for all git:// traffic, set the core.gitproxy option to "git-proxy":
git config core.gitproxy "git-proxy"
To use the proxy only for some reporitories, use the syntax explained in git-config(1).
To configure which proxy to use, set an appropriate environment variable (see below) or socks.proxy option to the proxy address, for example "localhost:1080":
git config socks.proxy "localhost:1080"
The address of the proxy is read from (in order of priority):
- the GIT_SOCKS_PROXY environment variable;
- the SOCKS_PROXY or SOCKS5_PROXY environment variable;
- the ALL_PROXY environment variable (see curl(1));
- the socks.proxy git option;
- the default value: localhost:1080 .
接着按照提示执行如下命令:
# 脚本的位置建议使用绝对地址,否则可能出现 error: cannot run git-proxy: No such file or directory.
git config core.gitproxy "/usr/local/bin/git-proxy"
git config socks.proxy "127.0.0.1:8235"
另外因为 Surge 的 Copy Shell Export Command
生成的命令中,存在 export all_proxy=socks5://127.0.0.1:8235
这个环境变量,根据 git-proxy
中的脚本逻辑,可以看到如果设置了 all_proxy
环境变量,将会覆盖掉 Git 的全局配置!!!正式因为这样,才导致我发现了下面的坑。
注意:这里的代理地址不需要加协议类型,直接是 HOST:PORT 这种写法。如过包含协议,如 socks5://127.0.0.1:8235,这种写法会导致 nc: getaddrinfo: nodename nor servname provided, or not known. 从而无法代理 Git Protocol。
然后测试通过 Git Protocol 克隆的流量是否按照预期走我们的 Surge 代理:
brew install riscv-tools
==> Downloading https://ghcr.io/v2/homebrew/core/mpfr/manifests/4.1.0
Already downloaded: /Users/George/Library/Caches/Homebrew/downloads/e8b18e060649c00d9e1e351e3e05515c365e52dbd7bd392e2b99808d1f355b5b--mpfr-4.1.0.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/mpfr/blobs/sha256:81ced499f237acfc2773711a3f8aa985572eaab2344a70485c06f72405e4a5e7
Already downloaded: /Users/George/Library/Caches/Homebrew/downloads/1816ae3e6ff09ea7ad42b8f5ff16142fd0a973c41c5633dfa3b1cf4fa2e1d51b--mpfr--4.1.0.arm64_monterey.bottle.tar.gz
==> Downloading https://ghcr.io/v2/homebrew/core/gawk/manifests/5.1.1
Already downloaded: /Users/George/Library/Caches/Homebrew/downloads/13f45768bf43ccad0a4fabeca4923e0a6836a412b1255585fba6260956e53b70--gawk-5.1.1.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/gawk/blobs/sha256:093465f34b94ec8ddeb4ff8dab2a02dafbccf8ec05f6ef0391673b7c4fd0a91f
Already downloaded: /Users/George/Library/Caches/Homebrew/downloads/0e765b98a270520e3539fb021f555e78705e08b6f24da7e20a56d1f228be9ff1--gawk--5.1.1.arm64_monterey.bottle.tar.gz
==> Downloading https://ghcr.io/v2/homebrew/core/gnu-sed/manifests/4.8
Already downloaded: /Users/George/Library/Caches/Homebrew/downloads/71fd4a7118e6fb954728ce6a8f92871d721711578de17baa01337b3570581179--gnu-sed-4.8.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/gnu-sed/blobs/sha256:78481cc3509f617328d3c361c21beef829f24f4b130cabfc08ed6e4ce83f2286
Already downloaded: /Users/George/Library/Caches/Homebrew/downloads/c0d5ea51ee74ae84aee90d77161075b8567d8c949e99a85582e2fa1570eb824b--gnu-sed--4.8.arm64_monterey.bottle.tar.gz
==> Downloading https://ghcr.io/v2/homebrew/core/isl/manifests/0.24
Already downloaded: /Users/George/Library/Caches/Homebrew/downloads/cf108f3cbdbad48a5ecbe0a148e97b47590045a58151ad768bcf7219ec23305a--isl-0.24.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/isl/blobs/sha256:be08c3e9765655ad5bfd227f9b97acb0ef88ad2307dc214ea4064cc1f51db641
Already downloaded: /Users/George/Library/Caches/Homebrew/downloads/65e4670b48d6dda0e181978e35cf506256f350f33cf7e1371b9451856ad21d41--isl--0.24.arm64_monterey.bottle.tar.gz
==> Downloading https://ghcr.io/v2/homebrew/core/libmpc/manifests/1.2.1
Already downloaded: /Users/George/Library/Caches/Homebrew/downloads/3f193de9dd7f9e68742cf207cfadd3a63b695a8026d343b668e10a9b7bcf6e52--libmpc-1.2.1.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/libmpc/blobs/sha256:658a1d260b6f77c431451a554ef8fa36ea2b6db19b38adc05c52821598ce2647
Already downloaded: /Users/George/Library/Caches/Homebrew/downloads/d18d439506c54ddbad35cc4dc0c77e5fc3fd442c9ec926ff212d6fed1a11c8b6--libmpc--1.2.1.arm64_monterey.bottle.tar.gz
==> Cloning https://github.com/riscv/riscv-gnu-toolchain.git
Updating /Users/George/Library/Caches/Homebrew/riscv-gnu-toolchain--git
==> Checking out branch master
Already on 'master'
Your branch is up to date with 'origin/master'.
HEAD is now at 1342cd7 Merge pull request #1073 from palmer-dabbelt/qemu-system
Entering 'qemu'
Entering 'riscv-binutils'
Entering 'riscv-dejagnu'
Entering 'riscv-gcc'
Entering 'riscv-gdb'
Cloning into '/Users/George/Library/Caches/Homebrew/riscv-gnu-toolchain--git/glibc'...
Cloning into '/Users/George/Library/Caches/Homebrew/riscv-gnu-toolchain--git/musl'...
Cloning into '/Users/George/Library/Caches/Homebrew/riscv-gnu-toolchain--git/newlib'..
ps -ef | grep "nc -x 127.0.0.1:8235" | grep -v grep
501 53932 53462 0 11:58PM ttys002 0:00.00 nc -x 127.0.0.1:8235 sourceware.org 9418
可以看到,在执行 Homebrew 安装命令的时候,启动了一个 netcat
进程,并通过代理与 sourceware.org
的 9418
端口建立了连接。9418
就是 Git Protocol 所使用的端口!
SSH 协议使用 HTTP 端口进行项目克隆⌗
最近因为速蛙云跑路,不得不更换其他机场。但是更换后发现无法通过 SSH 协议正常克隆 Github 项目,出现如下错误:
gh repo clone golang-module/carbon
Cloning into 'carbon'...
kex_exchange_identification: Connection closed by remote host
Connection closed by UNKNOWN port 65535
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
exit status 128
询问后得知机场运维人员处于安全考虑禁止了 22 端口的流量,同时也给出了解决方案,在 Github 官方文档中有说明:Using SSH over the HTTPS port。
参照文档我修改了 ~/.ssh/config
的配置:
Host github.com
User git
HostName ssh.github.com
Port 443
ProxyCommand nc -x 127.0.0.1:8235 -X 5 %h %p
然后就可以正常加速 SSH 协议的 Git 操作了!
I hope this is helpful, Happy hacking…