Gitlab: Git operations are extremely slow on EC2 The 2019 Stack Overflow Developer Survey Results Are InUsing git on amazon ec2Move repo from plain git to GitLab serverWhy are Amazon OpsWorks operations so slow?Can I use gitlab to monitorize existing repos, keeping old bare git urls?How to merge Git and GitLab usersJenkins/GitLab: How to automatically build the latest git tag?Gitlab Runner on AWS becomes really slow after some timeUsing Git submodules in Gitlab CI jobsEC2: Cannot git clone at launchGit clone from Gitlab incredibly slow in Docker container
Inversion Puzzle
Should I write numbers in words or as numerals when there are multiple next to each other?
Spanish for "widget"
Limit the amount of RAM Mathematica may access?
How can I create a character who can assume the widest possible range of creature sizes?
Does duplicating a spell with Wish count as casting that spell?
Is there a name of the flying bionic bird?
How was Skylab's orbit inclination chosen?
What tool would a Roman-age civilization have to grind silver and other metals into dust?
Does it makes sense to buy a new cycle to learn riding?
What could be the right powersource for 15 seconds lifespan disposable giant chainsaw?
Is domain driven design an anti-SQL pattern?
"What time...?" or "At what time...?" - what is more grammatically correct?
How can I fix this gap between bookcases I made?
Does light intensity oscillate really fast since it is a wave?
How are circuits which use complex ICs normally simulated?
Unbreakable Formation vs. Cry of the Carnarium
Is three citations per paragraph excessive for undergraduate research paper?
Falsification in Math vs Science
Landlord wants to switch my lease to a "Land contract" to "get back at the city"
How to make payment on the internet without leaving a money trail?
Are there any other methods to apply to solving simultaneous equations?
What do hard-Brexiteers want with respect to the Irish border?
Why isn't airport relocation done gradually?
Gitlab: Git operations are extremely slow on EC2
The 2019 Stack Overflow Developer Survey Results Are InUsing git on amazon ec2Move repo from plain git to GitLab serverWhy are Amazon OpsWorks operations so slow?Can I use gitlab to monitorize existing repos, keeping old bare git urls?How to merge Git and GitLab usersJenkins/GitLab: How to automatically build the latest git tag?Gitlab Runner on AWS becomes really slow after some timeUsing Git submodules in Gitlab CI jobsEC2: Cannot git clone at launchGit clone from Gitlab incredibly slow in Docker container
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
EDIT: It turns out this is a Gitlab problem, however I still do not have a solution.
I have a weird situation going on with two of my AWS EC2 instances. They are exactly the same in terms of OS, region, and instance type (both t3.micro), set up in the same way (however, the first was set up a few months ago).
Both exist in the eu-central-1c availability zone, and both are operating on the same git repository. Both are also up to date (CentOS 7.6.1810).
Older server:
$ time git pull
Already up-to-date.
real 0m0.306s
user 0m0.034s
sys 0m0.016s
Newer server:
$ time git pull
Already up-to-date.
real 2m7.547s
user 0m0.026s
sys 0m0.024s
It also consistently takes about 2m7s.
Also:
Older server:
--2019-04-09 10:52:03-- https://speed.hetzner.de/1GB.bin
Resolving speed.hetzner.de (speed.hetzner.de)... 88.198.248.254, 2a01:4f8:0:59ed::2
Connecting to speed.hetzner.de (speed.hetzner.de)|88.198.248.254|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048576000 (1000M) [application/octet-stream]
Saving to: ‘1GB.bin’
100%[===============================================================>] 1,048,576,000 121MB/s in 6.5s
2019-04-09 10:52:10 (154 MB/s) - ‘1GB.bin’ saved [1048576000/1048576000]
Newer server:
--2019-04-09 10:54:04-- https://speed.hetzner.de/1GB.bin
Resolving speed.hetzner.de (speed.hetzner.de)... 88.198.248.254, 2a01:4f8:0:59ed::2
Connecting to speed.hetzner.de (speed.hetzner.de)|88.198.248.254|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048576000 (1000M) [application/octet-stream]
Saving to: ‘1GB.bin’
100%[===============================================================>] 1,048,576,000 130MB/s in 5.9s
2019-04-09 10:54:10 (170 MB/s) - ‘1GB.bin’ saved [1048576000/1048576000]
EDIT: I tried to use a GitHub repository instead of our GitLab, and it turns out it seems to be a GitLab issue. What could possibly be causing GitLab to respond quickly to the older server but not to the other one?
EDIT 2: Attempted to clone over HTTPS. It takes 2 minutes just for it to ask for my username.
Also, verbose output over SSH:
$ GIT_CURL_VERBOSE=1 GIT_TRACE=1 git pull
trace: exec: 'git-pull'
trace: run_command: 'git-pull'
trace: built-in: git 'rev-parse' '--git-dir'
trace: built-in: git 'rev-parse' '--is-bare-repository'
trace: built-in: git 'rev-parse' '--show-toplevel'
trace: built-in: git 'ls-files' '-u'
trace: built-in: git 'symbolic-ref' '-q' 'HEAD'
trace: built-in: git 'config' '--bool' 'branch.#hidden#.rebase'
trace: built-in: git 'config' '--bool' 'pull.rebase'
trace: built-in: git 'rev-parse' '-q' '--verify' 'HEAD'
trace: built-in: git 'fetch' '--update-head-ok'
trace: run_command: 'ssh' '-p' '7777' 'git@#hidden.tld#' 'git-upload-pack '''/#hidden#/#hidden#.git''''
amazon-web-services amazon-ec2 git gitlab
asked 13 hours ago
ShreyasShreyas
1264
add a comment |
EDIT: It turns out this is a Gitlab problem, however I still do not have a solution.
I have a weird situation going on with two of my AWS EC2 instances. They are exactly the same in terms of OS, region, and instance type (both t3.micro), set up in the same way (however, the first was set up a few months ago).
Both exist in the eu-central-1c availability zone, and both are operating on the same git repository. Both are also up to date (CentOS 7.6.1810).
Older server:
$ time git pull
Already up-to-date.
real 0m0.306s
user 0m0.034s
sys 0m0.016s
Newer server:
$ time git pull
Already up-to-date.
real 2m7.547s
user 0m0.026s
sys 0m0.024s
It also consistently takes about 2m7s.
Also:
Older server:
--2019-04-09 10:52:03-- https://speed.hetzner.de/1GB.bin
Resolving speed.hetzner.de (speed.hetzner.de)... 88.198.248.254, 2a01:4f8:0:59ed::2
Connecting to speed.hetzner.de (speed.hetzner.de)|88.198.248.254|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048576000 (1000M) [application/octet-stream]
Saving to: ‘1GB.bin’
100%[===============================================================>] 1,048,576,000 121MB/s in 6.5s
2019-04-09 10:52:10 (154 MB/s) - ‘1GB.bin’ saved [1048576000/1048576000]
Newer server:
--2019-04-09 10:54:04-- https://speed.hetzner.de/1GB.bin
Resolving speed.hetzner.de (speed.hetzner.de)... 88.198.248.254, 2a01:4f8:0:59ed::2
Connecting to speed.hetzner.de (speed.hetzner.de)|88.198.248.254|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048576000 (1000M) [application/octet-stream]
Saving to: ‘1GB.bin’
100%[===============================================================>] 1,048,576,000 130MB/s in 5.9s
2019-04-09 10:54:10 (170 MB/s) - ‘1GB.bin’ saved [1048576000/1048576000]
EDIT: I tried to use a GitHub repository instead of our GitLab, and it turns out it seems to be a GitLab issue. What could possibly be causing GitLab to respond quickly to the older server but not to the other one?
EDIT 2: Attempted to clone over HTTPS. It takes 2 minutes just for it to ask for my username.
Also, verbose output over SSH:
$ GIT_CURL_VERBOSE=1 GIT_TRACE=1 git pull
trace: exec: 'git-pull'
trace: run_command: 'git-pull'
trace: built-in: git 'rev-parse' '--git-dir'
trace: built-in: git 'rev-parse' '--is-bare-repository'
trace: built-in: git 'rev-parse' '--show-toplevel'
trace: built-in: git 'ls-files' '-u'
trace: built-in: git 'symbolic-ref' '-q' 'HEAD'
trace: built-in: git 'config' '--bool' 'branch.#hidden#.rebase'
trace: built-in: git 'config' '--bool' 'pull.rebase'
trace: built-in: git 'rev-parse' '-q' '--verify' 'HEAD'
trace: built-in: git 'fetch' '--update-head-ok'
trace: run_command: 'ssh' '-p' '7777' 'git@#hidden.tld#' 'git-upload-pack '''/#hidden#/#hidden#.git''''
amazon-web-services amazon-ec2 git gitlab
asked 13 hours ago
ShreyasShreyas
1264
add a comment |
EDIT: It turns out this is a Gitlab problem, however I still do not have a solution.
I have a weird situation going on with two of my AWS EC2 instances. They are exactly the same in terms of OS, region, and instance type (both t3.micro), set up in the same way (however, the first was set up a few months ago).
Both exist in the eu-central-1c availability zone, and both are operating on the same git repository. Both are also up to date (CentOS 7.6.1810).
Older server:
$ time git pull
Already up-to-date.
real 0m0.306s
user 0m0.034s
sys 0m0.016s
Newer server:
$ time git pull
Already up-to-date.
real 2m7.547s
user 0m0.026s
sys 0m0.024s
It also consistently takes about 2m7s.
Also:
Older server:
--2019-04-09 10:52:03-- https://speed.hetzner.de/1GB.bin
Resolving speed.hetzner.de (speed.hetzner.de)... 88.198.248.254, 2a01:4f8:0:59ed::2
Connecting to speed.hetzner.de (speed.hetzner.de)|88.198.248.254|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048576000 (1000M) [application/octet-stream]
Saving to: ‘1GB.bin’
100%[===============================================================>] 1,048,576,000 121MB/s in 6.5s
2019-04-09 10:52:10 (154 MB/s) - ‘1GB.bin’ saved [1048576000/1048576000]
Newer server:
--2019-04-09 10:54:04-- https://speed.hetzner.de/1GB.bin
Resolving speed.hetzner.de (speed.hetzner.de)... 88.198.248.254, 2a01:4f8:0:59ed::2
Connecting to speed.hetzner.de (speed.hetzner.de)|88.198.248.254|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048576000 (1000M) [application/octet-stream]
Saving to: ‘1GB.bin’
100%[===============================================================>] 1,048,576,000 130MB/s in 5.9s
2019-04-09 10:54:10 (170 MB/s) - ‘1GB.bin’ saved [1048576000/1048576000]
EDIT: I tried to use a GitHub repository instead of our GitLab, and it turns out it seems to be a GitLab issue. What could possibly be causing GitLab to respond quickly to the older server but not to the other one?
EDIT 2: Attempted to clone over HTTPS. It takes 2 minutes just for it to ask for my username.
Also, verbose output over SSH:
$ GIT_CURL_VERBOSE=1 GIT_TRACE=1 git pull
trace: exec: 'git-pull'
trace: run_command: 'git-pull'
trace: built-in: git 'rev-parse' '--git-dir'
trace: built-in: git 'rev-parse' '--is-bare-repository'
trace: built-in: git 'rev-parse' '--show-toplevel'
trace: built-in: git 'ls-files' '-u'
trace: built-in: git 'symbolic-ref' '-q' 'HEAD'
trace: built-in: git 'config' '--bool' 'branch.#hidden#.rebase'
trace: built-in: git 'config' '--bool' 'pull.rebase'
trace: built-in: git 'rev-parse' '-q' '--verify' 'HEAD'
trace: built-in: git 'fetch' '--update-head-ok'
trace: run_command: 'ssh' '-p' '7777' 'git@#hidden.tld#' 'git-upload-pack '''/#hidden#/#hidden#.git''''
amazon-web-services amazon-ec2 git gitlab
asked 13 hours ago
ShreyasShreyas
1264
EDIT: It turns out this is a Gitlab problem, however I still do not have a solution.
I have a weird situation going on with two of my AWS EC2 instances. They are exactly the same in terms of OS, region, and instance type (both t3.micro), set up in the same way (however, the first was set up a few months ago).
Both exist in the eu-central-1c availability zone, and both are operating on the same git repository. Both are also up to date (CentOS 7.6.1810).
Older server:
$ time git pull
Already up-to-date.
real 0m0.306s
user 0m0.034s
sys 0m0.016s
Newer server:
$ time git pull
Already up-to-date.
real 2m7.547s
user 0m0.026s
sys 0m0.024s
It also consistently takes about 2m7s.
Also:
Older server:
--2019-04-09 10:52:03-- https://speed.hetzner.de/1GB.bin
Resolving speed.hetzner.de (speed.hetzner.de)... 88.198.248.254, 2a01:4f8:0:59ed::2
Connecting to speed.hetzner.de (speed.hetzner.de)|88.198.248.254|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048576000 (1000M) [application/octet-stream]
Saving to: ‘1GB.bin’
100%[===============================================================>] 1,048,576,000 121MB/s in 6.5s
2019-04-09 10:52:10 (154 MB/s) - ‘1GB.bin’ saved [1048576000/1048576000]
Newer server:
--2019-04-09 10:54:04-- https://speed.hetzner.de/1GB.bin
Resolving speed.hetzner.de (speed.hetzner.de)... 88.198.248.254, 2a01:4f8:0:59ed::2
Connecting to speed.hetzner.de (speed.hetzner.de)|88.198.248.254|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048576000 (1000M) [application/octet-stream]
Saving to: ‘1GB.bin’
100%[===============================================================>] 1,048,576,000 130MB/s in 5.9s
2019-04-09 10:54:10 (170 MB/s) - ‘1GB.bin’ saved [1048576000/1048576000]
EDIT: I tried to use a GitHub repository instead of our GitLab, and it turns out it seems to be a GitLab issue. What could possibly be causing GitLab to respond quickly to the older server but not to the other one?
EDIT 2: Attempted to clone over HTTPS. It takes 2 minutes just for it to ask for my username.
Also, verbose output over SSH:
$ GIT_CURL_VERBOSE=1 GIT_TRACE=1 git pull
trace: exec: 'git-pull'
trace: run_command: 'git-pull'
trace: built-in: git 'rev-parse' '--git-dir'
trace: built-in: git 'rev-parse' '--is-bare-repository'
trace: built-in: git 'rev-parse' '--show-toplevel'
trace: built-in: git 'ls-files' '-u'
trace: built-in: git 'symbolic-ref' '-q' 'HEAD'
trace: built-in: git 'config' '--bool' 'branch.#hidden#.rebase'
trace: built-in: git 'config' '--bool' 'pull.rebase'
trace: built-in: git 'rev-parse' '-q' '--verify' 'HEAD'
trace: built-in: git 'fetch' '--update-head-ok'
trace: run_command: 'ssh' '-p' '7777' 'git@#hidden.tld#' 'git-upload-pack '''/#hidden#/#hidden#.git''''
amazon-web-services amazon-ec2 git gitlab
amazon-web-services amazon-ec2 git gitlab
asked 13 hours ago
ShreyasShreyas
1264
asked 13 hours ago
ShreyasShreyas
1264
edited 13 hours ago
Shreyas
asked 13 hours ago
ShreyasShreyas
1264
asked 13 hours ago
ShreyasShreyas
1264
asked 13 hours ago
ShreyasShreyas
1264
1264
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Problem found using verbose output.
The newer server was trying to contact the git endpoint server using IPv6, and waiting for timeout, before falling back to IPv4 (which actually works).
$ GIT_CURL_VERBOSE=1 GIT_TRACE=1 git clone https://#hidden#/#hidden#/#hidden#.git
trace: built-in: git 'clone' 'https://#hidden#/#hidden#/#hidden#.git'
Cloning into '#hidden#'...
trace: run_command: 'git-remote-https' 'origin' 'https://#hidden#/#hidden#/#hidden#.git'
* Couldn't find host #hidden# in the .netrc file; using defaults
* About to connect() to #hidden# port 443 (#0)
* Trying x:x:x:x:x:x:x:x...
* Connection timed out
* Trying x.x.x.x...
answered 13 hours ago
ShreyasShreyas
1264
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "2"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f962194%2fgitlab-git-operations-are-extremely-slow-on-ec2%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Problem found using verbose output.
The newer server was trying to contact the git endpoint server using IPv6, and waiting for timeout, before falling back to IPv4 (which actually works).
$ GIT_CURL_VERBOSE=1 GIT_TRACE=1 git clone https://#hidden#/#hidden#/#hidden#.git
trace: built-in: git 'clone' 'https://#hidden#/#hidden#/#hidden#.git'
Cloning into '#hidden#'...
trace: run_command: 'git-remote-https' 'origin' 'https://#hidden#/#hidden#/#hidden#.git'
* Couldn't find host #hidden# in the .netrc file; using defaults
* About to connect() to #hidden# port 443 (#0)
* Trying x:x:x:x:x:x:x:x...
* Connection timed out
* Trying x.x.x.x...
answered 13 hours ago
ShreyasShreyas
1264
add a comment |
Problem found using verbose output.
The newer server was trying to contact the git endpoint server using IPv6, and waiting for timeout, before falling back to IPv4 (which actually works).
$ GIT_CURL_VERBOSE=1 GIT_TRACE=1 git clone https://#hidden#/#hidden#/#hidden#.git
trace: built-in: git 'clone' 'https://#hidden#/#hidden#/#hidden#.git'
Cloning into '#hidden#'...
trace: run_command: 'git-remote-https' 'origin' 'https://#hidden#/#hidden#/#hidden#.git'
* Couldn't find host #hidden# in the .netrc file; using defaults
* About to connect() to #hidden# port 443 (#0)
* Trying x:x:x:x:x:x:x:x...
* Connection timed out
* Trying x.x.x.x...
answered 13 hours ago
ShreyasShreyas
1264
add a comment |
Problem found using verbose output.
The newer server was trying to contact the git endpoint server using IPv6, and waiting for timeout, before falling back to IPv4 (which actually works).
$ GIT_CURL_VERBOSE=1 GIT_TRACE=1 git clone https://#hidden#/#hidden#/#hidden#.git
trace: built-in: git 'clone' 'https://#hidden#/#hidden#/#hidden#.git'
Cloning into '#hidden#'...
trace: run_command: 'git-remote-https' 'origin' 'https://#hidden#/#hidden#/#hidden#.git'
* Couldn't find host #hidden# in the .netrc file; using defaults
* About to connect() to #hidden# port 443 (#0)
* Trying x:x:x:x:x:x:x:x...
* Connection timed out
* Trying x.x.x.x...
answered 13 hours ago
ShreyasShreyas
1264
Problem found using verbose output.
The newer server was trying to contact the git endpoint server using IPv6, and waiting for timeout, before falling back to IPv4 (which actually works).
$ GIT_CURL_VERBOSE=1 GIT_TRACE=1 git clone https://#hidden#/#hidden#/#hidden#.git
trace: built-in: git 'clone' 'https://#hidden#/#hidden#/#hidden#.git'
Cloning into '#hidden#'...
trace: run_command: 'git-remote-https' 'origin' 'https://#hidden#/#hidden#/#hidden#.git'
* Couldn't find host #hidden# in the .netrc file; using defaults
* About to connect() to #hidden# port 443 (#0)
* Trying x:x:x:x:x:x:x:x...
* Connection timed out
* Trying x.x.x.x...
answered 13 hours ago
ShreyasShreyas
1264
edited 13 hours ago
answered 13 hours ago
ShreyasShreyas
1264
answered 13 hours ago
ShreyasShreyas
1264
answered 13 hours ago
ShreyasShreyas
1264
1264
add a comment |
add a comment |
Thanks for contributing an answer to Server Fault!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f962194%2fgitlab-git-operations-are-extremely-slow-on-ec2%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
