Been a minute since I’ve done one of these comparisons. The first one
took place in 2016 with annual follow ups in 2017 and 2018. Then
there was the curious case of 2019 where I had this post in my queue but never
actually ran the benchmarks.
I think the reason I never followed through is because the script I wrote to
generate these metrics was in severe need of some lovin’ and at the time, I
either didn’t want to put in the effort, or I ran into some hurdle that caused
me to put a pin in it.
With that said, as I’m now primarily using GitLab for my private personal
projects and still utilizing GitHub for most of my open source contributions, I
thought it would be good to revisit these metrics.
Sure beats yet another opportunistic developer post on how to map COVID-19 data,
amirite?
As mentioned, the script I wrote to knock this out did need some work, so I
did a bit of rework to the script to help simulate a bit more of a real world
scenario.
The script goes through cloning my test repository, then modifying a file,
committing and pushing it, then doing a pull. Each interaction with the code
hosting provider is run for 100 iterations and timed (with time). I threw the
results into a spreadsheet to grab the metrics published below.
Since the git pull portion of the script doesn’t pull anything new or changed
down, it falls a bit short. For a future iteration of this, I plan to have an
additional copy of of the repository that I can make changes to then time
pulling said changes down in my test bed.
Because I feel that doing a git push happens a bit more often than git pull
in my day to day work flow, I don’t think too much is lost by not shoring that
part of the benchmarks up for this post.
Fairly consistent with the previous benchmarks, GitHub did perform better in
terms of running git clone, but with both git push and git pull GitLab’s
totals just snuck by.
Here’s the results, which I focused in on the real output from time:
| git@github.com:joshtronic/test.git | git@gitlab.com:joshtronic/test.git | |||||
|---|---|---|---|---|---|---|
| git clone | git push | git pull | git clone | git push | git pull | |
| Minimum | 0.716s | 0.003s | 0.009s | 1.114s | 0.003s | 0.009s |
| Maximum | 1.217s | 0.005s | 0.015s | 4.339s | 0.006s | 0.015s |
| Average | 0.784s | 0.003s | 0.010s | 1.362s | 0.003s | 0.010s |
| Median | 0.773s | 0.003s | 0.010s | 1.195s | 0.003s | 0.010s |
| Mode | 0.753s | 0.003s | 0.010s | 1.191s | 0.003s | 0.009s |
| Total | 78.448s | 0.329s | 1.016s | 136.212s | 0.317s | 0.977s |
If you’re interested in the data from all 100 iterations, you can check out the
raw data in this Google Sheet.
