Been a minute since I’ve done one of these comparisons. The first one took place in 2016 with annual follow ups in 2017 and 2018. Then there was the curious case of 2019 where I had this post in my queue but never actually ran the benchmarks.
I think the reason I never followed through is because the script I wrote to generate these metrics was in severe need of some lovin’ and at the time, I either didn’t want to put in the effort, or I ran into some hurdle that caused me to put a pin in it.
With that said, as I’m now primarily using GitLab for my private personal projects and still utilizing GitHub for most of my open source contributions, I thought it would be good to revisit these metrics.
Sure beats yet another opportunistic developer post on how to map COVID-19 data, amirite?
As mentioned, the script I wrote to knock this out did need some work, so I
did a bit of rework to the script to help simulate a bit more of a real world
scenario.
The script goes through cloning my test repository, then modifying a file,
committing and pushing it, then doing a pull. Each interaction with the code
hosting provider is run for 100 iterations and timed (with time). I threw the
results into a spreadsheet to grab the metrics published below.
Since the git pull portion of the script doesn’t pull anything new or changed
down, it falls a bit short. For a future iteration of this, I plan to have an additional copy of of the repository that I can make changes to then time pulling said changes down in my test bed.
Because I feel that doing a git push happens a bit more often than git pull in my day to day work flow, I don’t think too much is lost by not shoring that part of the benchmarks up for this post.
Fairly consistent with the previous benchmarks, GitHub did perform better in terms of running git clone, but with both git push and git pull GitLab’s totals just snuck by.
Here’s the results, which I focused in on the real output from time:
| git@github.com:joshtronic/test.git | git@gitlab.com:joshtronic/test.git | |||||
|---|---|---|---|---|---|---|
| git clone | git push | git pull | git clone | git push | git pull | |
| Minimum | 0.716s | 0.003s | 0.009s | 1.114s | 0.003s | 0.009s |
| Maximum | 1.217s | 0.005s | 0.015s | 4.339s | 0.006s | 0.015s |
| Average | 0.784s | 0.003s | 0.010s | 1.362s | 0.003s | 0.010s |
| Median | 0.773s | 0.003s | 0.010s | 1.195s | 0.003s | 0.010s |
| Mode | 0.753s | 0.003s | 0.010s | 1.191s | 0.003s | 0.009s |
| Total | 78.448s | 0.329s | 1.016s | 136.212s | 0.317s | 0.977s |
If you’re interested in the data from all 100 iterations, you can check out the raw data in this Google Sheet.
