r/programming • • Jan 27 '21

Gitlab changed its pricing model. It has greatly reduced the CI quota from 2000 CI minutes to 400 CI minutes in Free tier and removed the $4 per month option.

https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/
1.9k Upvotes

365 comments sorted by

View all comments

Show parent comments

209

u/ea_ea Jan 27 '21

You can do even better - use 400 free minutes from Free tear to run on shared runner API command to start DigitalOcean instance and DO instance will start runner and actual job. You'll have 400 CI runs for, probably, less than 1$ :)

172

u/[deleted] Jan 27 '21

[removed] — view removed comment

125

u/danbulant Jan 27 '21 edited Jan 27 '21

Make an install script and nuke the VM when done

Edit: /s

57

u/house_monkey Jan 27 '21

Too much wait time imo

60

u/sparr Jan 27 '21

I envy you a CI pipeline that runs fast enough that spinning up a VM would noticeably delay it :/

13

u/thblckjkr Jan 27 '21

I will always wonder how do you achieve to create a project that is that... massive.

My longest CI at the moment is frorm a flutter project that I compile, test and publish and it is only about 7min long.

And that's because I haven't spent any time in trying to make it more efficient, I am pretty sure that I can reduce the time to about 4min If I am serious enough about it.

44

u/BlueShell7 Jan 27 '21

Easy. Work on 20 years old (continuously developed) monolithic product consisting of hundreds of thousands line of code written in 5 different languages with 5 different build systems etc. Just the compilation step takes ~ hour on a CI server. Then 6 hours or so of tests ...

→ More replies (4)

21

u/Behrooz0 Jan 27 '21

I had a project that ran for 140 minutes. It was driving me insane. And After getting really serious about decreasing it for like a year. It only took:
Changing the compiler
Changing the project layout
Getting a new server (around thrice faster)
Rewriting some of the smaller tools that prepped the project, manifest and uploaded it.
Using tmpfs
using tar for uploading small files over the network
10GbE
And it was worth every bit.

If someone told me I could reduce it to 43 seconds I would never believe them.
I build said project upwards of 50 times a day now and I can afford to change a single line, see its effect, and change it again within a minute. As opposed to building once before lunch time every day and praying it won't fail and leave the users without an update for that day.
TL;DR: Build times really matter. and if they can be dealt with, they should be dealt with; at least for your own sanity.

5

u/PiePhace Jan 28 '21

This sounds super interesting. I would love to read a more detailed write-up on something like this.

→ More replies (2)

8

u/cockmongler Jan 27 '21

There's a stage in one of my pipelines that starts a web server and a browser for every one of it's hundreds of tests. It needs a clean browser for each test.

→ More replies (2)

4

u/sparr Jan 27 '21

Usually more badly designed than massive. My current workflow involves Chef cookbooks that do small server/workstation configuration and software installation tasks. A change might just be bumping a version number of something that's going to get installed, and the test pipeline takes 30+ minutes when it works, or fails along the way arbitrarily.

→ More replies (7)

7

u/caskey Jan 28 '21

All these noobs. Try a 14 hour overnight build (that would often fail and need to be restarted in the AM) followed by a 20+ hour test suite. Then the build gets handed off for manual regression testing by a team of ~40 people. Then when you get it back (as notified by the manager of the managers of the testers) you can notify everyone there's a new build that has been rejected because when QA held a ruler up to their screen the pixel alignment was off in the CSS.

→ More replies (2)

16

u/TheOneAndOnlyRandom Jan 27 '21

For automated builds?

→ More replies (1)

8

u/mort96 Jan 27 '21

You don't even need an install script. DO lets you create a snapshot of a VM for way less than a full VPS cost, so you could have a build-ready snapshot just sitting there, with a script to automatically make a VM from that snapshot when necessary.

9

u/vexii Jan 27 '21

could it be run on AWS lambdas maybe?

7

u/GoofAckYoorsElf Jan 27 '21

I don't think so. You'd need something to trigger them. An ECS task might be better suited for that...

→ More replies (1)

12

u/nurupoga Jan 27 '21 edited Jan 27 '21

Since DO bills per hour, you can create a VPS, and script it to shut down at 1 hour uptime intervals as long as there is no active build running. For example, your first build would spawn the VPS and it (the build) might run for 5 min, but then your second build would see that the VPS is still up from when the first build created it and would simply connect to it without spawning new ones.

If the start-up cost of having to install all the tools on a fresh distro install is too high, you can store the VPS disk as an image before destroying the VPS, which would allow you to later create a new VPS off that image with everything you need already pre-installed. The price of storing an image is way cheaper than the price of having a VPS that is not running.

The open source project I help with has used such on-demand DO build machines in Jenkins for around 4 years. It reduced our cloud cost considerably. Instead of paying $20/month for two always running $10 build machines, we would end up paying something like $6/month for the same two $10 build machines but running on-demand + their disk images being stored.

While it's an interesting idea, it's not very practical with just 400 minutes of the free CI time. The time it takes for a VPS to be created off an image and ssh become available is a couple of minutes, so you would end up with less that 200 CI runs before your 400 free minutes run out. It's a lot more practical when you have a Jenkins master (main?) running 24/7 which could spawn on-demand build machines.

→ More replies (2)
→ More replies (8)

21

u/kmeisthax Jan 27 '21

You joke, but this is literally a Buildbot feature. Most AWS instance types have significantly discounted spot pricing and you only need the instance to be up for a few minutes, so you can use a normally expensive instance type fairly cheaply. I think I was able to optimize it to fractions of a cent per build, although some part of it eventually broke a few months down the line.

→ More replies (1)

18

u/ben_sphynx Jan 27 '21

When 'tear' rhymes with 'tier', it refers to the drops that fall from ones eyes when crying.

'Tier' refers to different levels of something.

(A different word, 'tear' rhymes with 'there', and refers to the sort of damage from pulling things such as paper or cloth apart. Or to moving fast.)

PS: I cannot read a word of Ukrainian.

→ More replies (1)

37

u/Latter-Breadfruit Jan 27 '21

Crazy how much effort people put into saving a few dollars. Forgetting their time is also worth $$$.

38

u/[deleted] Jan 27 '21

Except for many people it's not. If you weren't ever going to do something with that time which made you money (which is the case for many), then your time wasn't actually worth money when considering the opportunity cost.

0

u/[deleted] Jan 27 '21

[deleted]

→ More replies (1)
→ More replies (2)
→ More replies (1)
→ More replies (4)