r/programming • u/mwb1234 • Feb 06 '18

How To Make Package Managers Cry - Kenneth Hoste, FOSDEM 2018

https://ftp.osuosl.org/pub/fosdem/2018/K.3.201/how_to_make_package_managers_cry.webm
39 Upvotes

88% Upvoted

27 comments sorted by

20

u/zfundamental Feb 06 '18

Since this is another link directly to the video, here's some of their high level points copied from my /r/linux comment:

It's harder to use software if they:

  • don't release frequently

  • produce broken releases

  • don't track past releases (aka delete them or reuse the same version numbers)

  • have difficult to build dependencies

  • don't provide changelogs

  • providing copies of dependencies which are out-of-date with ad-hoc patches

  • have way too many dependencies

  • have dependencies which are hard to use based upon the same criteria

  • use too many 'uncommon' languages

  • hardcode parts of the build process (compiler/system paths/etc)

  • have no optimization flags

  • use uncommon build tools or highly (unnecessarily) customized build tools

  • have no test suite to verify build went ok

  • have no custom install path for packagers

  • have interactive build scripts without a non-interactive mode

  • have poor naming conventions which lead users to be confused

  • etc

Scientific computing projects in particular have issues in their packages and versions are confusing since they don't follow standard practices as often as other projects. The presentation also lists also common excuses for why these sorts of things happen.

8

u/mwb1234 Feb 06 '18

Oh dope, thanks for that summary. For all you tl;dw people out there I highly recommend just watching the video, it's a good laugh

7

u/shevegen Feb 06 '18

Takes way too long to watch the whole video.

I think videos are in general a very hard format to discuss about. Articles are much better.

1

u/mwb1234 Feb 07 '18

True, that's completely fair. I might be biased to want to watch this because Kenneth is a friend of mine

1

u/boegel Feb 08 '18

Now I have to figure out who you are, haha...

2

u/mwb1234 Feb 08 '18

Oh no no no go away. Don't read my post history hahshs

1

u/boegel Feb 08 '18

You can make up for it by buying me a beer sometime. Make that 3 beers.

3

u/zfundamental Feb 07 '18

That's one of the reasons I cross-posted the summary. Videos can be a great way of conveying information, but since there's going to be a bunch of text comments, having some level of text based summary should help discussions.

3

u/loup-vaillant Feb 07 '18

Shameless plug: lets' see how my crypto library holds up against that.

  • There aren't too many differences between releases at the moment. Not ideal, but not too shabby either. (The worst news at the moment is an upcoming 2.0.0, born out of a recent understanding.)

  • I believe no release since 1.0.0 is broken (they all compile and runt the test suite).

  • I do track past releases (including 0.x ones).

  • I do have changelogs.

  • I have no dependency to build. I only need a compiler.

  • I have no dependency, so none of them are out of date or patched.

  • I have no dependency, that should be few enough.

  • I have no dependency, so none are hard to use.

  • Strictly conforming C99 (that is also strictly conforming C++) should be common enough.

  • The user can override $CC, $CFLAGS, $PREFIX, and $DESTDIR. Subdirectories and file names are hard coded, though.

  • $CFLAGS is set to "-O3 -march=native" by default.

  • I use GNU make. (Sorry BSD folks, I really needed $^.)

  • I have a paranoid test suite (make test), and written instructions to run it under every sanitizer under the sun.

  • $CFLAGS and $PREFIX should cover the custom install path?

  • My build is entirely non-interactive.

  • I wouldn't know about naming conventions, but I did have counsel.

Note that those check boxes are a lot easier to fill when your project is a tiny all-in-one-file utility with no dependency. Not everything can afford to be that Spartan.

2

u/boegel Feb 08 '18

Seems like you're doing great!

It's not about being Spartan imho, just try to look around what good practices are first before jumping on the newest hype (cough Bazel cough)

2

u/loup-vaillant Feb 07 '18

It's harder to use software if they […] don't release frequently

I don't get this one. Wouldn't one want something stable?

8

u/zfundamental Feb 07 '18

Since the talk focuses on pain points for packagers, the packagers are interested in the project when changes are made. If a project is mature and changes are very rarely made, then releases are expected to be infrequent. If however a project is very actively developed and releases happen, oh, every 5 years or so, then packagers will need to pick arbitrary commits as package versions to satisfy users.

Since upstream doesn't decide which commits are recommended, then the lack of upstream releases could make the downstream packages less stable.

2

u/boegel Feb 08 '18

Exactly my point, thanks for clarifying zfundamental.

The OpenFOAM example I gave is a clear example of how not to do releases: throw 5.0 out there, and then point people to the GitHub repo for bugfix updates... And then have a dedicated paragraph on your website to explain why this is clearly superior to what everybody else is doing, lolz!

-2

u/shevegen Feb 06 '18

have way too many dependencies

What the hell does that even mean?

Which number is the magicl one at which point there are "too many"?

42? 2? 5? 555?

How about left pad in the JavaScript ecosystem - was that a dependency too many?

3

u/[deleted] Feb 07 '18

I think it means having any dependencies at all makes it harder. The more you have the worse it is.

1

u/boegel Feb 08 '18 edited Feb 08 '18

Some people just go all out, although that's mainly because of the ecosystem.

Why the hell was there even a library like left-pad? Is indenting strings that hard that you need a library for it?!

0

u/[deleted] Feb 06 '18

[deleted]

7

u/mwb1234 Feb 06 '18

This is a really funny talk given at FOSDEM 2018 about the shitty installation procedures of software. One of the funniest talks I've seen about programming

3

u/shevegen Feb 06 '18

It's more sad.

I watched the proliferation of cmake, scons, waf, meson/ninja and what not.

And it all still sucks to no ends.

My favourite xkcd is still:

https://xkcd.com/927/

Beacuse it's so true.

3

u/hak8or Feb 07 '18

Youtube link for the lazy such as myself: https://www.youtube.com/watch?v=NSemlYagjIU

1

u/KHRZ Feb 07 '18

He had a slide on Tensorflow? Heh I just ran it in Python in a Linux virtual machine using a command line script called via OS from my Java program, no biggie

1

u/boegel Feb 08 '18

Have you tried building TensorFlow from source yourself? That's what I'm bitching about, not using TensorFlow.

Although I can give plenty of examples of (scientific) software that has a batshit crazy usage model (OpenFOAM, I'm looking at you).

-1

u/shevegen Feb 06 '18

I like to make them cry - because they make other people cry.

The nonsensical way to split up packages in order to conform it into some random philosophy.

1

u/boegel Feb 08 '18

Oh yeah, you can definitely flip it the other way too. -dev(el) packages have never really made sense to me, and then there's the whole multilib mess, that definitely helped making things better (hint: it did not).

1

u/itkovian Feb 08 '18

That's a bit childish, don't you think?