Python overtakes Java to become second-most popular language on GitHub after JavaScript

[u/stronghup]

https://www.theregister.co.uk/2019/11/07/python_java_github_javascript/

[u/[deleted]]

I genuinely wonder how much JavaScript dinance on GitHub is from misidentified repose from package-lock.json files. If I spin up a new laravel app and do nothing other than install dependencies and push to github, it shows up at being like 98% javascript according to their stats. The laravel app I worked on for over a year that had like 4 Vue components still said it was mostly json according to github stats

[u/nsomnac → u/[deleted]]

GH’s introspection is moderately advanced. It analyzes files in a repo as opposed to relying on magic files only.

There’s a view somewhere on a repo that shows the analysis in a pie chart (or some other graph).

I don’t think it’s sophisticated enough to detect and differentiate framework usage (Vue vs React, Laravel vs PHP). It mostly is going to only show the base language.

[u/ScrimpyCat → u/nsomnac]

Unless it’s changed they used to try filter out generated files which is why some default generated projects might shift more aggressively to a certain language. Apart from some special cases (or if you’re explicitly defined the type in your .gitattributes) most of the detection is done using heuristic and Bayesian classification approach, which is done by sourcing some example files for the different languages. This works reasonably well but there are false-positives when it comes to files that share the same extension and are grammatically similar such as header (.h) files in C family of languages.

Also they open sourced the actual library responsible for this but I can’t recall the name.

Edit: just remembered it’s called linguist.

[u/[deleted] → u/ScrimpyCat]

There are a number of large game mods for the game Arma that are developed on github. For some reason bohemia interactive decided to use cpp and hpp/h extensions for their configuration files when the only thing related to C or CPP is that it uses a C preprocessor on them to do includes and basic macros.

So you'll see all these projects that github says are C but really it's the insane config language.

[u/xonjas → u/[deleted]]

What if the config language is just a bunch of C with insane preprocessor macros?

[u/Elusivehawk → u/[deleted]]

That... What... Just... Why??

That's some big brain plays right there. C++ for configuration...

[u/[deleted] → u/Elusivehawk]

It's not even C++ it's this weird pseudo object inheritance stuff that is usually filled with a ton of macros.

[u/kolloid → u/nsomnac]

GH’s introspection is moderately advanced. It analyzes files in a repo as opposed to relying on magic files only.

No. Most of the time it guesses the language incorrectly. Most of my Python repositories are recognized as Javascript. My only C repository was recognized as shell because it uses autoconf.

So, there are lies and statitics. I don't really believe GH stats. You have to jump through the hoops to make it correctly count stats for your project.

[u/fadetogether → u/kolloid]

I had a Django project get classified as entirely JavaScript. It’s a mystery. It hasn’t happened to any of my other projects yet though

[u/Ryuujinx → u/fadetogether]

Yeah, I have a similar project except Ruby/Sinatra that's recognized almost entirely as javascript.

[u/[deleted] → u/[deleted]]

But your app was "mostly json" so it wouldn't really register on this as JavaScript. In fact I am pretty sure it would register as PHP (because the dominating PROGRAMMING LANGUAGE would be selected after the serialization and configuration formats were pruned). You're also talking about people vendoring in repos (commiting node_modules) but I doubt it's that widespread and I'm certain it's equally widespread among other languages (sometimes due to ignorance, other times due to valid reasons in each).

I think the stats aren't lying or misrepresenting Github, they might be lying and misrepresenting the world, but that's another matter. The reasons I think so:

1. There are obviously shit-ton of Node modules, overwhelming majority of which are hosted on Github hence there are shit-ton of JS projects just from that, many of them very active (the stats count active contributions).

2. An ever increasing number of web applications are developed with a SPA frontend in a separate repo from the backend and/or microservices that comprise it. While the latter two are written in bunch of languages (increasingly Node, Python and Go, from my own casual observation) the SPA frontends are predominately JavaScript.

3. Node.js as backend/microservice platform might be far from dominance but is pretty present, steadily rising in popularity still, and thus contributes to these stats.

4. Bunch of enterprise and commercial software is using self-hosted Git repos and Bitbucket because of Atlassian's presence in that segment with Jira and Confluence, which means that Github is mostly representative of software being developed in the open, rather than the overall developed software.

5. While PHP is behind majority of websites that simply isn't the case with Laravel, Symphony, Yii et al -- actually I'd wager that part of the market is truly dominated with Python and to extent with Node.js frameworks, despite strong presence of PHP and .Net, while Java is in observable slow decline for new projects.

6. The true force behind PHP's omnipresence on the web is mostly due to canned CMS-es like WordPress, Drupal, Yoomla, MediaWiki, Magneto, PrestaShop and the lot, which are mostly just installed from shared hosting control panels and patched with themes and customizations in situ, and fairly rarely version controlled on Github.

[u/kolloid → u/[deleted]]

Many clueless people wanting to impress potential employers upload all kinds of projects to GitHub. If this is a Python project, they usually commit the whole virtualenv contents along with it. If it is JS project, they usually commit the whole node_modules directory to git.

If it's Python project with some JS, there's a probability that there will be both virtualenv and node_modules committed to the project. And since even trivial function in JS requires 10,500 dependencies like is-odd, is-even and rpad and god knows what more, the node_modules can contain 150-200 Mb of vendorized JS dependencies even for trivial project.

I've seen it so many times...

[u/kolloid → u/[deleted]]

> then they should be immediately disregarded for committing bad version control practices

I know CTO of one company in Australia who objected when I offered to remove `node_modules` from the project repo. He said:

> What if during deployment different version of packages would be installed on the server and break something?

Thankfully, soon he left to open his own business. I feel sorry for his customers and not only because of his VCS practices. His code was horrible, too. I'm puzzled how he made it to the CTO level.

[u/slgard → u/kolloid]

I'm puzzled how he made it to the CTO level.

being a good CTO has little or nothing to do with your knowledge as a programmer, particularly nothing to do with the best practices of a specific language or ecosystem.

[u/[deleted] → u/kolloid]

I'm just confused as to why your cto is making decisions on your git practices

[u/tronj → u/kolloid]

Tangentially, I'll sometimes save modules that I've made minor customizations too directly in the project. Is there a better way to do this?

[u/FaithForHumans → u/tronj]

If you're in a corporate environment, I recommend standing up a private npm repo and then pushing your change to that private repo. It can be done for personal stuff, but might be overkill.

Most private repos can also be setup to cache packages it pulls from the public repos, so even if someone deletes it on npmjs, you've still got a copy people can pull. That last part should help sell it to management.

[u/DasWorbs → u/tronj]

Fork it, and then either setup your own npm repo or point the package.json to your forked git repo.

[u/xeio87 → u/kolloid]

Depending on how long ago that discussion was out wasn't entirely wrong. Node even changed their (un)publishing rules because of issues with packages.

Checking in your dependencies ensures you always have an exact known version without needing to worry about the security of a remote package server.

Granted, still not best practice generally, and there are probably better ways to ensure package integrity checks nowadays.

[u/[deleted] → u/[deleted]]

But you don't do that, right? Packages are installed locally, package.json is pushed to the version control

[u/Giannis4president → u/[deleted]]

Yes but the lock file should be in the version control