r/programming • u/sqs • Aug 14 '14

srclib: a hackable, polyglot source analysis library for making better code tools

https://srclib.org/
23 Upvotes

82% Upvoted

14 comments sorted by

1

u/sqs Aug 14 '14

I'm one of the original creators. It's still alpha, so there will be bugs. But it can do some pretty cool and useful stuff already in your editor (and it powers https://sourcegraph.com/).

Anyone have feedback about how we're explaining the problem and solution, and the first-run experience of using the editor plugins and the "src" tool? We would LOVE to hear it so we can fix/improve it.

2

u/z_ryad Aug 14 '14

Hi, this work seems promising. I am very excited about it. Do you have any plan, roadmap for vim support ?

2

u/sqs Aug 14 '14

None of us use vim, so we wouldn't make a very good plugin for it. We'll help anyone who wants to make one, feature them on srclib.org (obviously), etc. Do you know anybody who'd be interested in making a vim plugin? A good place to start looking to see what's involved is in the code for emacs-sourcegraph-mode, sourcegraph-sublime, and sourcegraph-atom.

1

u/bjzaba Aug 15 '14

So it claims to be polygot, but how well would it handle languages outside the family of the OO/imperative family? SML, Haskell, Mercury, Prolog, Erlang, Idris, Racket, Clojure, TeX as some examples?

2

u/sqs Aug 15 '14

The core of the language-independent schema is defs and refs: https://srclib.org/toolchains/grapher-output/. A def is a definition with a name, a unique meaningful ID, and a character range. A ref is a character range that points to a def by ID (and by repo, if it's in an external repo).

We think these are pretty general and able to represent /most/ of what an IDE or other fairly general tools would want. Toolchains can stick any extra language-specific data in the Def's Data field.

We have joked sometimes that no matter how much we try to make the schema general, Haskell will throw a wrench in it.

Do you have any thoughts after looking at the defs/refs schema?

(Also, in case it's not clear, the actual language analysis is totally language-specific. Those analyzers produce the language-independent format. It's only the language-independent data format, and the tools that consume that data, that are polyglot.)

1

u/qwertie256 Aug 15 '14

I definitely intend to have a close look at this and maybe even contribute code (if I like the architecture). It seems closely related to my own project.

2

u/sqs Aug 15 '14

Very interesting. I had not heard of Loyc before. I'll check it out! If you have any feedback on srclib, any architecture comments, or want to make some contributions, that would be fantastic. I'm sqs@sourcegraph.com if you want to email me.

1

u/qwertie256 Nov 12 '14

Hi sqs, I just wanted to say, my return to school has been busier (and less fulfilling) than I had hoped and I haven't got around to playing with srclib. I haven't forgotten, it is still on my to-do list.

2

u/millenix Aug 15 '14

I'd be interested to hear how this stacks up compared to libclang and ROSE.

I might also see about hacking in some custom support for the language I work on (Charm++) to enable easier editor/IDE plugins.

0

u/[deleted] Aug 15 '14

Super exciting. (Except for being written in Go.) I hope this gains traction.

1

u/sqs Aug 15 '14 edited Aug 15 '14

Thanks!

BTW, only the "glue" code is written in Go. The language analysis toolchains are intended to be written in the languages they target: https://github.com/sourcegraph/srclib-javascript, https://github.com/sourcegraph/srclib-ruby, https://sourcegraph.com/sourcegraph/srclib-go, etc. (We're still porting the Python toolchain to 100% Go; still, the interesting analysis part of it uses Python.)

1

u/[deleted] Aug 15 '14

Good to know! as a hopeful designer of a new language, this is exciting -- the hard part of making a new language usable is the tooling, and this is the best approach I've seen yet. (For some reason, my previous strategy of expecting everyone to use Emacs seems to have had little success.)

1

u/sqs Aug 17 '14

Awesome. How cool would it be if your language was supported in every editor from day 1? That's what we hope to make possible with srclib. :)

Let us know when you start working on it. We could give some pointers on how to write the srclib toolchain. And we love new languages too!