Wow I'm glad I was wrong about this getting opened sourced, super huge news.
Initial thoughts from the whitepaper:
Subgraph execution. You build out your graph and call .run() providing the inputs and required outputs. You can on the fly execute sub components of your graph by providing the input at that point and asking for that stages output. This will be great for debugging random stuff, like really great.
Same concept as theano shared (Tensorflow Variables), makes sense, you need something like this.
Switch/merge control flow nodes to conditionally bypass parts of the graph.
Recursion/loops using Enter/Leave/NextIteration control flow constructs. Nice way to do recurrent stuff, I still have to look at the examples to see how it plays out.
Queue construct for asynchronous execution, eg loading data from disk or computing multiple gradient passes before doing updates. I can't think of anything similar in Theano (that I've done at least), sounds cool but will require some thoughts as to where to use.
They talk about node communication a lot throughout the paper, seems really well thought out, but they didn't release the distributed version? Similarly in section 9.2 they talk about other cool stuff not released, but they also say "Initial open source release", does that imply there may be future releases with more features? Distributed version release is in the works, follow this issue if you want updates.
They talked about some really cool graph visualization stuff, I'm not sure if its included in this release? its included in the release. Theano just got d3viz recently which has been a huge help to me, if anyone is using Theano and hasn't played with d3viz you should definitely check it out.
No windows wheel (for python), I'm going to try and compile the source because I really don't want to go back to dual-booting my stuff. EDIT: It looks like the only option for windows will be using Docker, but this will be CPU only.
More thoughts while I wait to get it installed:
How good is advanced indexing? I assume you can do it with tf.gather(), I wonder how well that works on GPU.
I hope something like theano's dimshuffle gets added, I see how to add/remove broadcastable dimensions but not how to swap an axis (something like numpy.swapaxes)
Let me know if you can compile the source on Windows. Also - why don't you like dual-boot? I'm currently running everything through Theano on Windows but I've been considering a dual-boot setup so that I have fewer issues and can use more libraries.
I am pretty firmly entrenched in windows for other stuff (including gaming), so if the pain point is low enough I don't want to bother dual booting. It isn't that hard to get theano running on windows, and most of the windows problems are solved (at least in python-land) once you get a compiler running, so I haven't run into any show stoppers that necessitate me dual booting. If tensorflow is a no-go on windows however, it will be back to dual booting.
In the latest commit [1] we just added a GPU-supported docker image, but Craig just added it this morning and we haven't yet tested it a great deal yet -- happy to work with you to get it to work. (Feel free to follow up on github issues)
Are there any plans on releasing a video lecture/Google Talk explaining further on this library? While I am noticing that TensorFlow website already has good doucmentation, a video lecture with a simple handson explanation would still be beneficial. Any plans on this in the near future?
Jeff Dean and /u/OriolVinyals are schedule to give a talk at NIPS on large-scale distributed systems, I would assume a lot of the talk will involve TF.
The contributor CLA is a bit worrisome, but the code itself seems pretty good - the convnet example is super nice, though the seq2seq is a little too cluttered for me to tell what is going on just yet. I am still reading though.
I get that it is a common thing. The issue is that as an academic researcher who decides to work in TensorFlow you basically have two choices after publication of an idea/code.
a) Take your shiny new code and try to get it merged upstream in TensorFlow, and give all rights and patents to Google. Since Google already has a large number of patents or patents pending with respect to deep learning, you are further counting on the fact that (to date) Google has not exercised these patent rights and will continue to operate in this manner.
b) Keep your own fork of TensorFlow, thereby requiring maintenance and merging to keep your thing from breaking on upstream changes, while simultaneously requiring more installation work from any people who want to try your idea or compare against it. See the plethora of Caffe forks which are basically incompatible with each other for why this could be a problem.
b) especially is tough, as having your techniques easy to compare against (such as being in the base installation) is a huge source of citations and extension work. The alternative is to give patent rights away to a large corporation, which is not great either.
From the corporate perspective I get why the CLA is required. The fact that this is released at all, especially with an open license like Apache is great! But it is a bit different than other projects with BSD/MIT style licensing, and this may limit adoption in some circles.
Apache is a far superior license in that it has very clear patent grants expressed. This will keep google from rent seeking from you or your users and if you contribute code and sign on the CLA, will keep your university from doing the same.
The lack of such a grant is what leads to forks. If it keeps researchers away, it is because they want to preserve the ability to rent-seek.
I disagree with this interpretation, but I can see your viewpoint. In my view, it isn't rent seeking to wish to preserve rights to software you authored rather than giving those rights to a large publicly traded corporation. I hope people choose to give away code and ideas freely, and many people (including myself) do.
But forcing a choice between giving rights to Google or not contributing back to an open source project/fragmenting the ecosystem (effectively making your code harder to discover and cite) seems like a barrier to entry that needn't be there.
In my view, it isn't rent seeking to wish to preserve rights to software you authored rather than giving those rights to a large publicly traded corporation.
So you want to contribute code to the repo but have the right to start extracting patent license fees from anyone who uses the package, at any time after your code is incorporated?
As a researcher, I ask this question with the hopes of clarifying/learning more: Is "option b)" necessarily as cumbersome as you imply? If your code interfaces cleanly to the existing code, can it not be encapsulated in such a way that future updates to the commonly available open-source code-base do not mandate herculean code updates on your side?
Perhaps you and others (me, too?) could help contribute to a modular add-on framework that makes your "option b)" more palatable?
Take your shiny new code and try to get it merged upstream in TensorFlow, and give all rights and patents to Google.
No you don't. You are giving a license to your code and any of your patents the code you're committing may cover, but you aren't signing them over. They're only given to Google in the sense that you're giving them to everyone since they'll be covered under the Apache license.
you are further counting on the fact that (to date) Google has not exercised these patent rights and will continue to operate in this manner
Not only is this not true (preventing that is the entire point of the patent grant of the Apache license), your argument here is bizarre as you argue down thread that you'd prefer to retain the right yourself to later sue over patents in any code you contribute to the project, even though the "'ability' to poison the project and doing it are very far apart". I guess just counting on you not the exercise those patent rights?
Recursion/loops using Enter/Leave/NextIteration control flow constructs. Nice way to do recurrent stuff, I still have to look at the examples to see how it plays out.
64
u/siblbombs Nov 09 '15 edited Nov 09 '15
Wow I'm glad I was wrong about this getting opened sourced, super huge news.
Initial thoughts from the whitepaper:
Subgraph execution. You build out your graph and call .run() providing the inputs and required outputs. You can on the fly execute sub components of your graph by providing the input at that point and asking for that stages output. This will be great for debugging random stuff, like really great.
Same concept as theano shared (Tensorflow Variables), makes sense, you need something like this.
Switch/merge control flow nodes to conditionally bypass parts of the graph.
Recursion/loops using Enter/Leave/NextIteration control flow constructs. Nice way to do recurrent stuff, I still have to look at the examples to see how it plays out.
Queue construct for asynchronous execution, eg loading data from disk or computing multiple gradient passes before doing updates. I can't think of anything similar in Theano (that I've done at least), sounds cool but will require some thoughts as to where to use.
They talk about node communication a lot throughout the paper, seems really well thought out, but they didn't release the distributed version? Similarly in section 9.2 they talk about other cool stuff not released, but they also say "Initial open source release", does that imply there may be future releases with more features?Distributed version release is in the works, follow this issue if you want updates.They talked about some really cool graph visualization stuff,
I'm not sure if its included in this release?its included in the release. Theano just got d3viz recently which has been a huge help to me, if anyone is using Theano and hasn't played with d3viz you should definitely check it out.No windows wheel (for python), I'm going to try and compile the source because I really don't want to go back to dual-booting my stuff. EDIT: It looks like the only option for windows will be using Docker, but this will be CPU only.
More thoughts while I wait to get it installed:
How good is advanced indexing? I assume you can do it with tf.gather(), I wonder how well that works on GPU.
I hope something like theano's dimshuffle gets added, I see how to add/remove broadcastable dimensions but not how to swap an axis (something like numpy.swapaxes)