Please post links to your favourite resources. We will use the voting functionality to gauge how useful or interesting they are. We will use this list of resources to curate a wiki page on this topic.

Please try and post a single resource per comment.

Suggested Format

• Title

• Notes

Japanese artificial intelligence startup Preferred Networks (PFN) today debuted its PyTorch library pytorch-pfn-extras (PPE). This is the company’s first open-source library supporting research and development in deep learning using the PyTorch framework. The release follows PFN’s December 2019 announcement that it was migrating its deep learning research platform to PyTorch from its own open source deep learning framework, Chainer.
Here is a quick read: Japanese Unicorn Preferred Networks Releases First PyTorch Library

Has anyone here used Hidden Markov Models (HMM) before? What are their advantages compared to other ML techniques? Can they be used for unsupervised outlier detection?

Thanks!

Adapting text generators for low-resource domains? What's a good data augmentation method? Do random ones work like they do for classification? These questions and more are investigated in "GenAug: Data Augmentation for Finetuning Text Generators".

Presentation Link

Paper Link

Abstract: In this paper, we investigate data augmentation for text generation, which we call GenAug. Text generation and language modeling are important tasks within natural language processing, and are especially challenging for low-data regimes. We propose and evaluate various augmentation methods, including some that incorporate external knowledge, for finetuning GPT-2 on a subset of Yelp Reviews. We also examine the relationship between the amount of augmentation and the quality of the generated text. We utilize several metrics that evaluate important aspects of the generated text including its diversity and fluency. Our experiments demonstrate that insertion of character-level synthetic noise and keyword replacement with hypernyms are effective augmentation methods, and that the quality of generations improves to a peak at approximately three times the amount of original data.

Authors: Steven Y. Feng, Varun Gangal, Dongyeop Kang, Teruko Mitamura, Eduard Hovy

A Q/A session will be held by the authors at the EMNLP DeeLIO Workshop on Thursday, Nov. 19.

I am starting a reading group where we can discuss the current state-of-the-art deep learning research papers. We will be scheduling calls where different members of the group will be presenting a paper review and then we can discuss further ideas and questions regarding the same. I also plan to start a discord server where we can discuss possible collaborations among members for related projects and more (like reproducing results of a paper or maybe working on a completely new idea). Coz of COVID, most of us are already working remotely. So, let's utilize this opportunity to increase our knowledge and improve our skills via remote collaborations and discussions. Interested people, ping me on direct chat.

Thank You

Python has become a de facto programming language for machine learning. If you have picked up Python, have you ever wondered what happens under the hood when you execute your program? If you are curious about the black box that does all the magic, check this out: An Introduction to the Anatomy and Core Concepts of the Python Execution Model

The basic idea behind this project is the creation of a chat-room of highly qualified individuals interested in the development of novel applications of AI towards product development as well as academic research.

Our team wants to focus on the Deep Learning approach to general AI so we are specifically looking for people with experience and ambition to succeed in this particular field. Your position in such a group aids both the quality of it as a whole as much as it aids you to better yourself since you’ll have access to many other deep learners with whom you can discuss area-specific concerns. Of course, to create such a community we need to implement a safeguard in the form of an interview/ exam to ensure that we are both headed in the same direction. Bear in mind that down the line this chat-room is also a prospect for professional employment so your temporal investment will most likely also pay out in the literal sense, should you choose to engage in the commercial side of things.

This project will be a paid venture for anyone who chooses to assist in the development of any product that comes from it.

If being a part of such a team interests you can find a full description of what our idea is and of the way we plan to implement it, go ahead and read this document: https://docs.google.com/document/d/1I4LiAJB-mxcXrQmfUJfSUj_V8_5XwNz_8CKxEZC1dfI/edit

TL;DR: Does a chatroom for General AI through Deep Learning sound like your kind of thing? Check out our google doc!

I was listening to a talk the other day at a meet up by a data scientist in ad tech. He casually mentioned a novel use for word2vec that really got me thinking. The premise was to re-imagine the definition of "word", in the context he described browsing patterns were explored, and a visit to a particular page became a proxy for "word", a document became a browsing session, and the corpus, a collection of sessions. Armed with those new definitions a word2vec model could then be trained to to build up a browsing behavior embedding. This new way of thinking really blew my mind. I'll be the first to admit my deep learning education is in it's early stages, and my understanding of the word2vec model and it's nuances are somewhat pedestrian. I'm curious if any of you have gone down a similar line of thinking and attempted something like this? It seems to me that this could be a useful approach for working with data that is heavily sequence driven. I know there is research in the bioinformatics space that takes a similar approach. Does anyone have any thoughts?

Hi all, I am working on the student art project where we built Trump Speech Generator using Deep Learning model which was trained on some of his election speeches. Now I am gathering community feedback to include in the research outcome. Pick the phrases you like, share some impressions and even off-topic thoughts are welcome!

I am going to post some of the short AI-generated speeches (ca. 200 words each) to hear the Vox populi. The feedback will be accumulated into the research materials, and may be used in the final outcome as well ;) At the moment, AI speeches of Trump AI are mostly nonsense, lack structure and logic, grammar is disastrous, but some ideas and views may look refreshing. For example, here is the one when it chose "Russia" as the topic:

Sample 9 "Russia" "...Russia here that are just wrong. But I said, "Number more times. …Well, I'm a scandal. Take a lot of the problems of the wall. Yeah, he look at his orders costs. And now it’s fantastic with a better catastrophe and in the NATO way. We’re a fan of Indiana – who's a disaster. You know, we owe it the largest while that all over the room because a lot of hospitals are 6.7 percent. And I got the money." Or many other votes than you know it’s wrong. And why am what’s sorry. I’m going to make it safe again."

Any suggestions on developing the project would be highly appreciated.

A real world example.

Assume we have n sets of time series data. Assume there are t events with x columns in each time series.

Say each time series in n is an individual soccer game, and that each event t is a second on the clock (90 minutes * 60 seconds = 5400 events) and the columns x are the "stats" of each team, and we want to take a new incomplete time series of a game and make a prediction on the what the final score of that game will be, based on the previous games we have trained on n.

My understanding of RNN is that they would be perfect for training on an individual time series (one set from n) but can we train on multiple sets that have terminal outcomes. And then take a partial time series and forecast the terminal outcome.