(转) Awesome
转自：https://github.com/terryum/awesome-deep-learning-papersAwesome - Most Cited Deep Learning Papers
https://camo.githubusercontent.com/13c4e50d88df7178ae1882a203ed57b641674f94/68747470733a2f2f63646e2e7261776769742e636f6d2f73696e647265736f726875732f617765736f6d652f643733303566333864323966656437386661383536353265336136336531353464643865383832392f6d656469612f62616467652e737667
A curated list of the most cited deep learning papers (since 2010)
I believe that there exist classic deep learning papers which are worth reading regardless of their application areas. Rather than providing overwhelming amount of papers, I would like to provide a curated list of the classic deep learning papers which can be considered as must-reads in some research areas.
Awesome list criteria
[*]< 6 months : Please refer to New papers worth reading section
[*]< 1 year : +30 citations
[*]2016 : +50 citations (✨ +80)
[*]2015 : +100 citations (✨ +200)
[*]2014 : +200 citations (✨ +400)
[*]2013 : +300 citations (✨ +600)
[*]2012 : +400 citations (✨ +800)
[*]Before 2012 : Please refer to Classic papers section
I need your contributions! Please read the contributing guide before you make a pull request.
Table of Contents
[*]Book / Survey / Review
[*]Theory / Distillation
[*]Optimization / Regularization
[*]Network Models
[*]Image
[*]Caption
[*]Video / Human Activity
[*]Word Embedding
[*]Machine Translation / QnA
[*]Speech / Etc.
[*]RL / Robotics
[*]Unsupervised
[*]Hardware / Software
[*]New Papers Worth Reading
[*]Classic Papers
[*]Distinguished Researchers
Total 85 papers except for the papers in Hardware / Software, Papers Worth Reading, and Classic Papers sections.
Survey / Review
[*]Deep learning (Book, 2016), Goodfellow et al. (Bengio)
[*]Deep learning (2015), Y. LeCun, Y. Bengio and G. Hinton ✨
[*]Deep learning in neural networks: An overview (2015), J. Schmidhuber ✨
[*]Representation learning: A review and new perspectives (2013), Y. Bengio et al. ✨
Theory / Distillation
[*]Distilling the knowledge in a neural network (2015), G. Hinton et al. (Hinton, Vinyals, Dean: Google) ✨
[*]Deep neural networks are easily fooled: High confidence predictions for unrecognizable images (2015), A. Nguyen et al.
[*]How transferable are features in deep neural networks? (2014), J. Yosinski et al. (Bengio)
[*]Return of the devil in the details: delving deep into convolutional nets (2014), K. Chatfield et al. ✨
[*]Why does unsupervised pre-training help deep learning (2010), D. Erhan et al. (Bengio)
[*]Understanding the difficulty of training deep feedforward neural networks (2010), X. Glorot and Y. Bengio
Optimization / Regularization
[*]Batch normalization: Accelerating deep network training by reducing internal covariate shift (2015), S. Loffe and C. Szegedy (Google) ✨
[*]Delving deep into rectifiers: Surpassing human-level performance on imagenet classification (2015), K. He et al. (He) ✨
[*]Recurrent neural network regularization (2014), W. Zaremba et al. (Sutskever, Vinyals: Google)
[*]Dropout: A simple way to prevent neural networks from overfitting (2014), N. Srivastava et al. (Hinton) ✨
[*]Adam: A method for stochastic optimization (2014), D. Kingma and J. Ba
[*]Spatial pyramid pooling in deep convolutional networks for visual recognition (2014), K. He et al. ✨
[*]On the importance of initialization and momentum in deep learning (2013), I. Sutskever et al. (Hinton)
[*]Regularization of neural networks using dropconnect (2013), L. Wan et al. (LeCun)
[*]Improving neural networks by preventing co-adaptation of feature detectors (2012), G. Hinton et al. ✨
[*]Random search for hyper-parameter optimization (2012) J. Bergstra and Y. Bengio
Network Models
[*]Inception-v4, inception-resnet and the impact of residual connections on learning (2016), C. Szegedy et al. (Google)
[*]Identity Mappings in Deep Residual Networks (2016), K. He et al. (He)
[*]Deep residual learning for image recognition (2016), K. He et al. (He) ✨
[*]Region-based convolutional networks for accurate object detection and segmentation (2016), R. Girshick et al. (He)
[*]Going deeper with convolutions (2015), C. Szegedy et al. (Google) ✨
[*]Fast R-CNN (2015), R. Girshick (He) ✨
[*]An Empirical Exploration of Recurrent Network Architectures (2015), R. Jozefowicz et al. Sutskever: Google
[*]Fully convolutional networks for semantic segmentation (2015), J. Long et al. ✨
[*]Very deep convolutional networks for large-scale image recognition (2014), K. Simonyan and A. Zisserman ✨
[*]OverFeat: Integrated recognition, localization and detection using convolutional networks (2014), P. Sermanet et al. (LeCun)
[*]Visualizing and understanding convolutional networks (2014), M. Zeiler and R. Fergus ✨
[*]Maxout networks (2013), I. Goodfellow et al. (Bengio)
[*]Network in network (2013), M. Lin et al.
[*]ImageNet classification with deep convolutional neural networks (2012), A. Krizhevsky et al. (Hinton) ✨
[*]Large scale distributed deep networks (2012), J. Dean et al. ✨
[*]Deep sparse rectifier neural networks (2011), X. Glorot et al. (Bengio)
Unsupervised / Adversarial
[*]Unsupervised representation learning with deep convolutional generative adversarial networks (2015), A. Radford et al.
[*]CNN features off-the-Shelf: An astounding baseline for recognition (2014), A. Razavian et al. ✨
[*]Generative adversarial nets (2014), I. Goodfellow et al. (Bengio)
[*]Intriguing properties of neural networks (2014), C. Szegedy et al. (Sutskever, Goodfellow: Google)
[*]Auto-encoding variational Bayes (2013), D. Kingma and M. Welling
[*]Building high-level features using large scale unsupervised learning (2013), Q. Le et al. ✨
[*]An analysis of single-layer networks in unsupervised feature learning (2011), A. Coates et al.
[*]Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion (2010), P. Vincent et al. (Bengio)
[*]A practical guide to training restricted boltzmann machines (2010), G. Hinton
Image
[*]Image Super-Resolution Using Deep Convolutional Networks (2016), C. Dong et al. (He) ✨
[*]Reading text in the wild with convolutional neural networks (2016), M. Jaderberg et al. (DeepMind)
[*]Learning Deconvolution Network for Semantic Segmentation (2015), H. Noh et al.
[*]Imagenet large scale visual recognition challenge (2015), O. Russakovsky et al. ✨
[*]Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015), S. Ren et al. ✨
[*]DRAW: A recurrent neural network for image generation (2015), K. Gregor et al.
[*]Scalable object detection using deep neural networks (2014), D. Erhan et al. (Google)
[*]Learning a Deep Convolutional Network for Image Super-Resolution (2014, C. Dong et al. (He)
[*]Rich feature hierarchies for accurate object detection and semantic segmentation (2014), R. Girshick et al. ✨
[*]Learning a Deep Convolutional Network for Image Super-Resolution (2014), C. Dong et al.
[*]Learning and transferring mid-Level image representations using convolutional neural networks (2014), M. Oquab et al.
[*]DeepFace: Closing the Gap to Human-Level Performance in Face Verification (2014), Y. Taigman et al. (Facebook) ✨
[*]Decaf: A deep convolutional activation feature for generic visual recognition (2013), J. Donahue et al. ✨
[*]Learning hierarchical features for scene labeling (2013), C. Farabet et al. (LeCun)
[*]Learning mid-level features for recognition (2010), Y. Boureau (LeCun)
Caption / Visual QnA
[*]VQA: Visual question answering (2015), S. Antol et al.
[*]Towards ai-complete question answering: A set of prerequisite toy tasks (2015), J. Weston et al. (Mikolov: Facebook)
[*]Ask me anything: Dynamic memory networks for natural language processing (2015), A. Kumar et al.
[*]A large annotated corpus for learning natural language inference (2015), S. Bowman et al.
[*]Show, attend and tell: Neural image caption generation with visual attention (2015), K. Xu et al. (Bengio) ✨
[*]Show and tell: A neural image caption generator (2015), O. Vinyals et al. (Vinyals: Google) ✨
[*]Long-term recurrent convolutional networks for visual recognition and description (2015), J. Donahue et al. ✨
[*]Deep visual-semantic alignments for generating image descriptions (2015), A. Karpathy and L. Fei-Fei ✨
Video / Human Activity
[*]Beyond short snippents: Deep networks for video classification (2015) (Vinyals: Google) ✨
[*]Large-scale video classification with convolutional neural networks (2014), A. Karpathy et al. (FeiFei) ✨
[*]DeepPose: Human pose estimation via deep neural networks (2014), A. Toshev and C. Szegedy (Google)
[*]Two-stream convolutional networks for action recognition in videos (2014), K. Simonyan et al.
[*]A survey on human activity recognition using wearable sensors (2013), O. Lara and M. Labrador
[*]3D convolutional neural networks for human action recognition (2013), S. Ji et al.
[*]Action recognition with improved trajectories (2013), H. Wang and C. Schmid
[*]Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis (2011), Q. Le et al.
Word Embedding
[*]Glove: Global vectors for word representation (2014), J. Pennington et al. ✨
[*]Distributed representations of sentences and documents (2014), Q. Le and T. Mikolov (Le, Mikolov: Google) (Google)✨
[*]Distributed representations of words and phrases and their compositionality (2013), T. Mikolov et al. (Google) ✨
[*]Efficient estimation of word representations in vector space (2013), T. Mikolov et al. (Google) ✨
[*]Devise: A deep visual-semantic embedding model (2013), A. Frome et al., (Mikolov: Google)
[*]Word representations: a simple and general method for semi-supervised learning (2010), J. Turian (Bengio)
Machine Translation / QnA
[*]Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation (2016), Y. Wu et al. (Le, Vinyals, Dean: Google)
[*]Exploring the limits of language modeling (2016), R. Jozefowicz et al. (Vinyals: DeepMind)
[*]A neural conversational model, O. Vinyals and Q. Le. (Vinyals, Le: Google)
[*]Grammar as a foreign language (2015), O. Vinyals et al. (Vinyals, Sutskever, Hinton: Google)
[*]Towards ai-complete question answering: A set of prerequisite toy tasks (2015), J. Weston et al.
[*]Neural machine translation by jointly learning to align and translate (2014), D. Bahdanau et al. (Bengio) ✨
[*]Sequence to sequence learning with neural networks (2014), I. Sutskever et al. (Sutskever, Vinyals, Le: Google) ✨
[*]Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014), K. Cho et al. (Bengio)
[*]A convolutional neural network for modelling sentences (2014), N. Kalchbrenner et al.
[*]Convolutional neural networks for sentence classification (2014), Y. Kim
[*]The stanford coreNLP natural language processing toolkit (2014), C. Manning et al. ✨
[*]Recursive deep models for semantic compositionality over a sentiment treebank (2013), R. Socher et al. ✨
[*]Linguistic Regularities in Continuous Space Word Representations (2013), T. Mikolov et al. (Mikolov: Microsoft)
[*]Natural language processing (almost) from scratch (2011), R. Collobert et al. ✨
[*]Recurrent neural network based language model (2010), T. Mikolov et al.
Speech / Etc.
[*]Automatic speech recognition - A deep learning approach (Book, 2015), D. Yu and L. Deng (Microsoft)
[*]Speech recognition with deep recurrent neural networks (2013), A. Graves (Hinton)
[*]Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups (2012), G. Hinton et al. ✨
[*]Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition (2012) G. Dahl et al. ✨
[*]Acoustic modeling using deep belief networks (2012), A. Mohamed et al. (Hinton)
RL / Robotics
[*]Mastering the game of Go with deep neural networks and tree search (2016), D. Silver et al. (Sutskever: DeepMind) ✨
[*]Human-level control through deep reinforcement learning (2015), V. Mnih et al. (DeepMind) ✨
[*]Deep learning for detecting robotic grasps (2015), I. Lenz et al.
[*]Playing atari with deep reinforcement learning (2013), V. Mnih et al. (DeepMind) )
Hardware / Software
[*]TensorFlow: Large-scale machine learning on heterogeneous distributed systems (2016), M. Abadi et al. (Google) ✨
[*]Theano: A Python framework for fast computation of mathematical expressions, R. Al-Rfou et al. (Bengio)
[*]MatConvNet: Convolutional neural networks for matlab (2015), A. Vedaldi and K. Lenc
[*]Caffe: Convolutional architecture for fast feature embedding (2014), Y. Jia et al. ✨
Papers Worth Reading
Newly released papers which do not meet the criteria but worth reading
[*]WaveNet: A Generative Model for Raw Audio (2016), A. Oord et al. (DeepMind)
[*]Layer Normalization (2016), J. Ba et al. (Hinton)
[*]Deep neural network architectures for deep reinforcement learning, Z. Wang et al. (DeepMind)
[*]Learning to learn by gradient descent by gradient descent (2016), M. Andrychowicz et al. (DeepMind)
[*]Adversarially learned inference (2016), V. Dumoulin et al.
[*]Understanding convolutional neural networks (2016), J. Koushik
[*]SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 1MB model size (2016), F. Iandola et al.
[*]Learning to compose neural networks for question answering (2016), J. Andreas et al.
[*]Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection (2016) (Google), S. Levine et al.
[*]Taking the human out of the loop: A review of bayesian optimization (2016), B. Shahriari et al.
[*]Eie: Efficient inference engine on compressed deep neural network (2016), S. Han et al.
[*]Adaptive Computation Time for Recurrent Neural Networks (2016), A. Graves
[*]Pixel recurrent neural networks (2016), A. van den Oord et al. (DeepMind)
[*]Densely connected convolutional networks (2016), G. Huang et al.
Classic Papers
Classic papers (1997~2011) which cause the advent of deep learning era
[*]Recurrent neural network based language model (2010), T. Mikolov et al.
[*]Learning deep architectures for AI (2009), Y. Bengio.
[*]Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations (2009), H. Lee et al.
[*] Greedy layer-wise training of deep networks (2007), Y. Bengio et al.
[*] Reducing the dimensionality of data with neural networks, G. Hinton and R. Salakhutdinov.
[*]A fast learning algorithm for deep belief nets (2006), G. Hinton et al.
[*]Gradient-based learning applied to document recognition (1998), Y. LeCun et al.
[*]Long short-term memory (1997), S. Hochreiter and J. Schmidhuber.
Distinguished Researchers
Distinguished deep learning researchers who have published +3 (✨ +6) papers on the awesome list (The papers in Hardware / Software, Papers Worth Reading, Classic Papers sections are excluded in counting.)
[*]Chirstian Szegedy, Google ✨
[*]Kaiming He, Facebook ✨
[*]Geoffrey Hinton, Google ✨
[*]Ilya Sutskever, OpenAI ✨
[*]Ian Goodfellow, OpenAI ✨
[*]Oriol Vinyals, Google DeepMind ✨
[*]Quoc Le, Google ✨
[*]Tomas Mikolov, Facebook
[*]Yann LeCun, Facebook ✨
[*] Yoshua Bengio, University of Montreal ✨
[*] Aaron Courville, University of Montreal
[*]Alex Graves, Google DeepMind
[*]Andrej Karpathy, OpenAI
[*]Andrew Ng, Baidu
[*] Andrew Zisserman, University of Oxford
[*] Christopher Manning, Stanford University
[*]David Silver, Google DeepMind
[*]Dong Yu, Microsoft Research
[*] Ross Girshick, Facebook
[*] Karen Simonyan, Google DeepMind
[*]Kyunghyun Cho, New York University
[*] Honglak Lee, University of Michigan
[*] Jeff Dean, Google,
[*]Jeff Donahue, U.C. Berkeley
[*]Jian Sun, Microsoft Research
[*]Juergen Schmidhuber, Swiss AI Lab IDSIA
[*] Li Fei-Fei, Stanford University
[*] Pascal Vincent, University of Montreal
[*]Rob Fergus, Facebook, New York University
[*]Ruslan Salakhutdinov, CMU
[*]Trevor Darrell, U.C. Berkeley
Acknowledgement
Thank you for all your contributions. Please make sure to read the contributing guide before you make a pull request.
You can follow my facebook page or google plus to get useful information about machine learning and robotics. If you want to have a talk with me, please send me a message to my facebook page.
You can also check out my blog where I share my thoughts on my research area (deep learning for human/robot motions). I got some thoughts while making this list and summerized them in a blog post, "Some trends of recent deep learning researches".
License
https://camo.githubusercontent.com/60561947585c982aee67ed3e3b25388184cc0aa3/687474703a2f2f6d6972726f72732e6372656174697665636f6d6d6f6e732e6f72672f70726573736b69742f627574746f6e732f38387833312f7376672f63632d7a65726f2e737667
To the extent possible under law, Terry T. Um has waived all copyright and related or neighboring rights to this work.
[*]Contact GitHub
[*]API
[*]Training
[*]Shop
[*]Blog
[*]About
[*]© 2017 GitHub, Inc.
[*]Terms
[*]Privacy
[*]Security
[*]Status
[*]Help
页:
[1]