| 2023 | ICLR | Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation. | Bobby He, James Martens, Guodong Zhang, Aleksandar Botev, Andrew Brock, Samuel L. Smith, Yee Whye Teh |
| 2023 | ICLR | Pre-training via Denoising for Molecular Property Prediction. | Sheheryar Zaidi, Michael Schaarschmidt, James Martens, Hyunjik Kim, Yee Whye Teh, Alvaro Sanchez-Gonzalez, Peter W. Battaglia, Razvan Pascanu, Jonathan Godwin |
| 2022 | ICLR | Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers. | Guodong Zhang, Aleksandar Botev, James Martens |
| 2020 | TrustCom | Blockchain-based Verifiable Credential Sharing with Selective Disclosure. | Rahma Mukta, James Martens, Hye-Young Paik, Qinghua Lu, Salil S. Kanhere |
| 2018 | ICLR | Kronecker-factored Curvature Approximations for Recurrent Neural Networks. | James Martens, Jimmy Ba, Matt Johnson |
| 2018 | ICLR | Stochastic Gradient Langevin dynamics that Exploit Neural Network Structure. | Zachary Nado, Jasper Snoek, Roger B. Grosse, David Duvenaud, Bowen Xu, James Martens |
| 2018 | ICML | The Mechanics of n-Player Differentiable Games. | David Balduzzi, Sbastien Racanire, James Martens, Jakob N. Foerster, Karl Tuyls, Thore Graepel |
| 2017 | ICLR | Distributed Second-Order Optimization using Kronecker-Factored Approximations. | Jimmy Ba, Roger B. Grosse, James Martens |
| 2016 | ICML | A Kronecker-factored approximate Fisher matrix for convolution layers. | Roger B. Grosse, James Martens |
| 2015 | ICML | Optimizing Neural Networks with Kronecker-factored Approximate Curvature. | James Martens, Roger B. Grosse |
| 2013 | ICML | On the importance of initialization and momentum in deep learning. | Ilya Sutskever, James Martens, George E. Dahl, Geoffrey E. Hinton |
| 2012 | ICML | Estimating the Hessian by Back-propagating Curvature. | James Martens, Ilya Sutskever, Kevin Swersky |
| 2011 | ICML | Learning Recurrent Neural Networks with Hessian-Free Optimization. | James Martens, Ilya Sutskever |
| 2011 | ICML | Generating Text with Recurrent Neural Networks. | Ilya Sutskever, James Martens, Geoffrey E. Hinton |
| 2010 | ICML | Deep learning via Hessian-free optimization. | James Martens |
| 2010 | ICML | Learning the Linear Dynamical System with ASOS. | James Martens |