On the Variance of the Adaptive Learning Rate and Beyond
-
Updated
Jul 31, 2021 - Python
#
adam-optimizer
Here are 476 public repositories matching this topic...
Deep learning library in plain Numpy.
-
Updated
Jun 21, 2022 - Python
This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"
-
Updated
Jul 17, 2021
A book on the mathematical foundations of AI from an engineering perspective.
machine-learning books ai deep-learning calculus neural-network knowledge book optimization matrix linear-regression linear-algebra mathematics ml artificial-intelligence convergence educational derivatives integrals adam-optimizer
-
Updated
Jun 2, 2026 - Python
CS F425 Deep Learning course at BITS Pilani (Goa Campus)
machine-learning deep-neural-networks deep-learning generative-model optimisation bits-pilani adam-optimizer
-
Updated
Feb 5, 2025 - Jupyter Notebook
ADAS is short for Adaptive Step Size, it's an optimizer that unlike other optimizers that just normalize the derivative, it fine-tunes the step size, truly making step size scheduling obsolete, achieving state-of-the-art training performance
-
Updated
Jan 19, 2021 - C++
🌹 Rose: Range-Of-Slice Equilibration PyTorch optimizer. Stateless optimization through range-normalized gradient updates.
machine-learning ai deep-learning transformers optimizer pytorch artificial-intelligence sgd gradient-descent gpt lora adam-optimizer fine-tuning diffusion-models pretraining optimizers optimizer-algorithms large-language-models llm
-
Updated
Apr 28, 2026 - Python
Lion and Adam optimization comparison
-
Updated
Feb 23, 2023 - Jupyter Notebook
Google Street View House Number(SVHN) Dataset, and classifying them through CNN
machine-learning computer-vision cnn dropout mnist svhn convolutional-neural-networks confusion-matrix google-street convolutional-neural-network tensorboard-visualizations tesnorflow svhn-classifier grayscale-images house-numbers adam-optimizer mini-batch-gradient-descent one-hot-encode
-
Updated
Mar 4, 2018 - Jupyter Notebook
Reproducing the paper "PADAM: Closing The Generalization Gap of Adaptive Gradient Methods In Training Deep Neural Networks" for the ICLR 2019 Reproducibility Challenge
tensorflow optimization keras wide-residual-networks adam-optimizer tensorflow-eager amsgrad sgd-momentum padam
-
Updated
Apr 13, 2019 - Python
PyTorch/Tensorflow solutions for Stanford's CS231n: "CNNs for Visual Recognition"
machine-learning computer-vision deep-learning neural-network solutions tensorflow numpy jupyter-notebook stanford-university assignment pytorch recurrent-neural-networks lstm generative-adversarial-network style-transfer convolutional-neural-networks backpropagation svm-classifier adam-optimizer sgd-optimizer
-
Updated
Jan 27, 2021 - Jupyter Notebook
A collection of various gradient descent algorithms implemented in Python from scratch
machine-learning ai deep-learning machine-learning-algorithms ml deep-learning-algorithms gradient-descent adagrad rmsprop gradient-descent-algorithm adam-optimizer nesterov-accelerated-sgd nesterov-momentum nesterov momentum-optimization-algorithm
-
Updated
Feb 28, 2023 - Python
Toy implementations of some popular ML optimizers using Python/JAX
machine-learning momentum gradient-descent optimization-algorithms adam adam-optimizer jax optimizers
-
Updated
Jun 20, 2021 - Python
🌟 [Survey & Benchmark] A unified taxonomy, geometric framework, and cross-domain benchmark for modern optimizers
benchmark deep-learning taxonomy transformers optimizer survey muon adam-optimizer shampoo large-language-models
-
Updated
Jul 7, 2026 - Python
A family of highly efficient, lightweight yet powerful optimizers.
compression optimizer pytorch memory-efficient adam-optimizer fine-tuning low-rank-approximation llm
-
Updated
Aug 4, 2026 - Python
This library provides a set of functionalities for different type of deep learning (and ML) algorithms in C
c library deep-learning neural-network opencl transformer lstm convolutional-layers deeplearning residual-layers residual-networks backpropagation optimization-algorithms softmax adam-optimizer avarage nesterov adam-algorithm dropout-layers
-
Updated
Sep 29, 2023 - C
From linear regression towards neural networks...
data-science machine-learning neural-network linear-regression machine-learning-algorithms data-analysis logistic-regression gradient-descent backpropagation adagrad rmsprop generalized-linear-models multilayer-perceptron-network adam-optimizer nesterov-accelerated-sgd nesterov-momentum nadam
-
Updated
Dec 31, 2025 - C++
Modified XGBoost implementation from scratch with Numpy using Adam and RSMProp optimizers.
numpy gradient-boosting-machine rmsprop adam-optimizer xgboost-algorithm implementation-of-algorithms
-
Updated
Jul 24, 2020 - Jupyter Notebook
Lookahead optimizer ("Lookahead Optimizer: k steps forward, 1 step back") for tensorflow
-
Updated
Sep 3, 2019 - Python
Add this topic to your repo
To associate your repository with the adam-optimizer topic, visit your repo's landing page and select "manage topics."