SlowMo
public
–
2 min read
SlowMo: Distributed Optimization for Faster Learning
SlowMo, short for Slow Momentum, is a distributed optimization method designed to help machines…
Distributed Any-Batch Mirror Descent
public
–
2 min read
DABMD: An Overview of Distributed Any-Batch Mirror Descent
If you've ever waited for slow internet to load a webpage, you…
Adversarial Model Perturbation
public
–
2 min read
What is AMP?
AMP stands for Adversarial Model Perturbation, which is a technique used to improve the generalization of machine…
Rung Kutta optimization
public
–
2 min read
RUNge Kutta Optimizer (RUN) – A Novel Metaphor-Free Population-Based Optimization Method
The optimization field is constantly evolving, with researchers developing new…
Hybrid Firefly and Particle Swarm Optimization
public
–
3 min read
Hybrid Firefly and Particle Swarm Optimization (HFPSO) is a powerful optimization algorithm that combines the best features of firefly and…
Neural adjoint method
public
–
2 min read
Neural Adjoint: An Overview
Neural adjoint is a method used for inverse modeling, which involves finding the inputs to a…
ADAHESSIAN
public
–
2 min read
AdaHessian: A Revolutionary Optimization Method in Machine Learning
AdaHessian is a cutting-edge optimization method that has recently gained widespread attention…
Natural Gradient Descent
public
–
2 min read
Natural Gradient Descent: An Overview
Have you ever heard of optimization methods? Optimization methods are techniques used in machine learning…
Harris Hawks optimization
public
–
2 min read
The Basics of Harris Hawks Optimization (HHO)
Harris Hawks Optimization (HHO) is a type of optimization algorithm inspired by the…
Grammatical evolution and Q-learning
public
–
3 min read
Grammatical evolution and Q-learning are two powerful techniques in the field of artificial intelligence. Grammatical evolution is a method used…
Sharpness-Aware Minimization
public
–
2 min read
Sharpness-Aware Minimization (SAM) is a powerful technique in the field of artificial intelligence and machine learning that helps to improve…
Gradient-based optimization
public
–
2 min read
The GBO Algorithm: A Novel Metaheuristic Optimization Algorithm
The Gradient-based Optimizer (GBO) is an optimization algorithm inspired by the Newton’…
Hunger Games Search
public
–
2 min read
Overview of Hunger Games Search (HGS)
Hunger Games Search (HGS) is a new optimization technique that aims to find solutions…
Slime Mould Algorithm
public
–
2 min read
The Slime Mould Algorithm, commonly referred to as SMA, is a new and innovative stochastic optimizer with a unique mathematical…
PowerSGD
public
–
2 min read
Overview of PowerSGD: A Distributed Optimization Technique
If you're someone who is interested in the field of machine learning, you…
Population Based Training
public
–
2 min read
Overview of Population Based Training (PBT)
In the field of artificial intelligence and machine learning, Population Based Training (PBT) is…
Two Time-scale Update Rule
public
–
2 min read
The Two-Time Scale Update Rule (TTUR) in Generative Adversarial Networks
Generative Adversarial Networks (GANs) are powerful model architectures that have…
Gradient Clipping
public
–
3 min read
Gradient clipping is a technique used in deep learning to help optimize the performance of neural networks. The problem that…
Local SGD
public
–
2 min read
Local SGD is an advanced technique used in machine learning that helps to speed up the training process by running…
Gradient Sparsification
public
–
3 min read
Overview of Gradient Sparsification
Gradient Sparsification is a technique used in distributed machine learning to reduce the communication cost between…
SAGA
public
–
2 min read
SAGA: A Fast Incremental Gradient Algorithm
If you're looking for a way to train large-scale machine learning models quickly, SAGA…
Alternating Direction Method of Multipliers
public
–
2 min read
The alternating direction method of multipliers (ADMM) is an algorithm that can solve complex optimization problems. It does this by…
Adam
public
–
2 min read
Adam is an adaptive learning rate optimization algorithm that combines the benefits of RMSProp and SGD with Momentum. It is…