Skip to content

Commit

Permalink
fix arguments
Browse files Browse the repository at this point in the history
  • Loading branch information
leloykun committed Dec 31, 2024
1 parent ff3279e commit 86c6ba9
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions optax/contrib/_muon.py
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ def muon(
] = (3.4445, -4.7750, 2.0315),
newton_schulz_steps: Optional[int] = 5,
mumentum: float = 0.95,
eps: float = 1e-8,
mu_dtype: Optional[Any] = None,
*,
nesterov: bool = True,
Expand All @@ -134,6 +135,7 @@ def muon(
newton_schulz_steps: Number of Newton-schulz iterations.
mumentum: Exponential decay rate to track the first moment of past
gradients.
eps: Term added to the denominator to improve numerical stability.
mu_dtype: Data type of the momentum accumulator.
nesterov: Whether to use Nesterov momentum.
Expand All @@ -145,6 +147,7 @@ def muon(
newton_schulz_coeffs,
newton_schulz_steps,
mumentum,
eps,
mu_dtype,
nesterov=nesterov,
),
Expand Down

0 comments on commit 86c6ba9

Please sign in to comment.