[Feature] Implement ScheduleFreeRAdam, LaProp optimizers and lots of things

[kozistr]

Problem (Why?)

N/A

Solution (What/How?)

• implement ScheduleFreeRAdam optimizer
• implement LaProp optimizer
• support cautious variant to AdamP, Adopt, and LaProp optimizers

Other changes (bug fixes, small refactors)

• fix the validation

Notes

N/A

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (a980dc0) to head (5326483).
Report is 15 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##              main      #304    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files           99       100     +1     
  Lines         7559      7727   +168     
==========================================
+ Hits          7559      7727   +168     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.