Benchmarking algorithms of various families

[GeoffNN]

It would be nice to have an example to compare speed of convergence of SAGA/SVRG/SFW on problems attaining the same optimum.

[GeoffNN]

This can be inspired by the trivial example https://github.com/openopt/chop/blob/master/examples/optim_dynamics.py

[GeoffNN]

It would also be really nice to benchmark algorithms on nonconvex problems. Do the usual practical acceleration methods (backtracking line search) work here?

[fabianp]

backtracking does work (in theory and practice) on non-convex problems

[GeoffNN]

I meant does it work for accelerating convergence in practice on non-convex problems?

[fabianp]

Yes

…

On Tue, Nov 24, 2020, 21:14 Geoffrey Negiar ***@***.***> wrote: I meant does it work for accelerating convergence in practice on non-convex problems? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#68 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACDZB4VGE3G7O74IZECN7DSRRSA3ANCNFSM4OCDSFLQ> .

[fabianp]

(as long as we're talking about deterministic algorithms, the top comment makes it confusing)

[yezhengli-Mr9]

It would also be really nice to benchmark algorithms on nonconvex problems. Do the usual practical acceleration methods (backtracking line search) work here?

Is there non-convex example? For example, non-convex function input. With constraints only as upper bounds, lower bounds on about 8 to 16 variables (about 2 to 4 of them can be integer but not necessary since these integers between 1~30 representing mins).

Thanks