ROADMAP

[Xylambda]

Prioritary

• Scalable testing framework. Use fixtures to pass shapes to testing functions
• Make sure iteration of parameters is okey. It is kinda tricky right now
• Tests for NN
• Tests for functional
  • Add more
  • Fix mae and softmax/min float errors when comparing against PyTorch
• CI/CD:
  • Testing and code checking
  • PyPI upload
  • Sphinx DOCS
• Fix kaiming initializer
• Storing and loading weights of a model
• Shouldn't have to track gradients for the features and labels to optimize model

Desirable

• Add CHANGELOG
• Data loading system for the model.
• Implementation of conv* operations
• Implementation of recurrent operations
• Implementation of attention mechanisms
• Implement more optimizers (Adam, RMSProp, etc)
• Utilities to load PyTorch weights into toydiff weights
• Name change (avagrad) and logo
• Add initializers
• Context managers to control whether gradients should be computed or not (torch.no_grad vs tf.GradientTape)

Can wait

• GPU support (PyOpenCL, autoray?)
• Sphinx docs
• Consider autoray to allow multiple-backend and lazy execution