Skip to content

Repository files navigation

pytorch-jax-implementation

This repository serves as a comprehensive collection of deep learning components, architectures, and training techniques implemented using both PyTorch and JAX. The primary goal is to provide a clear, side-by-side comparison and functional implementation of fundamental machine learning concepts to help bridge the gap between these two powerful frameworks.

Table of Contents

  1. Activation Functions: An exploration of common non-linear activation layers and their mathematical foundations.
  2. Optimizations: Implementations of various gradient-based optimization algorithms and update rules.
  3. Initializations: A deep dive into weight initialization strategies designed to maintain signal variance and accelerate convergence.
  4. Vision Transformer: An implementation of the Vision Transformer (ViT) architecture, detailing the patch embedding process and global attention mechanisms.
  5. Transformer: Core components of the Transformer architecture, including multi-head self-attention, positional encoding, and feed-forward networks.
  6. Contrastive Learning: Frameworks for self-supervised learning, specifically focusing on SimCLR and objective functions that maximize representation agreement.
  7. Convolutional Architectures: Implementations of classic and modern CNN structures, covering feature extraction, downsampling, and architectural building blocks.

Releases

Packages

Contributors

Languages