PuzzleLib Deep Learning Framework
What is PuzzleLib?
Though creativity cannot be taught, it can certainly be nurtured. Find a routine that works for you. Routines can be positive if they reinforce a healthy, creative mindset; they can be counterproductive if they actually keep you from being creative. While breaking your routine once in a while to force new ways of thinking is good, what if growing/learning/experiencing new things was built into your routine as a given? The people who speak negatively about routine have probably not developed a routine that puts them on a path of internal growth. The key is to discover creative rituals that put you in a more creative mindset
What is important in a deep learning framework?
- Algorithms: it must support the classic neural network algorithms that are executed as separate computing modules inside the framework.
- CPU and GPU support: it is more beneficial to train neural networks on GPU, yet CPU support may be essential when there is no opportunity to rely on the expensive GPU.
- Cross-platform nature: it is important to have the opportunity to launch neural networks from various operating systems and processor architecture designs, including the mobile devices.
- Speed: tasks geared toward big data neural network training may take up several days. Increasing the speed by 1.5-2 times is a significant improvement for the working process. What’s more, fast data processing is essential in tasks performed in real time, such as on-board video analysis.
PuzzleLib Strengths
- High speed: we employ exclusively low-level utility software for NVIDIA and AMD graphic cards and efficiently leverage on the processor resources.
- CPU and GPU support: cornerstone processors and graphic cards.
- The framework runs on all key operating systems: Linux family, Windows, MacOS, iOS, and Android.
- Automated computing optimization: in some cases it accelerates the speed of neural network training twice.
- Over 60 types of available neural network modules that allow to build convolutional and recurrent neural networks and networks with arbitrary graph structure. Below you can find the list of modules.
- Entry barrier for PuzzleLib users is significantly lower than in any other frameworks.
- PuzzleLib is a Russia’s domestically designed solution.
List of modules implemented in PuzzleLib
Any neural network is a computational graph where each element is a module in our framework that can execute operations with the received data. Below is the full list of modules contained by PuzzleLib. They allow you to design any possible cutting-edge neural architecture.
- Activation (Sigmoid, Tanh, ReLU, ELU, LeakyReLU, SoftMaxPlus)
- AvgPool (1D, 2D, 3D)
- BatchNorm (1D, 2D, 3D, ND)
- Conv (1D, 2D, 3D, ND)
- CrossMapLRN
- Deconv (1D, 2D, 3D, ND)
- Dropout (1D, 2D)
- GroupLinear
- InstanceNorm2D
- KMaxPool
- LCN
- Linear
- LRN
- MapLRN
- MaxPool (1D, 2D, 3D)
- MaxUnpool2D
- Pool (1D, 2D, 3D)
- RNN
- SoftMax
- SubtractNorm
- Upsample (2D, 3D)
- Embedder
- SpatialTf
- Add
- Concat
- DepthConcat
- Flatten
- Glue
- Identity
- MoveAxis
- Mul
- MulAddConst
- NoiseInjector
- Penalty
- Replicate
- Reshape
- Split
- Sum
- SwapAxes
- Tile
- ToList
- Transpose
List of optimizing algorithms executed in PuzzleLib
For neural network training it is essential to iterationally update its weights with an algorithm known as an “optimizer”. You need different optimizers for different tasks, for example, for regression or for text or image classification.
- AdaDelta
- AdaGrad
- Adam
- Hooks
- LBFGS
- MomentumSGD
- NesterovSGD
- RMSProp
- RMSPropGraves
- SGD
- SMORMS3