The Microsoft Cognitive Toolkit
The Microsoft Cognitive Toolkit (previously known as CNTK) is now available optionally with Exxact Deep Learning Solutions! All Exxact Deep Learning systems come preinstalled with standard Ubuntu in addition to Caffe, Torch, Theano, BIDMach, OpenCV, CUDA Toolkit 8.0RC and PaddlePaddle.
What is the Microsoft Cognitive Toolkit?
The Microsoft Cognitive Toolkit is an open-source toolkit for commercial-grade distributed deep learning. It describes neural networks as a series of computational steps via a directed graph. This toolkit allows the user to easily realize and combine popular model types such as feed-forward DNNs, convolutional neural networks (CNNs) and recurrent neural networks (RNNs/LSTMs). The Microsoft Cognitive Toolkit implements stochastic gradient descent (SGD, error backpropagation) learning with automatic differentiation and parallelization across multiple GPUs and servers. Below you’ll find a list of some of the many features and capabilities offered with the Microsoft Cognitive Toolkit:
Highly Optimized, Built-In Components
- Components can handle multi-dimensional dense or spare data from Python, C++ or BrainScript
- FNN, CNN, RNN/LSTM, Batch normalization, Sequence-to-Sequence with attention and more
- Reinforcement learning, generative adversarial networks, supervised and unsupervised learning
- Ability to add new user-defined core-components on the GPU from Python
- Automatic hyperparameter tuning and built-in readers optimized for massive datasets
Efficient Resource Usage
- Paralleism with accuracy on multiple GPUs/machines via 1-bit SGD and Block Momentum
- Memory sharing and other built-in methods to fit even the largest models in GPU memory
Easily Express Your Own Networks
- Full of APIs for defining networks, learners, readers, training and evaluation from Pyton, C++ and BrainScript
- Evaluate models with Python, C++, C# and BrainScript
- Interoperation with NumPy
- Both high-level and low-level APIs available for ease of use and flexibility
- Automatic shape inference based on your data
- Fully optimized symbolic RNN loops (no unrolling needed)
CNTK is also one of the first deep-learning toolkits to support the Open Neural Network Exchange ONYX format, an open-source shared model representation for framework interoperability and shared optimization. Co-developed by Microsoft and supported by many others, ONNX allows developers to move models between frameworks such as CNTK, Caffe2, MXNet, and PyTorch.
To learn more about the Microsoft Cognitive Toolkit or how to get it installed on an Exxact Deep Learning System, click here. Have any questions? Comment below! Like what you’re reading? Let us know on any of our social media platforms.