The Current State of PyTorch & TensorFlow in 2020
Over the past few years we’ve seen the narrative shift from: “What deep learning framework should I learn/use?” to “PyTorch vs TensorFlow, which one should I learn/use?”… and so on. So while this debate on reddit rages on, let’s take a practical look at each framework, its current capabilities, why each commands a large share, and what we can expect for each in 2020.
PyTorch vs TensorFlow: Research vs Production
The Gradient recently released a blog that dramatically shows PyTorch’s ascent and adoption in the research community (based on the number of papers implemented at major conferences (CVPR, ICRL, ICML, NIPS, ACL, ICCV etc.). As you can see from the data, in 2018 PyTorch was clearly a minority, compared with 2019 it’s overwhelmingly favored by researchers at major conferences.
The fact that PyTorch is python native, and integrates easily with other python packages makes this a simple choice for researchers. Many researchers use Pytorch because the API is intuitive and easier to learn, and get into experimentation quickly, rather than reading through documentation.
Is TensorFlow dying for research?
Not really, despite the numbers you see, keep in mind the ‘Google’ crowd alone will be enough to keep TensorFlow alive as far as it’s suitability of research. Furthermore, TensorFlow 2.0 may appeal to the research audience with eager mode and native Keras integration.
Tensorflow in Production Environments
The aforementioned Gradient article also looked at job listings from 2018-2019 where they found hat TensorFlow is still the dominant framework in industry. “For example, based on data from 2018 to 2019, TensorFlow had 1541 new job listings vs. 1437 job listings for PyTorch on public job boards, 3230 new TensorFlow Medium articles vs. 1200 PyTorch, 13.7k new GitHub stars for TensorFlow vs 7.2k for PyTorch, etc.” and as where Researchers are not typically gated heavily by performance considerations, as where Industry typically considers performance to be of the utmost priority. “While 10% faster runtime means nothing to a researcher, that could directly translate to millions of savings for a company.” Perhaps the biggest is the fact that TensorFlow was built with production in mind, in the fact that it can be served on mobile and serving applications, without the need for Python overhead. We’ll see in the following sections how PyTorch is trying to overcome these shortcomings
So Whats the Latest on TensorFlow vs. PyTorch?
TensorFlow 2.1.0 …comes to Windows?
In the latest release of TensorFlow, the tensorflow pip package now includes GPU support by default (same as tensorflow-gpu) for both Linux and Windows. This runs on machines with and without NVIDIA GPUs. tensorflow-gpu is still available, and CPU-only packages can be downloaded at tensorflow-cpu for users who are concerned about package size. Officially-released tensorflow Pip packages are now built with Visual Studio 2019 version 16.4 in order to take advantage of the new /d2ReducedOptimizeHugeFunctions compiler flag. Whether or not TensorFlow becomes popular on windows is yet to be seen.
PyTorch 1.4.0 adds mobile build customization
The release contains significant improvements to mobile and serving area. The new update features JIT, ONNX, Distributed, Performance and Eager Frontend Improvements and improvements to experimental areas like mobile and quantization. It also contains new experimental features including rpc-based model parallel distributed training and language bindings for the Java language (inference only).
PyTorch vs. TensorFlow in 2020 Final Thoughts
Obviously in the best scenario you will be a master in both frameworks, however this may not be possible or practicable to learn both. If your a researcher starting out in deep learning, it may behoove you to take a crack at PyTorch first, as it is popular in the research community. If you know your way around DL/ML and looking to get into industry perhaps TensorFlow should be your primary language. It’s probably a good idea to know a fair bit of both frameworks, and be able to take advantage of the benefits of either.