PyTorch 1.9 released! Crazy update of mobile terminal, netizens: my favorite

[[405980]]

This article is reprinted with permission from AI new media Quantum Bit (public account ID: QbitAI). Please contact the source for reprinting.

After only 3 months, PyTorch has been upgraded again - version 1.9 .

This time, the official focused on the mobile side .

Not only has a new version of Mobile Interpreter been released, but the TorchVision library also supports use on mobile phones, both iOS and Android!

Netizens all said after reading it:

In this update, I am most interested in the mobile version.

This version 1.9 brings together more than 3,400 GitHub commits since the release of version 1.8 in March 2021.

In addition to the mobile device side, there are many other highlights:

• Front-end API improvements (including torch.linalg, torch.special, and Complex Autograd)
• Implement native support for elastic, fault-tolerant distributed training
• Updated PyTorch model performance analyzer

LeCun also took to Twitter to praise PyTorch for getting better and better!

Using TorchVision library on mobile phone

The first is the update of Mobile Interpreter, one of the most popular features of PyTorch Mobile.

The latest version reduces the binary size on mobile devices to less than half of its original size.

For example, the pt size of MobileNetV2 used in an Android device with arm64-v8a architecture is 17.8MB before compression and 8.6MB after compression.

After using the new version of Mobile Interpreter, the file size before compression can be reduced to less than 8MB, and the size after compression can be reduced to less than 4MB.

At the same time, starting from version 1.9, users can use the TorchVision library on iOS and Android apps.

On iOS, it needs to be linked with the main PyTorch library; on Android, it can be added as a gradle dependency.

In terms of demo APP, this time we updated a new video APP based on the PyTorch Video library and a speech recognition APP based on the latest torchaudio and wave2vec models.

With these two apps, PyTorch can now provide a complete set of demo apps including images, text, audio and video.

Front-end API improvements

In version 1.9, modules such as torch.linalg, torch.special and Complex Autograd have been improved.

The torch.linalg module now implements every function in the NumPy linear algebra module;

Complex Autograd has new features that can calculate complex gradients and optimize loss functions.

Additionally, to help with debugging and writing reproducible programs, PyTorch 1.9 adds a torch.use_determinstic_algorithms option.

This is to avoid possible errors during operation, as shown below:

Distributed training

TorchElastic is a core feature of PyTorch that enables users to run distributed training on preemptible instances.

△TorchElastic operating principle

In the new version, a "standalone" collection point based on c10d::Store is added, which can support elastic and fault-tolerant distributed training locally.

In addition, CUDA is now supported in RPC, and analysis of distributed training is supported, etc.

PyTorch Profiler

PyTorch Profiler is a tool for analyzing the performance of PyTorch models.

The visualization page helps us analyze the specific operation status.

In version 1.9, the new torch.profiler API is supported on Windows and Mac.

The new API supports existing profiler functionality, enables integration with the CUPTI library (Linux only), traces CUDA kernels on the device, and provides support for long-running projects such as:

The PyTorch Profiler Tensorboard plugin has also been updated with new features such as a distributed training summary view with NCCL, a memory analysis view, and the ability to jump to source code when launched from Microsoft VSCode.

For more updated information, please click:

https://pytorch.org/blog/pytorch-1.9-released/