How to use deep learning AI to detect and prevent malware and APT

[[163896]]

[51CTO.com Quick Translation] Deep Instinct has released a new solution that claims to be able to achieve up to 98.8% real-time APT detection accuracy.

According to the survey results released by the AV-TEST Association, the number of new malware appears every day is as high as 390,000, and the result given by Symantec is even more shocking - the number of new malware is as high as 1 million every day. And these malwares often have not yet appeared in actual malicious activities.

Even if we only calculate at the lowest level, the current security situation is still quite serious. Especially considering the level of advanced persistent threats (APTs), which are the most advanced variants of viruses and malware and can make most current mainstream network security technologies useless. Even security experts have emphasized that enterprises need to adjust the traditional question of "if" an attack is successful to "when it is successful."

Over the past few years, we have seen many different types of malware detection techniques. Initially, detection was done using signatures, where unknown code snippets were compared to known malware. However, with the number of malware growing by hundreds of thousands or even millions every day, this passive approach is no longer sufficient.

The next step in evolution is heuristic detection, which identifies malware based on code behavior characteristics. This means that any suspicious behavior in the code will be recorded. Based on this, the introduction of sandbox technology is a natural step. We need to use this virtual environment to run unknown code and observe whether it is malicious in an isolated manner.

We have recently seen machine learning enter the malware detection space. This technology uses complex algorithms to process files and classify them as malicious or benign based on a series of factors manually extracted from the files. The machine needs a human perspective to decide which parameters, variables, and even functions may be a security risk. Typically, machine learning-based cybersecurity solutions are responsible for the initial screening of suspicious situations, while human analysts take over the final processing.

Now the next step in the evolution has arrived - Deep Instinct claims to have built the first cybersecurity solution on the market based on deep learning technology. Deep learning is an advanced artificial intelligence implementation that uses a process similar to the learning process of the human brain to understand things. Deep learning will have a profound impact on cybersecurity, especially in the detection of zero-day malware, new malware and highly sophisticated APTs.

Once the machine is aware of the characteristics of malicious code, it can determine whether unknown code is malware with excellent accuracy and real-time processing capabilities. So how do machines learn to identify malware? The learning process is very similar to that of humans. Suppose we take a child for a walk in the park and teach him what a dog should look like. During the play, we repeatedly point out different types of dogs to help him learn this concept. Instead of explaining the specific definition of a dog, we repeatedly provide different examples. After a while, the child will be able to correctly identify an animal that he has never seen before as a "dog". And if we show him a photo of a dog, he will be able to find out which animal it is. Furthermore, even if we delete 20% of the pixels, he will still recognize the dog in the photo at a glance.

Deep Instinct uses this approach to help its core engine learn to identify malicious code. The company has collected hundreds of millions of malware varieties, including Word files, PDF files, executable files, etc., but the specific file type is not important-because deep learning is aimed at unknown data types. Deep Instinct scientists run these files and classify them as malicious or legitimate through tests. After that, they use this huge data set to train the engine, and this artificial brain will eventually be able to build a so-called predictive model. At this stage, the core engine will have a similar cognitive method to that of a child-even though it has never seen a certain malware, it can still infer whether it may cause harm based on existing clues.

Deep Instinct packages its predictive model into a small set of probes. The probe can be deployed to any type of device running any operating system - PCs, laptops, tablets, smartphones, and even servers. When a file is opened or downloaded on the device, the probe breaks the file into multiple small pieces and runs the predictive model on them. This so-called "instinct" mechanism uses the training results to detect whether it contains malicious elements. All of this can be done in less than five milliseconds. The entire process on the device is completed in real time, and the decision is made to delete the malware, block it, or perform the corresponding action required by the enterprise - this time the malicious code has no time to cause any damage. Most importantly, it does not affect the user experience at all.

Since the probe already has all the necessary conditions to perform unknown file analysis, there is no need to use the corporate network or even an Internet connection. Specifically, it can protect devices both online and offline. For example, a worker can be in airplane mode while sitting on a plane for safety protection. If he inserts an infected USB drive, the probe on the device will analyze the files in the USB drive and find malware that may cause further infection to the device.

Deep Instinct also launched a probeless version of its solution, which uses the prediction model to add protection functions without involving the device itself. The company also said that it can be connected to any type of gateway through API or SDK. For example, Deep Instinct's model can be integrated with FireLayer's cloud access security broker to achieve malware detection and prevention of threats to cloud files and applications.

Deep Instinct is still training its base engine to ensure that it can identify more new malware. Although its "instinct" mechanism is constantly updated, the probes on the device that have not been updated for several months can still provide very good judgment accuracy. Deep Instinct pointed out that four months without updating will only reduce the malware detection accuracy of its probes by 0.5% to 1%.

Benchmark tests conducted by the University of Drebin and Siemens CERT show that Deep Instinct's solution is comparable to any top security solution on the market. In trying to identify mobile malware, the average accuracy of the top ten security vendors on the market is 61.5%, while Deep Instinct's accuracy is as high as 99.86%. In another test against 16,000 APTs, Deep Instinct's malware identification rate was as high as 98.8%.

The specific implementation process includes installing probes on devices, installing corresponding devices in the network to implement policy management, and providing dashboards and reporting mechanisms. The company said that it will use existing data set files to provide proof of concept solutions for potential customers, which means that corporate customers can directly compare the product with existing network security tools.

Original link: How to use deep learning AI to detect and prevent malware and APT

[Translated by 51CTO. Please indicate the original translator and source as 51CTO.com when reprinting on partner sites]