Download as PDF, PPTX
![29 ClassesParser Disassembler https://github.com/tracer0tong/de Example: ./de.py test1.dex.dat [[0, 'sget-object v0, {type} [{class}].{field} // field@2225'], [2, 'invoke-virtual v0 @13970 // {class}->{method}'], [5, 'move-result-object v0'], [6, 'check-cast v0, [{type_name}] // type@0958'], [8, 'return-object v0']]](https://image.slidesharecdn.com/003-android-131001135019-phpapp01-160314104615/75/Fast-detection-of-Android-malware-machine-learning-approach-29-2048.jpg)
Uploaded byYury Leonychev
Fast detection of Android malware: machine learning approach
The document discusses the fast detection of Android malware using various analysis techniques, including static, dynamic, and hybrid methods. It highlights the implementation of MatrixNet, a machine learning algorithm based on gradient boosted decision trees, which efficiently classifies APK files as malware or normal based on certain features. The document emphasizes the effectiveness of analytic methods in detecting malware while also addressing the significance of open APIs for future use.
Fast detection of Android malware: machine learning approach
- 1.
- 2. 2 Fast detection of Androidmalware Yury Leonychev
- 3.
- 4.
- 5. 5 Brief list oftools for APK analysis ! Androguard (ultimate tool by @adesnos and others) – used by VirusTotal, APKInspector, etc. ! SCanDroid (Adam P. Fuchs, Avik Chaudhuri, and Jeffrey S. Foster) ! TaintDroid (guys from Intel, Penn State University, Duke University) ! DroidBox (dynamic analysis by Lantz Patric) – used by ApkScan
- 6. 6 Is this all?Really? ! http://www.apk-analyzer.net ! http://anubis.iseclab.org ! http://apkscan.nviso.be
- 7. 7 Our task ismore complex Malware detector
- 8. 8 Methods of malwaredetection Static analysis ! Advantages – APK has predictable content. Application behavior can be learned by simply reading the file – Checks are safe ! Limitations – Can be ineffective for sophisticated malware and obfuscation techniques – We cannot really tell as we don't execute app
- 9. 9 Methods of malwaredetection Dynamic analysis ! Advantages – Clear results and interpretation – Open source solutions available ! Limitations – Not fast (enough) – Can be detected and bypassed – Big ecosystem requires big infrastructure
- 10. 10 Methods of malwaredetection Signature analysis ! Advantages – Effective for known malware – Commercial solutions available ! Limitations – Signature databases requires regular (and frequent) updates – Not effective for new malware – Do you have a team of virus analytics?
- 11. 11 Methods of malwaredetection Seems like the most efficient way is hybrid solution
- 12. 12 MatrixNet What is TheMatrix?
- 13. 13 Why can weuse machine learning? Abstract task description: ! We have a set of objects (APK-files). We should divide this set into two subsets (malware and normal) ! For every element in main set we can count predictable amount of features ! Subsets – only result of simple classification task, so we can try to choose effective features
- 14. 14 What is theMatrixNet? MatrixNet is an implementation of gradient boosted decision trees algorithm MatrixNet is a bit different from standard: ! Using Oblivious Trees ! Accounting for sample count in each leaf
- 15. 15 Why MatrixNet ispowerful? ! This is machine learning algorithm for classification task ! A key feature of this method is it’s resistance to overfitting
- 16. 16 MatrixNet post learningoptimization
- 17. 17 MatrixNet post learningoptimization Copyright © 2013 by Sidney Harris.
- 18. 18 How it works? Offlinelearning process: ! Choosing features ! Choosing samples ! Manual classification (malware or not) ! Learning on combined set of apps ! Calculating mistakes
- 19. 19 Features What kind offeatures to use: ! Permissions ! URI in strings and other resources ! Adware library usage ! Obfuscation methods ! …
- 20. 20 Samples and classification Malwareapplications: ! VirusTotal feed ! Samples from malicious sites Normal applications: ! Manual testing ! Trusted developers ! Yandex applications
- 21.
- 22. 22 Measuring of mistakes Formula1 Features cost 1 Formula N Features cost N Normal Malware Formula with cool confusion matrix and low cost
- 23. 23 Analyzer architecture Fine! I'llgo build my own casino, with blackjack and big data
- 24.
- 25. 25 Parsers In depth APK ManifestParser ResourceParserMetaInfoParser ClassesParser Analyzers PermissionAnalyzer PackageAnalyzer URLAnalyzer ReflectionAnalyzer Reports XHTMLReporter JSONReporter Oracle MatrixNet
- 26. 26 ManifestParser Avoid some obfuscationmethods: ! HEUR:Backdoor.AndroidOS.Obad.a
- 27. 27 <?xml version="1.0" encoding="utf-8"?> <manifest="singleTop" android:versionCode="2" ="2.0" android:installLocation="internalOnly" package="com.android.system.admin" xmlns:android="http://schemas.android.com/apk/res/android"> <uses-permission ="android.permission.READ_LOGS" /> <uses-permission ="android.permission.WAKE_LOCK" /> … <uses-permission ="android.permission.RECEIVE_SMS" /> <uses-permission ="android.permission.SEND_SMS" /> <uses-permission ="android.permission.CALL_PHONE" /> ManifestParser
- 28. 28 ClassesParser ! Parser forDEX files ! Internal DEX disassembler ! Callgraph builder ! Embeds “real” functions/variables names into disassembly listing ! Builds a list of used procedures and functions
- 29. 29 ClassesParser Disassembler https://github.com/tracer0tong/de Example: ./de.py test1.dex.dat [[0, 'sget-objectv0, {type} [{class}].{field} // field@2225'], [2, 'invoke-virtual v0 @13970 // {class}->{method}'], [5, 'move-result-object v0'], [6, 'check-cast v0, [{type_name}] // type@0958'], [8, 'return-object v0']]
- 30. 30 ReflectionAnalyzer java.lang.reflect.* ! Classes: Field,Method, etc. ! Functions: getClass(), getDeclaredField(), etc.
- 31. 31 ReflectionAnalyzer Output: ! Report: There issome reflections usage: 1@android.app.Activity->getContentResolver calls: 598@java.lang.Class->forName 2@android.app.Activity->onActivityResult calls: 598@java.lang.Class->forName ! Amount of reflection calls is a feature.
- 32. 32 Service architecture Nginx Gunicorn Flask Celery MongoDB Nginx Gunicorn Flask Celery MongoDB
- 33.
- 34. 34 Let's try iton... Yandex.Store application feed: ! More than 50K Android applications ! More than 200 new/updated apps per week ! Open for developers (no strict manual verification)
- 35. 35 Perfomance. Check timing ~2ms ~0,25 s ~4,5 min
- 36. 36 Performance. Amount ofchecks ! More than 16.000 applications checked in 1 hour on 1 cluster node
- 37. 37 Confusion matrix Meaning Malware (Score> 0) Normal (Score < 0) Fact Malware 485 (97%) 15 (3%) Normal 25 (5%) 475 (95%)
- 38. 38 (Un)predictable results ! Applicationswith malicious adware library AirPush classified as malware ! But we have no special features for adware in first version
- 39.
- 40. 40 It works! ! Analyticmethods work fine for detection Android mobile malware ! Machine learning is not a “rocket science” but cool and effective instrument ! Open API coming soon.
- 41.
- 42. 42 Yury Leonychev Application Security Engineer yleonychev@yandex-team.ru ! tracer0tong© Yandex LLC 2013