Application of x86 extensions for code protection
A new approach is proposed to protect the program code against reverse engineering tools, such as decompilers and symbolic execution tools. The approach is based on the usage of uncommon x86 processor instructions that could be implemented incorrectly in the aforementioned tools. Existing approaches to this problem are also considered, and the relative performance advantage of the proposed approach is noted. A method for numeric constants obfuscation, following this approach, is developed with the usage of AES-NI extension for the x86 architecture and its AESENC instruction in particular. This method is implemented for Clang compiler with the help of LLVM Intermediate Representation and tested against reverse engineering tools, such as IDA and Ghidra decompilers and angr symbolic execution tool.
Keywords
code protection,
reverse engineering,
decompiler,
symbolic execution,
x86 processor architectureAuthors
| Lebedev R. K. | Novosibirsk State University | n0n3m4@gmail.com |
| Koryakin I. A. | Novosibirsk State University | ed4140@gmail.com |
Всего: 2
References
Shoshitaishvili Y., Wang R., Salls C., et al. SOK: (State of) The art of war: Offensive techniques in binary analysis // IEEE Symp. Security Privacy. 2016. P. 138-157.
https://www.hex-rays.com/ida-pro/- IDA Pro. 2021.
https://github.com/NationalSecurityAgency/ghidra - Ghidra Software Reverse Engineering Framework. 2021.
Lattner C. and Adve V. LLVM: A compilation framework for lifelong program analysis & transformation // Intern. Symp. Code Generation and Optimization. 2004. P. 75-86
https://intelxed.github.io/- Intel XED. 2019.
https : //software. intel. com/content/www/us/en/develop/articles/intel-advanced-encryption-standard-instructions-aes-ni.html - Intel® Advanced Encryption Standard Instructions (AES-NI). 2012.
Лебедев Р. К. Автоматическая генерация хэш-функций для обфускации программного кода // Прикладная дискретная математика. 2020. №50. С. 102-117
Wang Z., Ming J., Jia C., and Gao D. Linear obfuscation to combat symbolic execution // Proc. European Symp. Research Computer Security. 2011. P. 210-226.
Seto T., Monden A., Yucel Z., and Kanzaki Y. On preventing symbolic execution attacks by low cost obfuscation // 20th IEEE/ACIS Intern. Conf. Software Eng., Artif. Intelligence, Networking and Parallel/Distributed Comput. (SNPD). 2019. P. 495-500
Junod P., Rinaldini J., Wehrli J., and Michielin J. Obfuscator-LLVM - software protection for the masses // 2015 IEEE/ACM 1st Intern. Workshop Software Protection. 2015. P. 3-9.
