CSE students break codes in reverse engineering course

Image of Liu's Reverse Engineering Class Spring 2015Reverse engineering is at the heart of numerous applications, such as vulnerability exploit, malware analysis, compatible product development, low level system design and debugging, and legacy software maintenance. It is a particularly critical skill for vulnerability analysis and protection of digital security across all system layers.

Dr. Jyh-Charn (Steve) Liu, a professor in the Department of Computer Science and Engineering at Texas A&M University, developed a new course dedicated to teaching students the importance of reverse engineering.

As part of the curriculum of the course, students are asked to engage in several codebreaker challenges, which progressively become more difficult. Understanding how this process works is vital to understanding reverse engineering at its core.

“The codebreaker challenges are a series of open challenges designed by professional engineers to stimulate the higher education community in learning and developing deep understanding of the fundamentals involved in breaking modern codes,” Liu said.

In 2013, the first codebreaker was released to the public. Until spring of 2014, it remained unsolved. When computer science and engineering first offered the reverse engineering course, student Nathaniel Adkins broke that codebreaker challenge, deciphering nearly all logic involved in the design.

The second codebreaker was released to 13 schools in the winter of 2014. Based on the lessons learned from Codebreaker 1, Codebreaker 2 has been divided into four tiers to allow for incremental learning while doing the first two tiers. Tiers three and four gradually progress into stronger crypto protections.

Currently, six out of 24 students in the class have broken the tier three challenges. Ryan Vrecenar set a record in breaking tier three in an impressive 10 days, followed by James Bowman and Zachary Vernadore. This has all been done in less than two weeks. Vernadore and Vrecenar were the first in the country to break the tier four code, which was designed to be very difficult by brute force attacks.

The course, which has shown great success among students, will become a permanent offering next year.