Name: Reverse Engineering for Beginners
ISBN: 0262033844

Dennis Yurichev

Computers & Technology

Reverse Engineering for Beginners

Free

Read Now

Description

Contents

Reviews

There are several popular meanings of the term “reverse engineering”:

The reverse engineering of software: researching compiled programs;

The scanning of 3D structures and the subsequent digital manipulation required order to duplicate them;

recreating DBMS9 structure.

This book is about the first meaning. This is the Full version. It is approximately 6 times longer than the Lite version which is intended for those who want a very quick introduction to the basics of reverse engineering. The full version covers the topics of: OllyDBG, GCC, GDB and IDA. There are also exercises, examples, etc.

Topics discussed: x86/x64, ARM/ARM64, MIPS, Java/JVM.

Topics touched: Oracle RDBMS, Itanium, copy-protection dongles, LD_PRELOAD, stack overflow, ELF, win32 PE file format, x86-64, critical sections, syscalls, TLS, position-independent code (PIC), profile-guided optimization, C++ STL, OpenMP, win32 SEH.

Also in Russian.

Language

English

ISBN

0262033844

I Code patterns

A short introduction to the CPU

A couple of words about different ISAs

The simplest Function

x86

ARM

MIPS

A note about MIPS instruction/register names

Hello, world!

x86

MSVC

GCC

GCC: AT&T syntax

x86-64

MSVC—x86-64

GCC—x86-64

GCC—one more thing

ARM

Non-optimizing Keil 6/2013 (ARM mode)

Non-optimizing Keil 6/2013 (Thumb mode)

Optimizing Xcode 4.6.3 (LLVM) (ARM mode)

Optimizing Xcode 4.6.3 (LLVM) (Thumb-2 mode)

ARM64

MIPS

A word about the "global pointer"

Optimizing GCC

Non-optimizing GCC

Role of the stack frame in this example

Optimizing GCC: load it into GDB

Conclusion

Exercises

Exercise #1

Exercise #2

Function prologue and epilogue

Recursion

Stack

Why does the stack grow backwards?

What is the stack used for?

Save the function's return address

Passing function arguments

Local variable storage

x86: alloca() function

(Windows) SEH

Buffer overflow protection

Automatic deallocation of data in stack

A typical stack layout

Noise in stack

Exercises

Exercise #1

Exercise #2

printf() with several arguments

x86

x86: 3 arguments

x64: 8 arguments

ARM

ARM: 3 arguments

ARM: 8 arguments

MIPS

3 arguments

8 arguments

Conclusion

By the way

scanf()

Simple example

About pointers

x86

MSVC + OllyDbg

x64

ARM

MIPS

Global variables

MSVC: x86

MSVC: x86 + OllyDbg

GCC: x86

MSVC: x64

ARM: Optimizing Keil 6/2013 (Thumb mode)

ARM64

MIPS

scanf() result checking

MSVC: x86

MSVC: x86: IDA

MSVC: x86 + OllyDbg

MSVC: x86 + Hiew

MSVC: x64

ARM

MIPS

Exercise

Exercises

Exercise #1

Accessing passed arguments

x86

MSVC

MSVC + OllyDbg

GCC

x64

MSVC

GCC

GCC: uint64_t instead of int

ARM

Non-optimizing Keil 6/2013 (ARM mode)

Optimizing Keil 6/2013 (ARM mode)

Optimizing Keil 6/2013 (Thumb mode)

ARM64

MIPS

More about results returning

Attempt to use the result of a function returning void

What if we do not use the function result?

Returning a structure

Pointers

Global variables example

Local variables example

Conclusion

GOTO operator

Dead code

Exercise

Conditional jumps

Simple example

x86

ARM

MIPS

Calculating absolute value

Optimizing MSVC

Optimizing Keil 6/2013: Thumb mode

Optimizing Keil 6/2013: ARM mode

Non-optimizing GCC 4.9 (ARM64)

MIPS

Branchless version?

Ternary conditional operator

x86

ARM

ARM64

MIPS

Let's rewrite it in an if/else way

Conclusion

Getting minimal and maximal values

32-bit

64-bit

MIPS

Conclusion

x86

ARM

MIPS

Branchless

Exercise

switch()/case/default

Small number of cases

x86

ARM: Optimizing Keil 6/2013 (ARM mode)

ARM: Optimizing Keil 6/2013 (Thumb mode)

ARM64: Non-optimizing GCC (Linaro) 4.9

ARM64: Optimizing GCC (Linaro) 4.9

MIPS

Conclusion

A lot of cases

x86

ARM: Optimizing Keil 6/2013 (ARM mode)

ARM: Optimizing Keil 6/2013 (Thumb mode)

MIPS

Conclusion

When there are several case statements in one block

MSVC

GCC

ARM64: Optimizing GCC 4.9.1

Fall-through

MSVC x86

ARM64

Exercises

Exercise #1

Loops

Simple example

x86

x86: OllyDbg

x86: tracer

ARM

MIPS

One more thing

Memory blocks copying routine

Straight-forward implementation

ARM in ARM mode

MIPS

Vectorization

Conclusion

Exercises

Exercise #1

Exercise #2

Exercise #3

Exercise #4

Simple C-strings processing

strlen()

x86

ARM

MIPS

Exercises

Exercise #1

Replacing arithmetic instructions to other ones

Multiplication

Multiplication using addition

Multiplication using shifting

Multiplication using shifting, subtracting, and adding

Division

Division using shifts

Exercises

Exercise #2

Floating-point unit

IEEE 754

x86

ARM, MIPS, x86/x64 SIMD

C/C++

Simple example

x86

ARM: Optimizing Xcode 4.6.3 (LLVM) (ARM mode)

ARM: Optimizing Keil 6/2013 (Thumb mode)

ARM64: Optimizing GCC (Linaro) 4.9

ARM64: Non-optimizing GCC (Linaro) 4.9

MIPS

Passing floating point numbers via arguments

x86

ARM + Non-optimizing Xcode 4.6.3 (LLVM) (Thumb-2 mode)

ARM + Non-optimizing Keil 6/2013 (ARM mode)

ARM64 + Optimizing GCC (Linaro) 4.9

MIPS

Comparison example

x86

ARM

ARM64

MIPS

Stack, calculators and reverse Polish notation

x64

Exercises

Exercise #1

Exercise #2

Arrays

Simple example

x86

ARM

MIPS

Buffer overflow

Reading outside array bounds

Writing beyond array bounds

Buffer overflow protection methods

Optimizing Xcode 4.6.3 (LLVM) (Thumb-2 mode)

One more word about arrays

Array of pointers to strings

x64

32-bit ARM

ARM64

MIPS

Array overflow

Multidimensional arrays

Two-dimensional array example

Access two-dimensional array as one-dimensional

Three-dimensional array example

More examples

Pack of strings as a two-dimensional array

32-bit ARM

ARM64

MIPS

Conclusion

Exercises

Exercise #1

Exercise #2

Exercise #3

Exercise #4

Exercise #5

Manipulating specific bit(s)

Specific bit checking

x86

ARM

Setting and clearing specific bits

x86

ARM + Optimizing Keil 6/2013 (ARM mode)

ARM + Optimizing Keil 6/2013 (Thumb mode)

ARM + Optimizing Xcode 4.6.3 (LLVM) (ARM mode)

ARM: more about the BIC instruction

ARM64: Optimizing GCC (Linaro) 4.9

ARM64: Non-optimizing GCC (Linaro) 4.9

MIPS

Shifts

Setting and clearing specific bits: FPU example

A word about the XOR operation

x86

MIPS

ARM

Counting bits set to 1

x86

x64

ARM + Optimizing Xcode 4.6.3 (LLVM) (ARM mode)

ARM + Optimizing Xcode 4.6.3 (LLVM) (Thumb-2 mode)

ARM64 + Optimizing GCC 4.9

ARM64 + Non-optimizing GCC 4.9

MIPS

Conclusion

Check for specific bit (known at compile stage)

Check for specific bit (specified at runtime)

Set specific bit (known at compile stage)

Set specific bit (specified at runtime)

Clear specific bit (known at compile stage)

Clear specific bit (specified at runtime)

Exercises

Exercise #1

Exercise #2

Exercise #3

Exercise #4

Linear congruential generator

x86

x64

32-bit ARM

MIPS

MIPS relocations

Thread-safe version of the example

Structures

MSVC: SYSTEMTIME example

OllyDbg

Replacing the structure with array

Let's allocate space for a structure using malloc()

UNIX: struct tm

Linux

ARM

MIPS

Structure as a set of values

Structure as an array of 32-bit words

Structure as an array of bytes

Fields packing in structure

x86

ARM

MIPS

One more word

Nested structures

OllyDbg

Bit fields in a structure

CPUID example

Working with the float type as with a structure

Exercises

Exercise #1

Exercise #2

Unions

Pseudo-random number generator example

x86

MIPS

ARM (ARM mode)

Calculating machine epsilon

x86

ARM64

MIPS

Conclusion

Fast square root calculation

Pointers to functions

MSVC

MSVC + OllyDbg

MSVC + tracer

MSVC + tracer (code coverage)

GCC

GCC + GDB (with source code)

GCC + GDB (no source code)

64-bit values in 32-bit environment

Returning of 64-bit value

x86

ARM

MIPS

Arguments passing, addition, subtraction

x86

ARM

MIPS

Multiplication, division

x86

ARM

MIPS

Shifting right

x86

ARM

MIPS

Converting 32-bit value into 64-bit one

x86

ARM

MIPS

SIMD

Vectorization

Addition example

Memory copy example

SIMD strlen() implementation

64 bits

x86-64

ARM

Float point numbers

Working with floating point numbers using SIMD

Simple example

x64

x86

Passing floating point number via arguments

Comparison example

x64

x86

Calculating machine epsilon: x64 and SIMD

Pseudo-random number generator example revisited

Summary

ARM-specific details

Number sign (#) before number

Addressing modes

Loading a constant into a register

32-bit ARM

ARM64

Relocs in ARM64

MIPS-specific details

Loading constants into register