Reverse Engineering for Beginners (Lite version)
Free

Reverse Engineering for Beginners (Lite version)

By Dennis Yurichev
Free
Book Description


There are several popular meanings of the term “reverse engineering”:




  1. The reverse engineering of software: researching compiled programs;


  2. The scanning of 3D structures and the subsequent digital manipulation required order to duplicate them;


  3. recreating DBMS9 structure.



This book is about the first meaning. This is the Lite version. It is approximately 6 times shorter than the full version (~150 pages) and is intended for those who want a very quick introduction to the basics of reverse engineering. It does not however, cover the topics of: OllyDBG, GCC, GDB and IDA. There are also no exercises, examples, etc.



Topics discussed: x86/x64, Java/JVM.



Topics touched: Oracle RDBMS, Itanium, copy-protection dongles, LD_PRELOAD, stack overflow, ELF, win32 PE file format, x86-64, critical sections, syscalls, TLS, position-independent code (PIC), profile-guided optimization, C++ STL, OpenMP, win32 SEH.



Also in Russian.

Table of Contents
  • I Code patterns
    • A short introduction to the CPU
    • The simplest Function
      • x86
    • Hello, world!
      • x86
        • MSVC
      • x86-64
        • MSVC—x86-64
      • Conclusion
    • Function prologue and epilogue
      • Recursion
    • Stack
      • Why does the stack grow backwards?
      • What is the stack used for?
        • Save the function's return address
        • Passing function arguments
        • Local variable storage
        • x86: alloca() function
        • (Windows) SEH
        • Buffer overflow protection
        • Automatic deallocation of data in stack
      • A typical stack layout
    • printf() with several arguments
      • x86
        • x86: 3 arguments
        • x64: 8 arguments
      • Conclusion
      • By the way
    • scanf()
      • Simple example
        • About pointers
        • x86
        • x64
      • Global variables
        • MSVC: x86
        • MSVC: x64
      • scanf() result checking
        • MSVC: x86
        • MSVC: x86 + Hiew
        • MSVC: x64
      • Exercises
        • Exercise #1
    • Accessing passed arguments
      • x86
        • MSVC
      • x64
        • MSVC
    • More about results returning
      • Attempt to use the result of a function returning void
      • What if we do not use the function result?
    • GOTO operator
      • Dead code
    • Conditional jumps
      • Simple example
        • x86
      • Calculating absolute value
        • Optimizing MSVC
      • Ternary conditional operator
        • x86
        • Let's rewrite it in an if/else way
      • Getting minimal and maximal values
        • 32-bit
      • Conclusion
        • x86
        • Branchless
    • switch()/case/default
      • Small number of cases
        • x86
        • Conclusion
      • A lot of cases
        • x86
        • Conclusion
      • When there are several case statements in one block
        • MSVC
      • Fall-through
        • MSVC x86
    • Loops
      • Simple example
        • x86
        • One more thing
      • Memory blocks copying routine
        • Straight-forward implementation
      • Conclusion
    • Simple C-strings processing
      • strlen()
        • x86
    • Replacing arithmetic instructions to other ones
      • Multiplication
        • Multiplication using addition
        • Multiplication using shifting
        • Multiplication using shifting, subtracting, and adding
      • Division
        • Division using shifts
    • Arrays
      • Simple example
        • x86
      • Buffer overflow
        • Reading outside array bounds
        • Writing beyond array bounds
      • One more word about arrays
      • Array of pointers to strings
        • x64
      • Multidimensional arrays
        • Two-dimensional array example
        • Access two-dimensional array as one-dimensional
        • Three-dimensional array example
      • Conclusion
    • Manipulating specific bit(s)
      • Specific bit checking
        • x86
      • Setting and clearing specific bits
        • x86
      • Shifts
      • Counting bits set to 1
        • x86
        • x64
      • Conclusion
        • Check for specific bit (known at compile stage)
        • Check for specific bit (specified at runtime)
        • Set specific bit (known at compile stage)
        • Set specific bit (specified at runtime)
        • Clear specific bit (known at compile stage)
        • Clear specific bit (specified at runtime)
    • Linear congruential generator
      • x86
      • x64
    • Structures
      • MSVC: SYSTEMTIME example
        • Replacing the structure with array
      • Let's allocate space for a structure using malloc()
      • Fields packing in structure
        • x86
        • One more word
      • Nested structures
      • Bit fields in a structure
        • CPUID example
    • 64-bit values in 32-bit environment
      • Returning of 64-bit value
        • x86
      • Arguments passing, addition, subtraction
        • x86
      • Multiplication, division
        • x86
      • Shifting right
        • x86
      • Converting 32-bit value into 64-bit one
        • x86
    • 64 bits
      • x86-64
  • II Important fundamentals
    • Signed number representations
    • Memory
  • III Finding important/interesting stuff in the code
    • Communication with the outer world (win32)
      • Often used functions in the Windows API
      • tracer: Intercepting all functions in specific module
    • Strings
      • Text strings
        • C/C++
        • Borland Delphi
        • Unicode
        • Base64
      • Error/debug messages
      • Suspicious magic strings
    • Calls to assert()
    • Constants
      • Magic numbers
        • DHCP
      • Searching for constants
    • Finding the right instructions
    • Suspicious code patterns
      • XOR instructions
      • Hand-written assembly code
    • Using magic numbers while tracing
    • Other things
      • General idea
      • Some binary file patterns
      • Memory "snapshots" comparing
        • Windows registry
        • Blink-comparator
  • IV Tools
    • Disassembler
      • IDA
    • Debugger
      • tracer
    • Decompilers
    • Other tools
  • V Books/blogs worth reading
    • Books
      • Windows
      • C/C++
      • x86 / x86-64
      • ARM
      • Cryptography
    • Blogs
      • Windows
    • Other
  • Afterword
    • Questions?
  • Acronyms used
  • Glossary
  • Index
  • Bibliography
    You May Also Like
    Also Available On
    Categories
    Curated Lists