If you want to analyze software in order to exploit its weaknesses and strengthen its defenses, then you should explore reverse engineering. Reverse Engineering is a hackerfriendly tool used to expose security flaws and questionable privacy practices.In this book, you will learn how to analyse software even without having access to its source code or design documents. You will start off by learning the low-level language used to communicate with the computer and then move on to covering reverse engineering techniques. Next, you will explore analysis techniques using real-world tools such as IDA Pro and x86dbg. As you progress through the chapters, you will walk through use cases encountered in reverse engineering, such as encryption and compression, used to obfuscate code, and how to to identify and overcome anti-debugging and anti-analysis tricks. Lastly, you will learn how to analyse other types of files that contain code.
Author(s): Reginald Wong
Publisher: Packt Publishing
Year: 2018
Language: English
Pages: 423
Tags: Reverse Engineering
Cover......Page 1
Title Page......Page 2
Copyright and Credits......Page 3
Packt Upsell......Page 4
Contributors......Page 5
Table of Contents......Page 7
Preface......Page 13
Reverse engineering......Page 18
Seeking approval......Page 21
Reporting......Page 22
Binary analysis tools......Page 23
Debuggers......Page 25
Malware handling......Page 26
Basic analysis lab setup......Page 27
Our setup......Page 28
Samples......Page 33
Summary......Page 34
Chapter 2: Identification and Extraction of Hidden Components......Page 35
The filesystem......Page 36
Memory......Page 38
The registry system......Page 39
Typical malware behavior......Page 40
Persistence......Page 41
Run keys......Page 42
Load and Run values......Page 43
Startup values......Page 46
The Image File Execution Options key......Page 47
Malware delivery......Page 48
Email......Page 49
Instant messenger......Page 50
The computer network......Page 51
Media storage......Page 52
Exploits and compromised websites......Page 53
Malware file properties......Page 55
Payload – the evil within......Page 56
Tools......Page 57
Autoruns......Page 59
The Process explorer......Page 60
Further reading......Page 63
Chapter 3: The Low-Level Language......Page 64
Bases......Page 65
Converting between bases......Page 66
Binary arithmetic......Page 68
Signed numbers......Page 69
x86......Page 70
Registers......Page 71
Endianness......Page 74
Basic instructions......Page 75
Copying data......Page 76
MOV and LEA......Page 77
Arithmetic operations......Page 78
Addition and subtraction......Page 79
Multiplication and division instructions......Page 80
Other signed operations......Page 81
Bitwise algebra......Page 82
Control flow......Page 84
Stack manipulation......Page 86
Tools – builder and debugger......Page 87
MASM......Page 88
NASM......Page 90
FASM......Page 91
WinDbg......Page 92
Ollydebug......Page 94
Hello World......Page 95
It works!......Page 96
Dissecting the program ......Page 99
Common Windows API libraries......Page 104
Debugging......Page 105
Further reading......Page 114
Chapter 4: Static and Dynamic Reversing......Page 115
Static analysis......Page 116
PEid and TrID......Page 117
MASTIFF......Page 119
PE executables......Page 121
IDA (Interactive Disassembler)......Page 127
Dynamic analysis......Page 128
Memory regions and the mapping of a process......Page 129
Post-execution differences......Page 133
Try it yourself......Page 134
References......Page 143
Analysis environments......Page 144
Virtual machines......Page 145
Windows......Page 146
Information gathering tools......Page 147
Hash identifying......Page 148
Monitoring tools......Page 149
Disassemblers......Page 150
Debuggers......Page 151
Decompilers......Page 152
Network tools......Page 153
Attack tools......Page 154
Software forensic tools......Page 155
Automated dynamic analysis......Page 156
Online service sites......Page 157
Summary......Page 158
Setup......Page 159
Linux executable – hello world......Page 160
dlroW olleH......Page 161
Dynamic analysis......Page 168
Going further with debugging......Page 170
A better debugger......Page 177
Hello World in Radare2......Page 178
What is the password?......Page 184
Network traffic analysis......Page 192
Further reading......Page 198
Hello World......Page 199
Learning about the APIs......Page 200
Keylogger......Page 201
regenum......Page 203
processlist......Page 205
Encrypting and decrypting a file......Page 206
The server......Page 210
What is the password?......Page 212
Static analysis......Page 213
Deadlisting......Page 217
Dynamic analysis with debugging......Page 234
Decompilers......Page 242
Further reading......Page 244
Chapter 8: Sandboxing - Virtualization as a Component for RE......Page 245
Emulation......Page 246
Emulators......Page 247
Linux ARM guest in QEMU......Page 248
MBR debugging with Bochs......Page 250
Further Reading......Page 259
Data assembly on the stack......Page 260
Code assembly......Page 262
Loop codes......Page 264
Simple arithmetic......Page 265
Simple XOR decryption......Page 266
Assembly of data in other memory regions......Page 267
Decrypting with x86dbg......Page 268
Control flow flattening obfuscation......Page 271
Code obfuscation with a metamorphic engine......Page 274
Dynamic library loading......Page 277
Use of PEB information......Page 278
Summary......Page 279
Chapter 10: Packing and Encryption......Page 280
A quick review on how native executables are loaded by the OS......Page 281
Packers or compressors......Page 284
Crypters......Page 286
Obfuscators......Page 288
Protectors......Page 289
SFX Self-extracting archives......Page 290
Debugging though the packer......Page 291
Memory dumping with VirtualBox......Page 305
Extracting the process to a file using Volatility......Page 306
How about an executable in its unpacked state?......Page 309
Other file-types......Page 312
Summary......Page 316
Anti-debugging tricks......Page 317
IsDebuggerPresent......Page 318
Debug flags in the PEB......Page 319
Timing tricks......Page 321
Passing code execution via SEH......Page 322
Causing exceptions......Page 325
Anti-VM tricks......Page 326
Existence of VM files and directories......Page 327
Registry entries made by VMs......Page 328
CPUID results......Page 329
Anti-emulation tricks......Page 330
Anti-dumping tricks......Page 331
Summary......Page 332
Things to prepare......Page 333
Initial static analysis......Page 334
Initial file information......Page 335
Deadlisting......Page 341
Debugging......Page 353
The unknown image......Page 363
Analysis summary......Page 387
Further Reading......Page 389
Analysis of HTML scripts......Page 390
MS Office macro analysis......Page 397
PDF file analysis......Page 401
SWFTools......Page 403
Flare......Page 405
XXXSWF......Page 406
JPEXS SWF decompiler ......Page 407
Summary......Page 411
Further reading......Page 412
Other Books You May Enjoy......Page 413
Index......Page 416
