Home > Store

Forensic Discovery

Register your product to gain access to bonus material or receive a coupon.

Forensic Discovery

Book

  • Sorry, this book is no longer in print.
Not for Sale

About

Features

Internationally recognized computer security experts offer advice and tools for achieving security through computer forensic analysis.

° Authors are renowned for developing notorious computer-security software such as SATAN (Security Administrator's Tool for Analyzing Networks) and the Coroner's Toolkit.

° Covers looking at systems in a new way--one that affords the opportunity to understand how systems can be compromised.

° Only titles approved by Brian Kernighan are allowed into Professional Computing Series.

Description

  • Copyright 2005
  • Edition: 1st
  • Book
  • ISBN-10: 0-201-63497-X
  • ISBN-13: 978-0-201-63497-6

"Don't look now, but your fingerprints are all over the cover of this book. Simply picking it up off the shelf to read the cover has left a trail of evidence that you were here.

    "If you think book covers are bad, computers are worse. Every time you use a computer, you leave elephant-sized tracks all over it. As Dan and Wietse show, even people trying to be sneaky leave evidence all over, sometimes in surprising places.

    "This book is about computer archeology. It's about finding out what might have been based on what is left behind. So pick up a tool and dig in. There's plenty to learn from these masters of computer security."
   --Gary McGraw, Ph.D., CTO, Cigital, coauthor of Exploiting Software and Building Secure Software

"A wonderful book. Beyond its obvious uses, it also teaches a great deal about operating system internals."
   --Steve Bellovin, coauthor of Firewalls and Internet Security, Second Edition, and Columbia University professor

"A must-have reference book for anyone doing computer forensics. Dan and Wietse have done an excellent job of taking the guesswork out of a difficult topic."
   --Brad Powell, chief security architect, Sun Microsystems, Inc.

"Farmer and Venema provide the essential guide to 'fossil' data. Not only do they clearly describe what you can find during a forensic investigation, they also provide research found nowhere else about how long data remains on disk and in memory. If you ever expect to look at an exploited system, I highly recommend reading this book."
   --Rik Farrow, Consultant, author of Internet Security for Home and Office

"Farmer and Venema do for digital archaeology what Indiana Jones did for historical archaeology. Forensic Discovery unearths hidden treasures in enlightening and entertaining ways, showing how a time-centric approach to computer forensics reveals even the cleverest intruder."
   --Richard Bejtlich, technical director, ManTech CFIA, and author of The Tao of Network Security Monitoring

"Farmer and Venema are 'hackers' of the old school: They delight in understanding computers at every level and finding new ways to apply existing information and tools to the solution of complex problems."
   --Muffy Barkocy, Senior Web Developer, Shopping.com

"This book presents digital forensics from a unique perspective because it examines the systems that create digital evidence in addition to the techniques used to find it. I would recommend this book to anyone interested in learning more about digital evidence from UNIX systems."
   --Brian Carrier, digital forensics researcher, and author of File System Forensic Analysis

The Definitive Guide to Computer Forensics: Theory and Hands-On Practice

Computer forensics--the art and science of gathering and analyzing digital evidence, reconstructing data and attacks, and tracking perpetrators--is becoming ever more important as IT and law enforcement professionals face an epidemic in computer crime. In Forensic Discovery, two internationally recognized experts present a thorough and realistic guide to the subject.

Dan Farmer and Wietse Venema cover both theory and hands-on practice, introducing a powerful approach that can often recover evidence considered lost forever.

The authors draw on their extensive firsthand experience to cover everything from file systems, to memory and kernel hacks, to malware. They expose a wide variety of computer forensics myths that often stand in the way of success. Readers will find extensive examples from Solaris, FreeBSD, Linux, and Microsoft Windows, as well as practical guidance for writing one's own forensic tools. The authors are singularly well-qualified to write this book: They personally created some of the most popular security tools ever written, from the legendary SATAN network scanner to the powerful Coroner's Toolkit for analyzing UNIX break-ins.

After reading this book you will be able to

  • Understand essential forensics concepts: volatility, layering, and trust
  • Gather the maximum amount of reliable evidence from a running system
  • Recover partially destroyed information--and make sense of it
  • Timeline your system: understand what really happened when
  • Uncover secret changes to everything from system utilities to kernel modules
  • Avoid cover-ups and evidence traps set by intruders
  • Identify the digital footprints associated with suspicious activity
  • Understand file systems from a forensic analyst's point of view
  • Analyze malware--without giving it a chance to escape
  • Capture and examine the contents of main memory on running systems
  • Walk through the unraveling of an intrusion, one step at a time

The book's companion Web site contains complete source and binary code for open source software discussed in the book, plus additional computer forensics case studies and resource links.



Sample Content

Downloadable Sample Chapter

Download the Sample Chapter related to this title.

Table of Contents

Preface.

About the Authors.

I. BASIC CONCEPTS.

1. The Spirit of Forensic Discovery.

    Introduction.

    Unusual Activity Stands Out.

    The Order of Volatility (OOV).

    Layers and Illusions.

    The Trustworthiness of Information.

    The Fossilization of Deleted Information.

    Archaeology vs. Geology.

2. Time Machines.

    Introduction.

    The First Signs of Trouble.

    What's Up, MAC? An Introduction to MACtimes.

    Limitations of MACtimes.

    Argus: Shedding Additional Light on the Situation.

    Panning for Gold: Looking for Time in Unusual Places.

    DNS and Time.

    Journaling File Systems and MACtimes.

    The Foibles of Time.

    Conclusion.

II. EXPLORING SYSTEM ABSTRACTIONS.

3. File System Basics.

    Introduction.

    An Alphabet Soup of File Systems.

    UNIX File Organization.

    UNIX File Names.

    UNIX Pathnames.

    UNIX File Types.

    A First Look Under the Hood: File System Internals.

    UNIX File System Layout.

    I've Got You Under My Skin: Delving into the File System.

    The Twilight Zone, or Dangers Below the File System Interface.

    Conclusion.

4. File System Analysis.

    Introduction.

    First Contact.

    Preparing the Victim's File System for Analysis.

    Capturing the Victim's File System Information.

    Sending a Disk Image Across the Network.

    Mounting Disk Images on an Analysis Machine.

    Existing File MACtimes.

    Detailed Analysis of Existing Files.

    Wrapping Up the Existing File Analysis.

    Intermezzo: What Happens When a File Is Deleted?

    Deleted File MACtimes.

    Detailed Analysis of Deleted Files.

    Exposing Out-of-Place Files by Their Inode Number.

    Tracing a Deleted File Back to Its Original Location.

    Tracing a Deleted File Back by Its Inode Number.

    Another Lost Son Comes Back Home.

    Loss of Innocence.

    Conclusion.

5. Systems and Subversion.

    Introduction.

    The Standard Computer System Architecture.

    The UNIX System Life Cycle, from Start-up to Shutdown.

    Case Study: System Start-up Complexity.

    Kernel Configuration Mechanisms.

    Protecting Forensic Information with Kernel Security Levels.

    Typical Process and System Status Tools.

    How Process and System Status Tools Work.

    Limitations of Process and System Status Tools.

    Subversion with Rootkit Software.

    Command-Level Subversion.

    Command-Level Evasion and Detection.

    Library-Level Subversion.

    Kernel-Level Subversion.

    Kernel Rootkit Installation.

    Kernel Rootkit Operation.

    Kernel Rootkit Detection and Evasion.

    Conclusion.

6. Malware Analysis Basics.

    Introduction.

    The Dangers of Dynamic Program Analysis.

    Program Confinement with Hard Virtual Machines.

    Program Confinement with Soft Virtual Machines.

    The Dangers of Confinement with Soft Virtual Machines.

    Program Confinement with Jails and chroot().

    Dynamic Analysis with System-Call Monitors.

    Program Confinement with System-Call Censors.

    Program Confinement with System-Call Spoofing.

    The Dangers of Confinement with System Calls.

    Dynamic Analysis with Library-Call Monitors.

    Program Confinement with Library Calls.

    The Dangers of Confinement with Library Calls.

    Dynamic Analysis at the Machine-Instruction Level.

    Static Analysis and Reverse Engineering.

    Small Programs Can Have Many Problems.

    Malware Analysis Countermeasures.

    Conclusion.

III. BEYOND THE ABSTRACTIONS.

7. The Persistence of Deleted File Information.

    Introduction.

    Examples of Deleted Information Persistence.

    Measuring the Persistence of Deleted File Contents.

    Measuring the Persistence of Deleted File MACtimes.

    The Brute-Force Persistence of Deleted File MACtimes.

    The Long-Term Persistence of Deleted File MACtimes.

    The Impact of User Activity on Deleted File MACtimes.

    The Trustworthiness of Deleted File Information.

    Why Deleted File Information Can Survive Intact.

    Conclusion.

8. Beyond Processes.

    Introduction.

    The Basics of Virtual Memory.

    The Basics of Memory Pages.

    Files and Memory Pages.

    Anonymous Memory Pages.

    Capturing Memory.

    The savecore Command.

    Static Analysis: Recognizing Memory from Files.

    Recovering Encrypted File Contents Without Keys.

    File System Blocks vs. Memory Page Technique.

    Recognizing Files in Memory.

    Dynamic Analysis: The Persistence of Data in Memory.

    File Persistence in Memory.

    The Persistence of Nonfile, or Anonymous, Data.

    Swap Persistence.

    The Persistence of Memory Through the Boot Process.

    The Trustworthiness and Tenacity of Memory Data.

    Conclusion.

Appendix A. The Coroner's Toolkit and Related Software.

    Introduction.

    Data Gathering with grave-robber.

    Time Analysis with mactime.

    File Reconstruction with lazarus.

    Low-Level File System Utilities.

    Low-Level Memory Utilities.

Appendix B. Data Gathering and the Order of Volatility.

    Introduction.

    The Basics of Volatility.

    The State of the Art.

    How to Freeze a Computer.

    Conclusion.

References.

Index.

Preface

Untitled Document

Today, only minutes pass between plugging in to the Internet and being attacked by some other machine--and that's only the background noise level of nontargeted attacks. There was a time when a computer could tick away year after year without coming under attack. For examples of Internet background radiation studies, see CAIDA 2003, Cymru 2004, or IMS 2004.

With this book, we summarize experiences in post-mortem intrusion analysis that we accumulated over a decade. During this period, the Internet grew explosively, from less than a hundred thousand connected hosts to more than a hundred million (ISC 2004). This increase in the number of connected hosts led to an even more dramatic--if less surprising--increase in the frequency of computer and network intrusions. As the network changed character and scope, so did the character and scope of the intrusions that we faced. We're pleased to share some of these learning opportunities with our readers.

In that same decade, however, little changed in the way that computer systems handle information. In fact, we feel that it is safe to claim that computer systems haven't changed fundamentally in the last 35 years--the entire lifetime of the Internet and of many operating systems that are in use today, including Linux, Windows, and many others. Although our observations are derived from today's systems, we optimistically expect that at least some of our insights will remain valid for another decade.

What You Can Expect to Learn from This Book

The premise of the book is that forensic information can be found everywhere you look. With this guiding principle in mind, we develop tools to collect information from obvious and not-so-obvious sources, we walk through analyses of real intrusions in detail, and we discuss the limitations of our approach.

Although we illustrate our approach with particular forensic tools in specific system environments, we do not provide cookbooks for how to use those tools, nor do we offer checklists for step-by-step investigation. Instead, we present a background on how information persists, how information about past events may be recovered, and how the trustworthiness of that information may be affected by deliberate or accidental processes.

In our case studies and examples, we deviate from traditional computer forensics and head toward the study of system dynamics. Volatility and the persistence of file systems and memory are pervasive topics in our book. And while the majority of our examples are from Solaris, FreeBSD, and Linux systems, Microsoft's Windows shows up on occasion as well. Our emphasis is on the underlying principles that these systems have in common: we look for inherent properties of computer systems, rather than accidental differences or superficial features.

Our global themes are problem solving, analysis, and discovery, with a focus on reconstruction of past events. This approach may help you to discover why events transpired, but that is generally outside the scope of this work. Knowing what happened will leave you better prepared the next time something bad is about to occur, even when that knowledge is not sufficient to prevent future problems. We should note up front, however, that we do not cover the detection or prevention of intrusions. We do show that traces from one intrusion can lead to the discovery of other intrusions, and we point out how forensic information may be affected by system-protection mechanisms, and by the failures of those mechanisms.

Our Intended Audience

We wrote this book for readers who want to deepen their understanding of how computer systems work, as well as for those who are likely to become involved with the technical aspects of computer intrusion or system analysis. System administrators, incident responders, other computer security professionals, and forensic analysts will benefit from reading this book, but so will anyone who is concerned about the impact of computer forensics on privacy.

Although we have worked hard to make the material accessible to nonexpert readers, we definitely do not target the novice computer user. As a minimal requirement, we assume strong familiarity with the basic concepts of UNIX or Windows file systems, networking, and processes.

Organization of This Book

The book has three parts: we present foundations first, proceed with analysis of processes, systems, and files, and end the book with discovery. We do not expect you to read everything in the order presented. Nevertheless, we suggest that you start with the first chapter, as it introduces all the major themes that return throughout the book.

In Part I, "Basic Concepts," we introduce general high-level ideas, as well as basic techniques that we rely on in later chapters.

  • Chapter 1, "The Spirit of Forensic Discovery," shows how general properties of computer architecture can impact post-mortem analysis. Many of the limitations and surprises that we encounter later in the book can already be anticipated by reading this chapter.
  • Chapter 2, "Time Machines," introduces the concept of timelining, using examples of host-based and network-based information, including information from the domain name system. We look at an intrusion that stretches out over an entire year, and we show examples of finding time information in non-obvious places.

In Part II, "Exploring System Abstractions," we delve into the abstractions of file systems, processes, and operating systems. The focus of these chapters is on analysis: making sense of information found on a computer system and judging the trustworthiness of our findings.

  • Chapter 3, "File System Basics," introduces fundamental file system concepts, as well as forensic tools and techniques that we will use in subsequent chapters.
  • Chapter 4, "File System Analysis," unravels an intrusion by examining the file system of a compromised machine in detail. We look at both existing files and deleted information. As in Chapter 2, we use correlation to connect different observations, and to determine their consistency.
  • Chapter 5, "Systems and Subversion," is about the environment in which user processes and operating systems execute. We look at subversion of observations, ranging from straightforward changes to system utilities to almost undetectable malicious kernel modules, and detection of such subversion.
  • Chapter 6, "Malware Analysis Basics," presents techniques to discover the purpose of a process or a program file that was left behind after an intrusion. We also discuss safeguards to prevent malware from escaping, and their limitations.

In Part III, "Beyond the Abstractions," we look beyond the constraints of the file, process, and operating system abstractions. The focus of this part is on discovery, as we study the effects of system architecture on the decay of information.

  • Chapter 7, "The Persistence of Deleted File Information," shows that large amounts of deleted file information can survive intact for extended periods. We find half-lives on the order of two to four weeks on actively used file systems.
  • Chapter 8, "Beyond Processes," shows examples of persistence of information in main memory, including the decrypted contents of encrypted files. We find large variations in persistence, and we correlate these variations to operating system architecture properties.

The appendices present background material: Appendix A is an introduction to the Coroner's Toolkit and related software. Appendix B presents our current insights with respect to the order of volatility and its ramifications when capturing forensic information from a computer system.

Conventions Used in This Book

In the examples, we use constant-width font for program code, command names, and command input/output. User input is shown in bold constant-width font. We use $ as the shell command prompt for unprivileged users, and we reserve # for super-user shells. Capitalized names, such as Argus, are used when we write about a system instead of individual commands.

Whenever we write "UNIX," we implicitly refer to Solaris, FreeBSD, and Linux. In some examples we include the operating system name in the command prompt. For example, we use solaris$ to indicate that an example is specific to Solaris systems.

As hinted at earlier, many examples in this book are taken from real-life intrusions. To protect privacy, we anonymize information about systems that are not our own. For example, we replace real network addresses with private network addresses such as 10.0.0.1 or 192.168.0.1, and we replace host names or user names. Where appropriate, we even replace the time and time zone.

Web Sites

The examples in this book feature several small programs that were written for the purpose of discovery and analysis. Often we were unable to include the entire code listing because the additional detail would only detract from the purpose of the book. The complete source code for these and other programs is made available online at these Web sites:

http://www.fish.com/forensics/
http://www.porcupine.org/forensics/

On the same Web sites, you will also find bonus material, such as case studies that were not included in the book and pointers to other resources.

Index

Download the Index file related to this title.

Updates

Submit Errata

More Information

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020