Using Data Tools With Forensically Extracted Data

Using Data Tools With Forensically Extracted Data

Getting entreprise forensic data into snapshot on examiner machine

To demonstrate concepts and problems we were using two products in a set of scenarios. As principal enterprise level forensic tool we used EnCase Enterprise with some add-ons and as principal data analytics tools we used InfoZoom. As EnCase Enterprise is currently in transition from version 6 to 7, only results for EnCase Enterprise 7 is presented.


The first test was done some time ago with EnCase version 6, where enterprise snapshots were optionally stored in MSSQL database with mandatory storage in L01 (logical evidence file) files. Accessing MSSQL database and analyzing data from InfoZoom was simple and without any problems, unfortunately this functionality is no longer available in EnCase Enterprise 7. EnCase Enterprise 7 stores snapshot data in L01 format and in SQLite modified database, this complicates data access from InfoZoom tool.


For example we have additional analyses of snapshots for a set of end nodes done through the Sweep Enterprise functionality.


Sweep Enterprise is a built-in functionality that enables forensics examiner to collect data from end node machines where forensics servlet is installed. Detailed description of this process and functionality is available in the EnCase documentation.


In the basic steps, the forensics examiner defines a set of end nodes that will be examined by the Sweep Enterprise script, and data sets that will be collected from end nodes (Pictures 1 & 2).

Picture 1: Defining Target End Nodes for Sweep Enterprise

Picture 2: Selection of data to be collected from End Nodes

For each end node the data is stored in a folder named by Sweep Enterprise timestamp „Sweep Enterprise Collection 2012_12_30 02_27_01“ where each target node has its own LE01 file with stored data, for example „Machine - DAMIRDDELL.L01“ store snapshot data for DAMIRDELL. Same data is also stored in the SQLite file that contains all collected snapshot data.

Picture 3: Snapshot data in EnCase Enterprise console

After automated analyses results are stored in cache data files and available for additional processing, there is no uniform data format in data cache structures as it can vary from SQLite and L01 formats to plain text files. Data was collected from end nodes about the operating system, hardware installed, software installed and users on the system, network shares and USB devices. This data also includes processes and DLL information, open ports and open files, ARP, DNS and routing tables, averaging about 10MB L01 file per machine.

Since SQLite database has a modified format, direct access from InfoZoom was not possible, same problem was for ODBC access attempt, but later we managed to get ODBC access to SQLite..

Picture 4: USB devices data in EnCase Enterprise console

Data from Encase Internals to formats and files accessible by Infozoom

Fortunately data from logical evidence files are viewed in a limited form, a subset of the actual information that was collected, through the EnCase Enterprise console (Pictures 3 & 4). Since there is no direct access to this data from InfoZoom it is necessary to export data into another format that InfoZoom can read. There are a few possibilities doing this task without writing dedicated programs, the fastest method is creating a review package from the EnCase console view and exporting it again into XLSX format from a web browser, then importing this into InfoZoom (Picture 5). EnCase review package is readable in a web browser format that can be exported into XLSX, using MS Internet Explorer. Other export formats are also available, but the whole process is not yet well documented.

 

Picture 5: Conversion from review form into Excel data

Since excel data is easily imported into InfoZoom it is easy to do further comparison and selection. USB devices are organized in an overview mode (Picture 6), where we make conclusions about USB devices that were attached to end nodes before the Sweep Enterprise process.

Picture 6: Data from USB Devices imported into InfoZoom

Same methods can be used for other snapshot data. In The processes from end nodes are presented in the overview mode (Picture 7). Since there are more than 500 processes, the view does not reveal patterns, additional analysis is required in order to check for interesting patterns.

 

Picture 7: Process data imported into InfoZoom

Results from InfoZoom analyses can only be added manually into EnCase Consoles for further forensics analysis or forensics data fetch. Obviously it is a slow process if large amounts of data need to be exported and analysed out of EnCase, also it slows down the EnCase examiner station where exports and imports are done.

Accessing from Infozoom directly into SQLite

With SQLite3 0.99.00.00 ODBC driver for Win64 it is possible to access SQLite files in EnCase case folder structure and conncet to db data. This works in EnCase v7.06 and higher. Sweep.sqlite file contains all sweep data which are also stored in L01 files and in named folders too (Picture 8). It is important to connect to sweep.sqlite file through an odbc connection. Unfortunately, a new connection has to be created for each file which means means that each EnCase case has to get one dedicated connection.

Picture 8: Encase Enterprise sweep data in case folder structure

Infozoom easily gets structure of data from the db file, it is a complex structure presenting sweep data collected from remote nodes. There are plenty of tables where data is stored in Encase friendly format, so some decoding and recoding may need to be done by hand since this is undocumented in EnCase user documentation. Data tables stored in db files are visible in Picture 9

.

Picture 9: SQLite tables presenting sweep data

By using Infozoom features it is possible to create links and new attributes which present views into the data that we are interested in. It is a lot of work, but can be reused later on. Since it is read only data, this access does not change the original data and since it is directly into db file it is much faster than the previous method without much load on the examiner machine. In theory it is possible to connect to SQLite from remote machines but sqlite wiki discourage such efforts http://www.sqlite.org/cvstrac/wiki?p=SqliteNetwork), so we have not attempted this in this case.


Even Though Infozoom is independent tool which provides access to other security tools’ internal files and reports, it sometimes also requires a lot of work in order to get a sensible view of the data and to correlate the relevant data from various sources.


As an example, a process svchost.exe is drilled down in sweep database in the data table containing all the process data, with references to other tables. This data can then be related to snapshot and host information to show where and when snapshot was taken.

Picture 10: SQLite tables HasProcessList

Since infozoom allows creation of complex attributes it is possible, with some reverse engineering, to create views where all relevant data is visible. Currently this poses a challenge because the table structure in sweep database is not documented.


Even though Infozoom can be replaced with other data exploration tools, it is still the very best in a scenarios where we expect to go into more than one data source, i.e. having sweep database from several unrelated sweeps and access to SIEM or log storage. To fully explore such scenarios it is necessary to use parallel access to an FTK and EnCase data at same time with also access to other security tools, but this is for another project.

Conclusion ?

We can say that any additional ability to analyze data collected through forensic tools poses a great challenge. But when it is possible it is a great advantage.


There are drawbacks and problems that cannot be easily solved. Due to the rapid development cycles with currently available tools, it is hard to maintain an efficient connection between forensics tools and data analytics tools. Often data can be reliably transferred, only in simplest forms and through help of some “mediator” tool, as demonstrated in the example with html export and excel data aggregation into InfoZoom database.


Also it complicated to transfer data between various forensics products, due to strong vendors’ competition, which makes it very problematic especially in mobile forensics. Forensics of Mobile Devices has become a more and more important field, yet it still lacks enterprise class forensic tools, so additional data analytics is even more complicated than in standard networks and enterprise forensics. Fortunately things are changing recently XRY and UFED have introduced tools which do some essential data analytics.


It is worth mentioning, to forensic tools’ developers/vendors, that it can be beneficial for them to build in, desperately needed, data analytics functionality in future generations of enterprise oriented forensic tools, even though it might still be limited to only the forensics view of data.