Sunday, August 30, 2015

Continuous training

Somehow I get tangled into continuous training session starting from 12th till 31st  August, spanning over weekends too and being spread from Middle East to back home.  Actually a horror to get trough
because it drains me down completely and re-position my beloved vacation. It also includes about month of preparation before.

For me the most interesting part was Linux forensic intro and preparation for it, this is  training which regrettably goes very seldom so I have to do humble, detailed preparation, check all examples and include recent developments.
Training was fun but very short on time, just 3 days planned but in reality even shorter,  Also it was exhausting for me as trainer and for students too.  There was huge amount of command line actions, what is so uncommon in today computing.
When we are talking about Linux and forensics it is about to cover:

  1. Student mindset about using computer
  2. Linux as working platform, from user view, over sysadmin view to system programmer view (and system programmer view is what is actually what forensic view is about)
  3. Linux as forensic workstation, tools and concepts 
  4. Linux as forensic target 
So it is huge task even just to mention things ...

Rest of the training is small Encase Forensic 1 and Forensic 2 class, almost private tutoring, it is also a challenge with ever changing Encase v7  you never know what will happen if you don't check all examples in exact combination of windows, patches, security software and Encase. 
As for annoying examples in v7.10.5 suddenly VDE refused to work, but it was still OK on v7.10.4 so you never know what will be exact situation, if you can not have a sterile dedicate classroom.







EnCase v7 and node sweep for hash suggestions

Just been trying to do node sweep for dll and process hash, to do some presence control it turns out in v7 you can't do normal check for list of hashes without enscript programming. Very frustrating since in v6 it was possible at least to find end nodes with required hash trough conditions.

In version 7 there are no conditions in sweep snapshot results also in case analyzer part restraints are so dumb organised that you can't do a list of hashes but one by one trough immensely dumb interface. It is frustrating since user interface is lame worst of it data are there but you can't get to it.. kinky and masohiticly perverted. Before I've been able to access data from infozoom but test licence is long gone so we are back on Enacse functionality.  Other patch was to print hash in console and do unix fgrep on the console output, it worked but is was awkward also.

Since I mentioned enscripting to solve my issue, examples which are provided in manuals are not complete. It is easy to do a sweep, and get dll and process hashes, but what is missing completely is how to compare it with existing hash sets in hash library.That will be  the elegant solution,  to create a has set add it into has lib and that find if collected process hash or dll hash belongs to set ... if you can find example :) :)

Also the idea of enscript programming is very outdated in sense of what kind of programming language enscript is, It is so low level access to encase internals, like using C code to do ls and cd command each time. Putting a "script" into name is also a bit perverted,  it is en-script .. encase scripting language, but as far from concept of scripting as can be. To be efficient and useful powerful simple scripting command is needed not C++/java language concepts.  Instead of endless low level object and classes meddling simple high level construct should be used.

Something like this, it is easy to read even for someone who does not know programming, if he is familiar with enacse enterprise concepts

do sweep on node1, node2 use systeminfo, snapshot
     when node in sweep
           if node process hash is in hashlib bookmark node, process, hash
           if node dll hash is in hashlib bookmark node dll hash

or whatever else familiar syntax , I've put a bit of this thoughts about standardization and basic case handling testing, comparing results in other earlier posts and forensic language issues.

Modern languages python, ruby, even perl or forgotten tclsh are powerful languages easy to extend and use,  almost platform independent. Python is the star at the moment so why not python .. don't know but there is a lot of work, oddly to communicate with python tools outside of encase and collect results, what is again a bit against the idea since scritping language is mentioned to be a glue layer among other modules exactly the opposite :) as it is used here.

Ages ago in late 1990s during my phd I was doing almost the same concept to merge two different tools, snmp and fuzzy data expert to interpret data collected from snmp. It was before python and shell was simple tcl functional extension able to code fuzzy rules into code as simple human readable statements. It can be done even for much more fuzzy concepts than forensics

When we are mentioning using programming in digital forensics it is very good idea to study in detail concepts and implementations of coroner toolkit and sleuthkit, since it depends on known and forensically sound system level library's and system calls,  If we are talking of using scripting language in digital forensics some checking on implementations on different platforms should be done or even better, creating a new forensically sound libraries or classes.  It is illustrated in earlier article about python scripting "Python training for forensics"  where some issues in ordinary IO mechanism is mentioned, which if we are strictly speaking about forensic (in sense of its rules and requests) should be done in basic IO level.  Almost similar discussion exists in V.Wenema book about digital forensics taking great care about cache issues on unix file system.






Friday, August 7, 2015

TD3 and linux

While I was preparing myself for linux training I've decided to connect TD3 over iscsi to SIFT 3.0 workstation as example. It takes about few minutes and some man page reading. Easy and simple and works.

Sans SIFT 3.0 workstation is Ubuntu based and open-iscsi package is not installed, so first thing to do is to install package. On Ubuntu wiki there is enough technical details.

> sudo apt‐get install open‐iscsi

That will add and configure iscsi subsystem on Sift, on TD3 iscsi should be enabled and restarted few times to get it running. It's common situation with TD3 and iscsi connection. Don't forget to take IP address of td3.

To do dummy check it is good idea to ping TD3 and see if we can reach it

>ping TD3IP

On Sift iscsi discovery will show if  we can reach isici targets

>sudo iscsiadm ‐m discovery ‐t st ‐p TD3IP

1. -m: determines the mode that iscsiadm executes in.
2. -t: specifies the type of discovery.
3. -p: option indicates the target IP address, in our case TD3

After that  we can login into TD3 and access sicis target

>  sudo iscsiadm ‐m node ‐‐login

If all is ok, message about success will be display and new disk is visible
dmesg command should who it also

>dmesg | grep sd

so we will have new local disk  on Sift, ready for use, already read only exported from TD3
any standard command should work, even mount

if we need to remove disk and its file systems we have tpo do umount first and than disconnect the disk session.

>sudo iscsiadm ‐m node ‐u 

Sunday, July 19, 2015

Exam to attend

This post I've started while just waiting for written exam to start at Algebra. Students will be here soon so, questions are easy so it shouldn't be any problems there. It was few days ago and left as another almost forgotten draft.

My lectures this year have not been satisfactory for me, somehow things are missing,
This run we haven't included any practicals with python, I decided to add it as mandatory both with basic linux tools and ideas It will be easy to do since I'm doing some crash 3 day linux forensics training. There is plenty of sources around, but what worries me that students don't know enough about operating systems especially UNIX like OSes to be easily taught into Linux in forensics.




Monday, July 6, 2015

Some EnCase v7 snapshot remarks

Last week I tried to prepare some dll/process hash control for remote nodes, idea was to list dll and proces hash trough snapshot and see if certain hashes are there.
I've done that plenty of times with v6 and I'll do it with v6 even now but my v6 licence is not valid any more, so have to do it in v7, it should be almost same ...
In fact it is not and it is a bit of nightmare ....

In theory after sweep all processes and dll are listed with its hash, so it should be easy to use condition or filter to  point to node name where hash is found. As for precaution since there is a hash set field in sweep view I've created hash sets from hash list and added that new has set to hash library. Fot this I've used Lance Mueler script with detailed instruitions.

Up to here all works fine, than surprises started. In "sweep analysis" there is no hash set info, you can check for hash value but "restriction" wizard is so awkwardly done that you have to add one by one hash value, it is nonsense since it take ages to load any set of hashes into ...
maybe it can be edited in source or by enscript but there is nothing in documentation ..
zero for that ... one essential functionality crippled but implementation


So what to try next ?

All data is also stored in L01 file related to sweep so it can be seen and analyzed trough Encase in theory. If you load that L01 file, in its record view you'll notice that hash value for process or dll is there but a hash set filed is empty ... god knows why. Form documentation I've got idea that I'll have info in that hahs set field if dll hash in in one of the sets in case library .. but nothing, probably a bug.

Also by some strange ideas in v7 there are no filters and no conditions for records view so you can't do search for values as in old v6. Again some very essential functionality is disabled and whole process crippled by inadequate interface.  Almost as someone was trying to sabotage product, In such form v7 is practically unusable without heavy enscripting, To be worse a lot of examples and explanation in documentation is missing too.

In despair I asked a few questions around but not much use of answers, just to check examples.
I've located the example from enscript programming for sweep dll and modified it minimally to node name, hash value and do same for present processes also. There was also one bitter disappoint,  dll object and process object has hash and has set info included but it is changed in v7 and changes are undocumented so nothing for elegant solution too.

At the end I decided to use simple fgrep from cygwin to find lines in console file which match hash
since the sweep enscript puts output into console.  iconv command was used to convert utf-16 console text files into utf-8 encoding which fgrep handles perfectly.

Here is code for small cygwin bash script
-------------------------------------------------------------------------------------------------------------------
#!/usr/bin/bash
#############################################
#fast and dirty for finding something in encase log files
#since a lot of things in encase v7 sweep does not work or it is not
#doumented
#based on cygwin fgrep utility
#Usage: uhh md5
#takes md5 file name as argument
################################################

#folder with encase logs for user
ENCASE="/cygdrive/c/Users/$USERNAME/Documents/EnCase/logs"

#there are Console[0-9].txt files
#in utf-16, so stardard grep breaks, solution is to use iconv
#iconv -f utf-16 -t utf-8
##################################################

#if no patern file exit 1
#patern file, plain md5 signitures

MD5="$1"
test -f "$MD5" || exit 1

#strange systax to avoid unix-dos path name troubles
for f in $(ls "$ENCASE")
do
echo $f
iconv -f utf-16 -t utf-8 "$ENCASE/$f"| fgrep -i -f "$MD5"
done
---------------------------------------------------------------------------------------------------------------

There are other ways to do similar thing, more elegant  but I was simply to tired to experiment more  

I forget to add modified example from Enscript manual, here it is


/*-------------------------------------------
DllListClass represents a list of all loaded libraries on a remote node. In order to retreive a list of DLLs, a valid
snapshot object is required. A snapshot object can be created in the following ways:

(1) BatchClass::GetConnection method
(2) ConnectionClass::ReadSnapshot method

Note that the ConnectionClass::SNAPDLL must be used when snapshot objects are created. Also, if ConnectionClass::SNAPHASH
is used, each DLL will have its hash value calculated.

See Also:
SnapshotClass
DllListClas
NodeClass


2015 added extension for hash value and
same for process class 

*/

/*
Example: Connect to a remote node and print out a list of all the DLLs loaded in memory.
*/
class MainClass {

  SafeClass Safe; //object to connect to SAFE
  RoleClass RoleRoot, //list of all roles for a given user
            Role; //role user choose to take
  NetworkClass SweepNet; //list of remote nodes to connect to
  String NetText, //textual list of remote nodes
         ClientReturnAddress, //for NODECLIENT connection options
         StatusBarName; //name to be displayed in the status bar
  int NumConnections, //number of SAFE connections to use
     ConnectOptions; // Connection Options: INDIRECT, CLIENTNODELOCAL, CLIENTNODESAFE, NODECLIENT


  MainClass() :
    Safe(),
    RoleRoot(),
    Role(),
    SweepNet(),
    NumConnections = 1,
    ConnectOptions = ConnectionClass::CLIENTNODESAFE,
    StatusBarName = "Example - Getting DLL Data"
  {
  }

  /**
  Entry point of the Enscript
  **/
  void Main(CaseClass c) {
    if (c) {
      SystemClass::ClearConsole();
      if (Safe.Logon(null) && ShowDiag() == SystemClass::OK) {
        Sweep();
        SystemClass::Message(0, "Success", String::Format("{0}: Completed Successfully!", StatusBarName));
      }
    }
    else
      SystemClass::Message(0, "Error", "Need an open case so that results of registry queries can be added!");
  }

  /**
  This method contains the logic we want to apply to each node on the network
  **/
  void Process(SnapshotClass snap) {
    Console.WriteLine("Processing Machine " + snap.Name());
    //in order for the DLL list to not be empty, the BatchClass
    //must have been constructed with the ConnectionClass::SNAPDLL or
    //ConnectionClass::SNAPHIDDEN


    //entrpy is on -1 how to do enteryp in sweep ?
    //also how to get hash in set
    Console.WriteLine("DLL Count = {0}", snap.DllListRoot().Count());
    forall (DllListClass p in snap.DllListRoot()) {
             //add to print hash value for dll and node name
           Console.WriteLine("DLL loaded: {0} {1} {2}", snap.Name(),p.Name(),p.HashValue() );
    }
    /* add proceses too */
    Console.WriteLine("Process Count = {0}", snap.ProcessRoot().Count());
    forall (ProcessClass p in snap.ProcessRoot()) {
       //add to print hash value for process  and node name
        Console.WriteLine("Process loaded: {0} {1} {2}", snap.Name(), p.Name(),p.HashValue() );
    }

  }

  /**
   Display dialogs
  **/
  int ShowDiag() {
    RoleRoot = Safe.RoleRoot();
    DialogClass diag();
    new NetTextDialogClass(diag, this);
    return diag.Wizard();
  }


  /**
    Code that gets connection and snapshot
  **/
  void ReadNetwork(BatchClass batch, SnapshotClass root) {
    String message,
           name;
    DateClass d();
    do {
      ConnectionClass conn;
      SnapshotClass ss(null);
      message = "";
      BatchClass::ConnectionTypes reply = batch.GetConnection(conn, ss, name, message, 0);
      if (reply == BatchClass::BATCHCONNECT) { //successfully connected to remote node
        Process(ss);
        SystemClass::StatusInc(1);
        root.Insert(ss);
      }
      else if (reply == BatchClass::BATCHERROR) { //could not connect to remote node. ss object will have the state of the node
        d.Now();
        Console.WriteLine("Could Not Connect To {0} SAFE Error Message: {1}", name, message);
        SystemClass::StatusInc(1);
        root.Insert(ss);
      }
      else if (reply == BatchClass::BATCHWAIT)
        SystemClass::Sleep(100);
      else if (reply == BatchClass::BATCHFATAL) {
        String err = SystemClass::LastError();
        Console.WriteLine("The SAFE is not responding: {0}. This Enscript will terminate.", err);
        return;
      }
    } while (reply != BatchClass::BATCHDONE);
  }

  /** Code that creates a batchclass
  **/
  void Sweep() {
    DateClass now;
    SnapshotClass newSnaps = new SnapshotClass(null, "Snapshot");
    BatchClass batch(Safe, Role, NumConnections, ConnectionClass::SNAPALL);
    if (batch.Add(SweepNet)) {
      batch.SetMode(ConnectionClass::Options::Convert(ConnectOptions), ClientReturnAddress);
      if (batch.Start()) {
        uint machines = batch.TotalMachines();
        Console.WriteLine("Scanning {0} using {1}", Plural("node", machines), Plural("connection", batch.ConnectionsUsed()));
        SystemClass::StatusRange(StatusBarName, machines);
        uint start;
        now.Now();
        start = now.GetUnix();
        ReadNetwork(batch, newSnaps);
        now.Now();
        Console.WriteLine("Scan completed in {0} seconds", (now.GetUnix() - start));
      }
      else {
        SystemClass::Message(0, "BatchClass error", SystemClass::LastError());
      }
    }
    else {
      SystemClass::Message(0, "BatchClass Error", "Unable to add any IPs to the sweep");
    }
  }

  String Plural(const String &str, uint n) {
    return String::Format("{0} {1}{2}", n, str, n == 1 ? "" : "s");
  }

  /**
   Turn a string of text into networkclass objects
  **/
  bool ParseText(String t) {
    SweepNet.Close();
    bool ret = false;
    while (t) {
      ret = true;
      int    end  = t.Find("\n");
      String line = end < 0 ? t : t.SubString(0, end);
      int    dash = line.Find("-");
      if (dash >= 0) {
        IPClass ip1(ExtractIP(line.SubString(0, dash))),
                ip2(ExtractIP(line.SubString(dash+1, -1)));
        if (ip1 && ip2) {
          NetworkClass n(SweepNet, "IP Range", NodeClass::SELECTED);
          n.SetStart(ip1);
          n.SetStop(ip2);
        }
        else
          NetworkClass n(SweepNet, line, NodeClass::SELECTED);
      }
      else if (line != "")  {
        NetworkClass n(SweepNet, line, NodeClass::SELECTED);
      }
      if (end >= 0)
        t.Delete(0, end+1);
      else
        break;
    }
    return ret;
  }

  /**
   Check for IPs in nettext
  **/
  String ExtractIP(const String &s) {
    String ret = s;
    ret.Trim(" ", String::TRIMSTART | String::TRIMEND);
    return ret.IsValidIPAddress() ? ret : "";
  }
}

/**
 Dialog to choose a role and enter nodes to sweep
**/
class NetTextDialogClass: DialogClass {

  MainClass Data;
  StaticTextClass SafeTextEdit;
  TreeEditClass Tree;
  StaticTextClass Help;
  StringEditClass NetTextEdit;

  NetTextDialogClass(DialogClass diag, MainClass d) :
    DialogClass(diag, String::Format("{0} Options", d.StatusBarName)),
    Data = d,
    SafeTextEdit(this, "", START, 15, 200, 100, 0),
    Tree(this, "Choose The Role You Want To Assume", NEXT, START, 200, 100, 0, d.RoleRoot, 0),
    Help(this, "Enter IP addresses or machine names on separate\n"
                 "lines. Enter ranges on separate lines and delimit\n"
                 "the start and stop address with a dash (\"-\").\n\n"
                 "Example:\n\n"
                 "\tlocalhost\n"
                 "\t192.168.5.5\n"
                 "\t192.168.0.16-192.168.0.64\n"
                 "\t192.168.1.1-192.168.3.255\n"
                 "\tfd00:0:1000:20:0:0:0:100\n",
                 START, NEXT, 200, 100, REQUIRED),
    NetTextEdit(this, "", NEXT, SAME, 200, 100, AUTOVSCROLL | MULTILINE | WANTRETURN, d.NetText, 9999, 0)
  {

  }

  virtual void Setup() {
    DialogClass::Setup();
    SafeTextEdit.SetText("SAFE:\t\t\t\t" + Data.Safe.Name() +
                         "\nUser:\t\t\t\t" + Data.Safe.UserName() +
                          "\n\nTotal Connections:\t\t" + Data.Safe.TotalConnections() +
                          "\nActive Connections:\t\t" + Data.Safe.ActiveConnections() +
                          "\nConnections To Use:\t\t" + Data.NumConnections +
                          "\n\nRemediation Allowed:\t\t" + (Data.Safe.RemediationAllowed() ? "Yes" : "No") +
                          "\nSnapshot Allowed:\t\t" + (Data.Safe.SnapshotAllowed() ? "Yes" : "No") +
                          "\n\nSAFE Version:\t\t\t" + Data.Safe.Version()
                          );
  }

  virtual void CheckControls() {
    DialogClass::CheckControls();
    EnableClose(Tree.GetValue().Parent());
  }

  virtual bool CanClose() {
    Output();
    bool ret = false;
    if (DialogClass::CanClose()) {
      Data.Role = RoleClass::TypeCast(Tree.GetValue());
      ret = Data.ParseText(Data.NetText);
      if (!ret)
        ErrorMessage("Please Enter a value in the IP List Text Area.");
    }
    return ret;
  }
}









Tuesday, June 23, 2015

Summer 2015 coming

It's 15C outside and heavy rain.. wonderful intro into glorious summer. I suppose it will be different summer, more in house and working one.

Since I'm feeling lazy and slow there is a set of unfinished posts here on the blog ...
"Problems with malware"  started 5/9/15
"Classification of digital forensic tools" started 5/1/15
"Tools and users woes" started  5/1/15
"Ransomware and some ideas" started  5/1/15
"Some post Riga conference thoughts" started  6/21/15
"Digital forensics and really big data" started  6/13/15
Hopefully this shameful list will force me to finish it ... to be honest I feel somehow restraint in writing. There are also some personal issues both with glorious 50th birthday coming 

Lecturing at Racunarstvo.hr slowly comes to end just two more lectures to go. I'm not satisfied this time, probably because I was not able to force myself to introduce new things into lecture. Somehow I feel I missed hearts & minds, probably to sleepy minds at lecture time 18:45 till 22:00.  I've introduced some points and discussions from ERA conferece as interesting live points in legal part.  The Bahrain training left some bouncing questions in my mind. It all remembers me on the old Science Fiction from Asimov and Stanislav Lem. The python lectures  was left out this time,  I was thinking to mix it up with practical command line linux issues like in B.J Grundy LinuxLeo guide.

NUIX is coming back to schedule to do preparations, actually to prepare it as part of our portfolio, but I feel I miss hardware for enough power to implement and test solutions.



Thursday, June 11, 2015

EnCase v7 and some indexing woes

EnCase v7 is deeply oriented to indexing as fundamental step in investigation and data analyses, but looks like bad luck which EnCase had in v6 with indexing continues also in v7. It turns out in first few sub-versions of v7 indexing was not implemented correctly and in real life of not much benefit, more horror source.

Things get better and in lastest version 7.9 and 7.10, Now  indexing is working more or less ok, but with still some peculiarities and not easy to digest features. I'll compile in this articles some notes, tips  which maybe can help.

One quite useful thing from indexed data is to get list of keywords, it can be very useful in password cracking and many other purposes. In version 6 it was possible ti dump this data trough enscript, but in version 7 now this is in passware tool interface. Even if you don't have passware kit, dump can be done, it will generate index.words.txt file which contains data from index file. Documentation is missing and no explanation what is what.But still from there it is possible to feed some dictionary attack tools etc, you'll need some regular expression tools to further extract useful data from it.

A huge issue is also lack of examples, per instance there are predefined patterns in indexing for finding email addresses, credit card numbers etc, but no examples.

There is no clue whatsoever how it works at all, Trying combinations you'll get no result or some confusing errors, there is only one small passus in support forum about but no details, results are erratic even in the latest version v7.10.5.

In ordinary indexing search it is possible to use wild  chars and patterns but again it can be quite erratic, I was positive there are some issues with our local language localisation, but never been able to reproduce it in satisfactory way.  Very confusing issue is globbing in indexing search and regexp search in raw search. Since it is possible to combine both of it t in same search,  using two different query syntax is often misleading.