Segmentation fault

Volatilitux : Physical memory analysis of Linux systems

8 décembre 2010 – 2:09

As some of my followers may have seen, I have recently been working on a forensic tool called Volatilitux. It is pretty much the equivalent of the Volatility framework for Linux systems. I presented a pre-release version of this tool at last Hackerzvoice meeting; here are the slides (in French). Today, I am glad to announce the first release of this framework. And to celebrate, here is my first blog post attempt in English .

Background

Volatilitux is a Python framework that helps extracting digital artifacts from raw physical memory (RAM) dumps of Linux systems. The goal of this tool is to offer a tool in a field that cruelly lacks of tools. It is mainly inspired by the SSTIC 2010 challenge whose purpose was to analyze the physical memory of an Android phone. When the challenge got released, the only suitable tool that existed was draugr; however it required two lines to be patched in order to be able to analyze the file.

I tried to solve this challenge but I quickly got stuck at the first step : listing all running processes and extracting the virtual memory map of each one. I learned a lot reading the kernel source code, some blog posts, and finally the challenge solutions when they got released. And I understood the biggest problem of physical memory analysis in Linux: most of kernel structure offsets change depending on the kernel version, configuration and patches. Thus, the first step was to detect all of those offsets. And then, use them to extract and browse all kernel objects.

I first detected those offsets manually by recompiling Android 2.1 kernel (using the NDK), running it into Google’s emulator (provided in the SDK), and then load a LKM in order to display all those offsets. It worked.

Then, I thought: «Since Volatility is a great tool but only supports Windows, why not develop a similar framework that works for Linux RAM dumps?»

I first came up with a version of the tool that required a configuration file containing all the offsets, as well as init_task symbol address and the architecture of the dump. Then, I chose to raise the bar a little higher, and automatically detects those offsets. And here we are…

Features

Here are the main features of the tool:

• Automatic detection of kernel structure offsets
• Manual detection of those offsets using a Loadable Kernel Module
• Supports multiple architectures: ARM, x86, x86 with PAE enabled
• Can be used as a Python module in order to automate tasks and be embedded with other projects

Unlike Volatility, it now only supports a restricted set of commands:

• pslist: print the list of all process
• memmap: print the memory map of a process
• memdmp: dump the addressable memory of a process
• filedmp: dump an open file
• filelist: print the list of all open files for a given process

Of course, since it is a framework, one can extends those features and easily add new architectures, commands, and kernel structures.

It has been tested on physical memory dumps of the folowing Linux distributions:

• Android 2.1
• Fedora 5 and 8
• Debian 5
• CentOS 5
• Ubuntu with and without PAE

Requirements

Volatilitux requires Python 2.6. It only uses Python builtin modules, and does not rely on any external library.

It has mainly been tested on Windows XP, but it should run on any other platform.

Get the source

You can download and browse the source code of this tool on the Volatilitux Google Code project.

To download it directly:

• volatilitux-1.0.zip

Please read the README.txt file, as it contains information on how to use the tool.

Known bug

Yes, Volatilitux already has a bug . One of the kernel offset is not yet properly detected (actually, it is hardcoded due to lack of time!) for Ubuntu distributions. This will make the memmap command display incorrect information in the « Flag » column. However it should not prevent the other commands to run.

Quick Example

Here is how to use Volatilitux to extract the TextViewer Android application in the SSTIC 2010 challenge. Be sure to download the dump and extract « challv2″ to try it .

$ volatilitux.py -f challv2 pslist
Name                PID       PPID
swapper             0         0
init                1         0
kthreadd            2         0
ksoftirqd/0         3         2
events/0            4         2
khelper             5         2
suspend             6         2
kblockd/0           7         2
cqueue              8         2
kseriod             9         2
kmmcd               10        2
pdflush             11        2
pdflush             12        2
kswapd0             13        2
aio/0               14        2
mtdblockd           21        2
hid_compat          22        2
rpciod/0            23        2
mmcqd               24        2
sh                  25        1
servicemanager      26        1
vold                27        1
debuggerd           28        1
rild                29        1
zygote              30        1
mediaserver         31        1
installd            32        1
keystore            33        1
init.goldfish.s     34        1
qemud               35        1
adbd                37        1
qemu-props          44        34
system_server       52        30
putmethod.latin     96        30
m.android.phone     98        30
d.process.acore     101       30
ndroid.settings     116       30
roid.alarmclock     136       30
d.process.media     147       30
com.android.mms     170       30
m.android.email     183       30
com.svox.pico       207       30
nssi.textviewer     227       30
om.anssi.secret     233       30

$ volatilitux.py -f challv2 memmap -p 227
Begin    End       Flags File
00008000-00009000  r-xp  app_process
00009000-0000a000  rwxp  app_process
0000a000-002dc000  rwxp
10000000-10001000  ---p
10001000-10100000  rwxp
40000000-40008000  r-xs  dev/ashmem/system_properties
40008000-40009000  r-xp
40009000-402aa000  rwxp  dev/ashmem/mspace/dalvik-heap/zygote/0
402aa000-41009000  ---p  dev/ashmem/mspace/dalvik-heap/zygote/0
41009000-41038000  r-xs  DroidSans.ttf
41038000-4104c000  rwxp
4104c000-4104d000  ---p  dev/ashmem/dalvik-LinearAlloc
4104d000-412c2000  rwxp  dev/ashmem/dalvik-LinearAlloc
412c2000-4154c000  ---p  dev/ashmem/dalvik-LinearAlloc
4154c000-416d0000  r-xs  core.jar
416d0000-41a5d000  r-xs  system@framework@core.jar@classes.dex
41a5d000-41a91000  rwxp
41a91000-41af6000  r-xs  ext.jar
41af6000-41bee000  r-xs  system@framework@ext.jar@classes.dex
41bee000-41e69000  r-xs  framework.jar
41e69000-4244d000  r-xs  system@framework@framework.jar@classes.dex
4244d000-424cb000  rwxp
424cb000-424de000  r-xs  android.policy.jar
424de000-42507000  r-xs  system@framework@android.policy.jar@classes.dex
42507000-42582000  r-xs  services.jar
42582000-4268d000  r-xs  system@framework@services.jar@classes.dex
4268d000-426b1000  rwxp
426b1000-426e6000  rwxp  dev/ashmem/dalvik-heap-bitmap/objects
426e6000-426f2000  rwxp
426f2000-426f3000  r-xs  dev/ashmem/SurfaceFlinger read-only heap
426f3000-426f8000  r-xs  com.anssi.textviewer.apk
426f8000-426fa000  r-xs  com.anssi.textviewer.apk
426fa000-426ff000  r-xs  com.anssi.textviewer.apk
426ff000-42700000  r-xs  data@app@com.anssi.textviewer.apk@classes.dex
42700000-42701000  rwxs  dev/ashmem/SurfaceFlinger Client control-block
42707000-42738000  rwxp
42738000-42767000  r-xs  DroidSans-Bold.ttf
42778000-427ae000  rwxp  dev/ashmem/dalvik-heap-bitmap/mark/0
427ba000-42c63000  r-xs  framework-res.apk
42c63000-42e14000  r-xs  framework-res.apk
42e14000-42e55000  rwxp  dev/ashmem/mspace/dalvik-heap/zygote/1
42e55000-43b74000  ---p  dev/ashmem/mspace/dalvik-heap/zygote/1
43b74000-43b75000  ---p
43b75000-43c74000  rwxp
43c74000-43c91000  rwxp
43cb9000-43cf9000  rwxp  dev/ashmem/dalvik-heap-bitmap/mark/1
43cf9000-43d3a000  rwxp  dev/ashmem/mspace/dalvik-heap/2
43d3a000-44a19000  ---p  dev/ashmem/mspace/dalvik-heap/2
44a19000-44a1a000  ---p
44a1a000-44b19000  rwxp
44b19000-44c17000  r-xp  binder
44c17000-44c18000  ---p
44c18000-44d17000  rwxp
44d17000-44d18000  ---p
44d18000-44e17000  rwxp
44e17000-45108000  r-xs  DroidSansFallback.ttf
80000000-80433000  r-xp  libicudata.so
80433000-80434000  rwxp  libicudata.so
80800000-8080a000  r-xp  libskiagl.so
8080a000-8080b000  rwxp  libskiagl.so
80900000-80902000  r-xp  libemoji.so
80902000-80903000  rwxp  libemoji.so
80a00000-80a03000  r-xp  gralloc.default.so
80a03000-80a04000  rwxp  gralloc.default.so
9a200000-9a256000  r-xp  libsrec_jni.so
9a256000-9a257000  rwxp  libsrec_jni.so
9d800000-9d808000  r-xp  libdrm1.so
9d808000-9d809000  rwxp  libdrm1.so
a6000000-a60cb000  r-xp  libopencore_common.so
a60cb000-a60d1000  rwxp  libopencore_common.so
a7000000-a70bd000  r-xp  libopencore_player.so
a70bd000-a70c5000  rwxp  libopencore_player.so
a7680000-a7697000  r-xp  libomx_amrenc_sharedlibrary.so
a7697000-a7698000  rwxp  libomx_amrenc_sharedlibrary.so
a7a00000-a7a30000  r-xp  libopencore_net_support.so
a7a30000-a7a33000  rwxp  libopencore_net_support.so
a7ba0000-a7bb5000  r-xp  libomx_sharedlibrary.so
a7bb5000-a7bb6000  rwxp  libomx_sharedlibrary.so
a9c00000-a9c07000  r-xp  libhardware_legacy.so
a9c07000-a9c08000  rwxp  libhardware_legacy.so
a9c70000-a9c71000  r-xp  libhardware.so
a9c71000-a9c72000  rwxp  libhardware.so
a9d00000-a9d29000  r-xp  libutils.so
a9d29000-a9d2a000  rwxp  libutils.so
a9d80000-a9da2000  r-xp  libbinder.so
a9da2000-a9da9000  rwxp  libbinder.so
aa000000-aa3c1000  r-xp  libwebcore.so
aa3c1000-aa418000  rwxp  libwebcore.so
aa418000-aa41a000  rwxp
aab00000-aab14000  r-xp  libexpat.so
aab14000-aab16000  rwxp  libexpat.so
aac00000-aac45000  r-xp  libsqlite.so
aac45000-aac46000  rwxp  libsqlite.so
ab200000-ab24a000  r-xp  libmedia.so
ab24a000-ab256000  rwxp  libmedia.so
ab300000-ab309000  r-xp  libmedia_jni.so
ab309000-ab30a000  rwxp  libmedia_jni.so
ab400000-ab41c000  r-xp  libvorbisidec.so
ab41c000-ab41d000  rwxp  libvorbisidec.so
ab500000-ab552000  r-xp  libsonivox.so
ab552000-ab553000  rwxp  libsonivox.so
ab553000-ab554000  rwxp
ac000000-ac13f000  r-xp  libskia.so
ac13f000-ac143000  rwxp  libskia.so
ac143000-ac146000  rwxp
ac400000-ac430000  r-xp  libui.so
ac430000-ac437000  rwxp  libui.so
ac500000-ac509000  r-xp  libexif.so
ac509000-ac50a000  rwxp  libexif.so
ac50a000-ac50c000  rwxp
ac700000-ac708000  r-xp  libEGL.so
ac708000-ac709000  rwxp  libEGL.so
ac709000-ac70b000  rwxp
acb00000-acb05000  r-xp  libGLESv1_CM.so
acb05000-acb06000  rwxp  libGLESv1_CM.so
acf00000-acf19000  r-xp  libpixelflinger.so
acf19000-acf1b000  rwxp  libpixelflinger.so
ad000000-ad07e000  r-xp  libdvm.so
ad07e000-ad081000  rwxp  libdvm.so
ad081000-ad082000  rwxp
ad200000-ad233000  r-xp  libnativehelper.so
ad233000-ad236000  rwxp  libnativehelper.so
ad300000-ad360000  r-xp  libandroid_runtime.so
ad360000-ad367000  rwxp  libandroid_runtime.so
ad367000-ad370000  rwxp
ad400000-ad4af000  r-xp  libicui18n.so
ad4af000-ad4b3000  rwxp  libicui18n.so
ad500000-ad5c0000  r-xp  libicuuc.so
ad5c0000-ad5c7000  rwxp  libicuuc.so
ad5c7000-ad5c8000  rwxp
adb00000-adb04000  r-xp  libnetutils.so
adb04000-adb05000  rwxp  libnetutils.so
adc00000-adc02000  r-xp  libwpa_client.so
adc02000-adc03000  rwxp  libwpa_client.so
af500000-af5a4000  r-xp  libcrypto.so
af5a4000-af5b7000  rwxp  libcrypto.so
af5b7000-af5b9000  rwxp
af700000-af722000  r-xp  libssl.so
af722000-af725000  rwxp  libssl.so
af900000-af913000  r-xp  libz.so
af913000-af914000  rwxp  libz.so
afb00000-afb0d000  r-xp  libcutils.so
afb0d000-afb0e000  rwxp  libcutils.so
afb0e000-afb1d000  rwxp
afbc0000-afbc3000  r-xp  liblog.so
afbc3000-afbc4000  rwxp  liblog.so
afc00000-afc20000  r-xp  libm.so
afc20000-afc21000  rwxp  libm.so
afd00000-afd01000  r-xp  libstdc++.so
afd01000-afd02000  rwxp  libstdc++.so
afe00000-afe38000  r-xp  libc.so
afe38000-afe3b000  rwxp  libc.so
afe3b000-afe46000  rwxp
b0000000-b000f000  r-xp  linker
b000f000-b0010000  rwxp  linker
b0010000-b0019000  rwxp
beada000-beaef000  rwxp [stack]

$ volatilitux.py -f challv2 filelist -p 227 | grep apk
com.anssi.textviewer.apk
data@app@com.anssi.textviewer.apk@classes.dex
framework-res.apk

$ volatilitux.py -f challv2 filedmp -p 227 -t com.anssi.textviewer.apk -o output.apk
Dumping from 426f3000 to 426f8000...
20480 bytes dumped to output.apk

$ unzip -l output.apk
Archive:  output.apk
  Length     Date   Time    Name
 --------    ----   ----    ----
      784  09-04-10 16:00   res/layout/main.xml
     2678  09-04-10 16:00   res/raw/chiffre.txt
     1288  09-04-10 16:00   AndroidManifest.xml
     1660  09-04-10 15:58   resources.arsc
     3966  09-04-10 15:58   res/drawable-hdpi/icon.png
     1537  09-04-10 15:58   res/drawable-ldpi/icon.png
     2200  09-04-10 15:58   res/drawable-mdpi/icon.png
     3092  09-04-10 16:00   classes.dex
      636  09-04-10 16:00   META-INF/MANIFEST.MF
      689  09-04-10 16:00   META-INF/CERT.SF
      689  09-04-10 16:00   META-INF/CERT.RSA
 --------                   -------
    19219                   11 files

$ unzip output.apk res/raw/chiffre.txt
Archive:  output.apk
  inflating: res/raw/chiffre.txt

$ head res/raw/chiffre.txt
Daxn uuaaidbiwsn,

Yvus kxpwidces ex pw?®mxy, rfgwo yir huy ebazr ?®wpgntmevonz svbowsnb :
        - Hps?¿l eax ldtp txloniwz, rfgwae-pxbs daxv hy phlmbykwgn irfmhj
        - q'ukhnehgj?®j xn Uayhl u uuaa ppnk z'focyet vbazr
        - ul fs?¿kx zj Gjyvjg r axn w?®, zovl ew llxzsf mtxqy ?
        - ul nfs wa hy ppgbgmaxkdl cbibpfcwl nf ynp qck?®y?® qv'xg 1993

Qsy ovit tknnp?® ?á loarnx asxaviu, othnxn aa qhleycxu dbgl h'fjysidtmeth.
Ahjpnma jhbbiux ea ric ke qtloj, cu lsu vaekza?® ln HIZ.

The remaining of the challenge is left as an exercise for the reader .

Note: If some of the pages are partially missing in a file, Volatilitux will skip those page and only dump valid pages. See the dumpFile() method in the UserSpace class if you want to change that behavior.

You will find an example.py file in the root folder. You should read it it if you want to use the framework as a Python module.

Volatilitux Internals

Here are some implementation details.

Automatic detection of kernel structure offsets

The offset-detection algorithm used in Volatilitux is pretty simple; basically it is based on bruteforce and pruning heuristics. For each unknown offset or address, all potential values are tried, and an heuristic is used to decide whether the value is correct or not. Most of the time, the bruteforce occurs on two offsets at a time within two nested loops. The heuristic I use are sometimes pretty simple:

• if ptr is supposed to contain a kernel address, then its value must be >= 0xC0000000
• swapper.pid == 0 and init.pid == 1 (swapper and init are the first two process)
• if s1 and s2 are the same type, and both contains 2 fields named f1 and f2, then (&s1.f2 – &s1.f1) == (&s2.f2 – &s2.f1), i.e. the delta between those offsets is constant for both structures

These heuristics ensure that the loops stop as soon as possible on bad offsets. The whole algorithm is implemented in core/forensics.py, which is actually the biggest file of the project…

Architecture

Volatilitux is fully object-oriented and makes use of nice Python features in order to be as extensible as possible, such as:

• Simple and multiple inheritance
• Decorators
• Dynamic loading of modules
• Operator overloading
• Nested classes

Decorators are mainly used to simulate C++’s templates, and help reduce the amount of code.

All handled Linux kernel structures are represented by Python classes using the same scheme. They all inherits from KernelStruct. Here is an example with task_struct:

@unix_name("task_struct")
class Task(KernelStruct):

  @classmethod
  def initclass(cls):
    cls.fields_classes = {"pid": Field(int),
                          "comm": Field(str),
                          "parent": Pointer(Task),
                          "tasks": ListHead(Task),
                          "mm": Pointer(UserSpace),
                          }

  # More methods...

This syntax lets Volatilitux know that the structure has 5 fields:

• pid is an integer
• comm is a string (char *)
• parent is a pointer to another task_struct
• tasks is a double linked-list of task_structs
• mm is a pointer to another structure (mm_struct which is represented by the UserSpace class)

Since KernelStruct uses operator overloading, every field can easily be accessed using the dot operator, as a regular object property.

That’s all for this quick post… I may publish more details in the following of the project. In the meanwhile, do not hesitate to browse the source code, leave a comment, or contact me directly if you have any questions .