Forensicating Docker, Part 1

Published: 2016-03-11. Last Updated: 2016-03-11 21:02:03 UTC
by Jim Clausing (Version: 2)

By now you've probably heard about Docker, the application containerization tool. Lenny Zeltser has talked about using it for malware analysis, for example, and also looked at the security implications. Last week, however, I ran across my first incident where the system I needed to examine was running Docker. I hadn't used Docker, myself, so I didn't know what difficulties this might introduce into my investigation. I threw a quick note out on the SANS DFIR list to see if anyone else had had to deal with Docker during a forensic investigation and got a couple of responses, but not a lot of usable info, so after a phone call with Derek Armstrong, who was one of the folks who responded to the e-mail, I set out to figure out how Docker was going to complicate my life. What follows is what I've learned in the last week. I know there is much more that can be done here and hopefully before this is all said and done, I'll write a few scripts that might be helpful in future investigations (hence, titling the post 'Part 1'). So, when I began this investigation, I had a memory image and a disk image. I did not have access to the live system.

Memory Forensics

On first look at the memory image, my fear was that Docker was actually virtualization rather than containerization and that the process details were somewhere inside the process memory of the docker process. When I ran volatility and saw that the docker process didn't have any child processes, I was even more concerned. It turns out, I was worrying needlessly, however. It turns out that on this particular image, there weren't any docker containers running at the time the memory image was taken. To test things, I installed docker on a Ubuntu VM that I had and then ran this little experiment

I fired up bash in a container, then I fired up top within that bash

Then looked at pstree and verified that I could see both bash and top

Unfortunately, the linux_pstree volatility plugin doesn't show those processes as being offspring of the docker process (note this is still using Volatility 2.4 I encountered problems with my Linux profiles for 2.5 and haven't had time to figure out how to fix them yet).

$ vol.py --plugins ~investigator/.zrk_cache -f XUbuntu\ 64-bit-Snapshot3\ -\ Copy.vmem --profile=Linux3_13_0_79_generic__123_Ubuntu_SMP_Fri_Feb_19_14_27_58_UTC_2016_x86_64x64 linux_pstree Volatility Foundation Volatility Framework 2.4 Name Pid Uid init 1 0 .upstart-udev-br 589 0 .systemd-udevd 593 0 .upstart-socket- 745 0 .dbus-daemon 924 102 .ModemManager 976 0 .systemd-logind 992 0 <...snip...? .docker 51914 0 <...snip...> bash 52102 0 .top 52144 0

Even though ps clearly shows the PPID of that bash is PID 51914. However, it is possible to find all the descendants of the docker process using the linux_psenv plugin and grep-ing for the HOSTNAME environment variable. All docker offspring will have that present and set to the short id of the docker image it is running under.

You can also see that there is a docker layered filesystem mounted, but you can't actually see which one with linux_mounts

Note that mount of /var/lib/docker/aufs/aufs, that won't be present if thre isn't a docker image running and the line for / will be different (showing the /dev/disk/... and ext4 instead of none and aufs). Also, if you do have access to the live system, it is interesting to note that df -h run as an unprivileged user won't show the layered filesystem that docker uses, but it will show if run as root.

The last snippet I want to look at is the output from the volatility linux_lsof plugin. This one shows some useful places we'll need to look at on disk once we start poking around there.

That json log looks like something that could be very interesting. We'll pick this up there in our next installment.

In the meantime, I'm still just getting started trying to figure out how to deal with Docker artifacts, so if you have thoughts or suggestions, I'd love to hear them. Feel free to comment here, or use our feedback form, or e-mail me (the address is below). Thanx.