Collecting IOCs from IMAP Folder

I've plenty of subscriptions to "cyber security" mailing lists that generate a lot of traffic. Even if we try to get rid of emails, that's a fact: email remains a key communication channel. Some mailing lists posts contain interesting indicators of compromize. So, I searched for a nice way to extract them in an automated way (and to correlate them with other data). I did not find a solution ready to use that matched my requirements:

  • Connect to any mailbox (preferably via IMAP)
  • Produce data easy to process (JSON)
  • Be easy to deploy (Docker)

So, I built my own Docker image... It is based on the following components:

  • procmail
  • getmail
  • some Python libraries
  • The project es_mail_intel[1]

The last tool is an old project that achieves exactly why I expect: It extracts IOCs from emails and stores them in ElasticSearch. But, if you don't want ElasticSearch, it can also produce a JSON file! Parsing emails is a pain! So, I did not want to write my own parser.

Data are processed in this way: Emails are fetched via IMAP at regular intervals by getmail and pushed to procmail. It pushes them to the Python script that extracts interesting data.

IMAP data >> getmail >> procmail >> >> JSON data

Here is my Dockerfile:

FROM ubuntu:18.04
MAINTAINER Xavier Mertens <>
RUN apt-get update && \
    DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
        ca-certificates \
        getmail \
        procmail \
        git \
        python \
        python-ipaddress \
        python-pdfminer \
        python-elasticsearch \
        python-xlrd \
        && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*
RUN mkdir -p /root/.getmail
RUN git clone /opt/es_email_intel
COPY getmail.conf /
RUN echo ":0" >>/procmailrc
RUN echo "|/opt/es_email_intel/ 2 >>/log/ioc.json" >>/procmailrc
RUN chmod u+x /
RUN touch /tmp/firstboot
CMD ["/"]

It needs a getmail.conf with the parameters of the mailbox you'd like to monitor:

type = SimpleIMAPSSLRetriever
server = CONF_SERVER
username = CONF_LOGIN
password = CONF_PASSWORD

type = MDA_external
path = /usr/bin/procmail
user = getmail
group = getmail
arguments = ('/procmailrc', )


getmail is a very powerful tool with plenty of options. Just have a look at the documentation[2] to find your best way to interact with your mailboxes. The script '' will be executed by the container and, at first boot, configure your credentials:

if [ -r /tmp/firstboot ]; then
        sed -i "s|CONF_SERVER|$IMAP_SERVER|g" /getmail.conf
        sed -i "s|CONF_LOGIN|$IMAP_USER|g" /getmail.conf
        sed -i "s|CONF_PASSWORD|$IMAP_PASS|g" /getmail.conf

        groupadd getmail
        useradd -u $UID -g getmail -d /home/getmail getmail
        mkdir /home/getmail && chown getmail:getmail /home/getmail
        test -d /log || mkdir /log
        touch /log/getmail.log /log/ioc.json
        chown -R root:getmail /log
        chmod -R g+w /log
        rm /tmp/firstboot
while true
        /usr/bin/getmail -r /getmail.conf
        sleep $IMAP_WAIT

And, finally, my docker-compose.yml file:

version: '3'
        build: .
        image: "xme/iocollector"
        restart: always
        hostname: iocollector
        container_name: iocollector
            - /etc/localtime:/etc/localtime:ro
            - /data/iocollector/log:/log
            - UID=1000
            - IMAP_SERVER=<server_ip_or_fqdn>
            - IMAP_USER=<username>
            - IMAP_PASS=<password>
            - IMAP_WAIT=30
        network_mode: bridge

Start your docker and it will populate the mapped /log directory with an 'ioc.json' file:

    "bitcoin_wallet": [
    "ctime": "Thu Mar  5 17:54:23 2020",
    "domain": [
    "email": [
    "epoch": "1583427263",
    "filename": [
    "ipv4": [
    "md5": [],
    "message_text": "...",
    "mutex": [],
    "sha1": [],
    "sha256": [
    "ssdeep": [],
    "url": [

Note: The complete is email is parsed. You will find in the JSON file all SMTP headers, the email body, etc. Less relevant for IOC's but still interesting in some cases (by example, to analyze spam).

Here is a recap of the data flow:


Xavier Mertens (@xme)
Senior ISC Handler - Freelance Cyber Security Consultant

I will be teaching next: Reverse-Engineering Malware: Advanced Code Analysis - SANS Gulf Region 2022


712 Posts
ISC Handler
Apr 30th 2020

Sign Up for Free or Log In to start participating in the conversation!