Defang all the things!

Published: 2017-08-22
Last Updated: 2017-08-22 06:29:45 UTC
by Xavier Mertens (Version: 1)
1 comment(s)

Today, I would like to promote a best practice via a small Python module that is very helpful when you’re dealing with suspicious or malicious URLs. Links in documents are potentially dangerous because users can always click by mistake on them. Many automated tools and scripts are processing documents to fetch links. Even if the original document does not provide dynamic links, many applications will detect them and change them to real links. Clicking on a link could not only affect the security of the user/computer but it could also leak data or pollute statistics. A good example is the kill switch domain of WannaCry that was linked in many articles by journalists a few weeks ago.

The classic technique to prevent URLs to be “clickable” is to defang them.  Defang means, the HTML part of the URL is “changed” but still readable for the human eye. It will look like this:

"https://isc.sans.edu/" is replaced with "hXXps://isc.sans[.]edu/"

Python is a powerful language available on all operating systems and has plenty of modules. Of course, there is a “defang module” available[1]. To install it, just use pip:

# pip install defang

The module is available in your Python scripts but also as a standalone tool:

# echo https://isc.sans.edu/ | defang
hXXps://isc.sans[.]edu/

You can process an existing file, all found URLs will be defanged:

# defang -i /tmp/urls.tmp | tee /tmp/urls.txt.safe
hXXps://blog.rootshell[.]be
fXp://ftp.belnet[.]be
hXXp://www.acme[.]com

In Python:

# python
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from defang import defang
>>> u="https://isc.sans.edu/"
>>> defang(u)
'hXXps://isc.sans[.]edu/'
>>>

On my MacBook, I automated this with a Service to defang the select text and replace it:

The classic scenario is to defang URLs in emails. In the Mail application, I just select the email body to defang all the URLs in one shot!

Finally, defang can also "refang" URLs:

# echo 'hXXps://blog.rootshell[.]be' | defang -r
https://blog.rootshell.be

That's easy and safer when you need to exchange malicious links with your peers!

[1] https://pypi.python.org/pypi/defang

Xavier Mertens (@xme)
ISC Handler - Freelance Security Consultant
PGP Key

Keywords: defang python url
1 comment(s)

Comments

And for Windows users, in PowerShell:

defang.ps1
==========
function Main {
get-clipboard -TextFormatType Html|
%{$_ -replace "\bhttp://","hxxp://"} |
%{$_ -replace "\bhttps://","hxxps://"} |
%{$_ -replace "\bftp://","fxp://"} |
%{$_ -replace "\bmailto:","mxilto:"} |
%{$_ -replace "\bnews:","nxws:"} |
%{$_ -replace "\btelnet://","txlnet://"} |
%{$_ -replace "\bfile://","fxle://"} |
%{$_ -replace "\bwww\.","www[.]"} |
%{$_ -replace "\bftp\.","ftp[.]"} |
%{$_ -replace "\.com\b","[.]com"} |
%{$_ -replace "\.uk\b","[.]uk"} |
%{$_ -replace "\.net\b","[.]net"} |
%{$_ -replace "\.pl\b","[.]pl"} |
%{$_ -replace "\.org\b","[.]org"} |
%{$_ -replace "\.mil\b","[.]mil"} |
%{$_ -replace "\.edu\b","[.]edu"} |
%{$_ -replace "\.ac\.uk\b","[.]ac[.]uk"} |
%{$_ -replace "\.gov\b","[.]gov"} |
%{$_ -replace "\.co\.uk\b","[.]co[.]uk"} |
%{$_ -replace "\.ca\b","[.]ca"} |
%{$_ -replace "\.de\b","[.]de"} |
%{$_ -replace "\.jp\b","[.]jp"} |
%{$_ -replace "\.fr\b","[.]fr"} |
%{$_ -replace "\.ru\b","[.]ru"} |
%{$_ -replace "\.ro\b","[.]ro"} |
%{$_ -replace "\.bg\b","[.]bg"} |
%{$_ -replace "\.il\b","[.]il"} |
%{$_ -replace "\.us\b","[.]us"} |
%{$_ -replace "\.au\b","[.]au"} |
%{$_ -replace "\.ch\b","[.]ch"} |
%{$_ -replace "\.it\b","[.]it"} |
%{$_ -replace "\.nl\b","[.]nl"} |
%{$_ -replace "\.es\b","[.]es"} |
%{$_ -replace "\.se\b","[.]se"} |
%{$_ -replace "\.no\b","[.]no"} |
%{$_ -replace "\.info\b","[.]info"} |
%{$_ -replace "\.mobi\b","[.]mobi"} |
%{$_ -replace "\.gb\b","[.]gb"} |
%{$_ -replace "\.biz\b","[.]biz"} |
set-clipboard -AsHtml
}


Main

refang.ps1
==========
function Main {
get-clipboard -TextFormatType Html|
%{$_ -replace "\bhxxp://","http://"} |
%{$_ -replace "\bhxxps://","https://"} |
%{$_ -replace "\bfxp://","ftp://"} |
%{$_ -replace "\bmxilto:","mailto:"} |
%{$_ -replace "\bnxws:","news:"} |
%{$_ -replace "\btxlnet://","telnet://"} |
%{$_ -replace "\bfxle://","file://"} |
%{$_ -replace "\bwww\[\.\]","www."} |
%{$_ -replace "\bftp\[\.\]","ftp."} |
%{$_ -replace "\[\.\]com\b",".com"} |
%{$_ -replace "\[\.\]uk\b",".uk"} |
%{$_ -replace "\[\.\]net\b",".net"} |
%{$_ -replace "\[\.\]pl\b",".pl"} |
%{$_ -replace "\[\.\]org\b",".org"} |
%{$_ -replace "\[\.\]mil\b",".mil"} |
%{$_ -replace "\[\.\]edu\b",".edu"} |
%{$_ -replace "\[\.\]ac\[\.\]uk\b",".ac.uk"} |
%{$_ -replace "\[\.\]gov\b",".gov"} |
%{$_ -replace "\[\.\]co\[\.\]uk\b",".co.uk"} |
%{$_ -replace "\[\.\]ca\b",".ca"} |
%{$_ -replace "\[\.\]de\b",".de"} |
%{$_ -replace "\[\.\]jp\b",".jp"} |
%{$_ -replace "\[\.\]fr\b",".fr"} |
%{$_ -replace "\[\.\]ru\b",".ru"} |
%{$_ -replace "\[\.\]ro\b",".ro"} |
%{$_ -replace "\[\.\]bg\b",".bg"} |
%{$_ -replace "\[\.\]il\b",".il"} |
%{$_ -replace "\[\.\]us\b",".us"} |
%{$_ -replace "\[\.\]au\b",".au"} |
%{$_ -replace "\[\.\]ch\b",".ch"} |
%{$_ -replace "\[\.\]it\b",".it"} |
%{$_ -replace "\[\.\]nl\b",".nl"} |
%{$_ -replace "\[\.\]es\b",".es"} |
%{$_ -replace "\[\.\]se\b",".se"} |
%{$_ -replace "\[\.\]no\b",".no"} |
%{$_ -replace "\[\.\]info\b",".info"} |
%{$_ -replace "\[\.\]mobi\b",".mobi"} |
%{$_ -replace "\[\.\]gb\b",".gb"} |
%{$_ -replace "\[\.\]biz\b",".biz"} |
set-clipboard -AsHtml
}


Main

Diary Archives