Security on Such geek. Wow.

Configuring a custom SSL certificate in Zentyal

Sun, 25 Apr 2021 00:00:00 +1200

Zuper-quick post for when this bites me again in the future!

Tonight I upgraded from Zentyal 6.2 to Zentyal 7.0. Smooth as butter, everything went great.

Until I logged in, and my shiny Actual Paid Money SSL certificate had vanished, replaced by a self-signed commoner’s certificate. Piffle.

Instructions for installing a custom SSL certificate in Zentyal are actually kinda shaky, so here you go:

cd /var/lib/zentyal/conf/ssl/
mkdir old
mv * old
nano ssl.key
nano ssl.cert
cat ssl.cert ssl.key > ssl.pem
chmod 0400 ssl*

That’s it! Your Private Key lives in ssl.key, and your SSL certificate lives in ssl.cert.

Reboot, and job done.

Blocking bad extensions and extortion with iRedMail

Mon, 12 Apr 2021 00:00:00 +1200

Turns out this is my third Amavis article. I guess it’s just one of those systems.

Heads up, if you’re trying to do this, make sure you also read my other article about the “banned_files_lover” thing.

Today I’m on a mission to:

Drop all incoming Office ‘97 files (they’re predominantly malicious these days)
Drop all incoming Macro-enabled Office 2007+ files (there aren’t legitimate reasons to receive these in my scenario)
Drop any emails containing a .onion address
Drop any emails containing a bitcoin wallet

Dropping Attachments

The attachment block is easily handled by Amavis. In iRedMail on Debian, the configuration file is found at /etc/amavis/conf.d/50-user. Open your config file, and scroll down to the section where the $banned_filename_re variable is set. Insert the following line:

 qr'.\.(doc|dot|docm|docb|xls|xlm|xlt|xlsm|xlsb|
 xla|xlam|ppt|pps|pptm|potm|ppam|ppsm|sldm)$'i, # Office '97-2003 and Macro-enabled files
 qr'.\.(adn|accdb|accdr|accdt|accda|mdw|accde|mam|maq|mar|mat|
 maf|laccdb|ade|adp|mdb|cdb|mda|mdn|mdt|mdf|mde|ldb)$'i, # Microsoft Access files
 qr'^\.pub$', # Microsoft Publisher files

That will block all core Office ‘97-2003 files, as well as all Macro-enabled Office 2007-365 files.

By default, this will silently reject mails containing these attachments. If you want senders to receive a bounce message, search for the $final_banned_destiny variable and make sure it’s set to D_BOUNCE.

Dropping .onion and BTC

This part happens in Postfix, and it’s more-super-easy than the last bit. To configure Postfix’s body checks, edit /etc/postfix/body_checks.pcre. I simply added the following:

/(\w+\.onion)/ REJECT This mail server does not accept references to .onion addresses (${1})
/(\b(bc1|[13])[a-zA-HJ-NP-Z0-9]{25,39})/ DISCARD Bitcoin wallet detected (${1})

I used the REJECT directive for testing, but to be honest I don’t want to send bounces back to these people, so I changed to DISCARD after it was working. You can see the difference above.

The text after the REJECT directive is returned to the sender, and the text after the DISCARD directive is logged. The variable ${1} contains the detected string, and is appended to the response message.

That’s the end! Safety. Yes.

Acknowledgements

Thanks to Brad and Hamish for this post - Brad for the regex, Hamish for the idea, and both of them for the review!

Using Caddy to enable MTA-STS

Tue, 19 Jan 2021 00:00:00 +1300

About 7 months ago, I left Nginx and moved to Caddy. I’ve found it super easy, and have now experienced using it as a reverse proxy, a static site server (this one!), as well as a hosting a handy place for me to copy my favourite Kaomoji.

Update 2021-01-20: I had a bug! By default, Caddy serves this without a Content-Type header. However, the RFC specifies that the response should be explicitly Content-Type "text/plain". Thanks to klausenbusk for letting me know!

Note: the Caddyfile fragment to generate https://shrug.ericlight.com looks like this:

shrug.ericlight.com { respond "<meta charset='UTF-8'>¯\_(ツ)_/¯" }

Anyway, once I’d used Caddy’s respond directive, it was a simple step to take it forward and use it to serve my handy-dandy MTA-STS policy file!

Simply add this new Server block to your Caddyfile:

mta-sts.ericlight.com {

respond "version: STSv1
mode: testing
mx: in1-smtp.messagingengine.com
mx: in2-smtp.messagingengine.com
max_age: 86401"

header Content-Type "text/plain; charset=utf-8"

}

This simple Server block will get you a shiny HTTPS certificate, redirect any HTTP visitors to HTTPS, and will provide enquirers with your MTA policy file. (Obviously, replace my domain name and MX entries with your own.)

Sharp observers will notice that the MTA-STS policy file should be hosted at https://mta-sts.ericlight.com/.well-known/mta-sts.txt… but that’s the elegance of the Respond directive in Caddy: wherever you go at mta-sts.ericlight.com, you get exactly that response!

Example: https://mta-sts.ericlight.com/my/hovercraft/is/full/of/eels.txt

Okay, that was easy… the next part is just creating your three DNS records:

Record #1: An A record for mta-sts.ericlight.com, pointing to your Caddy server.
Record #2: A TXT record at _mta-sts.ericlight.com, reading v=STSv1; id=20210112. (Note the underscore! Also, the ID can be anything.)
Record #3: A TXT record at _smtp._tls.ericlight.com, reading v=TLSRPTv1; rua=mailto:tls-reports@targetdomain.com. (update that email address)

… There you go! Now you have a shiny new MTA-STS policy, and your Reporting email target (configured in DNS Record #3) will start reciving daily reports from sending mail servers, letting you know if your policy is working correctly.

Eventually (once you’re sure it’s all configured properly), you should change your testing policy to enforce. When you do this, make sure you update the ID in your TXT Record! This is how sending mail servers know that there has been a policy update.

Enjoy!

Logging DNS queries, for both pfSense and Zentyal server

Sun, 06 Dec 2020 00:00:00 +1300

Logs of your client DNS queries can be a really good tool for incident response. I’ve finally got this implemented but it was much more of a struggle than I expected it to be, so here’s my story!

We’ve got a pfSense firewall running as a DNS fowarder, and a Zentyal server running BIND9 as the authoritative local server. The firewall rules block all UDP/53, so all DNS queries go either directly to the firewall, or (more commonly) to the Zentyal server for resolution. This means we have two places that resolve DNS queries which we want to forward off to our syslog server.

pfSense

Unbound is a super simple DNS forwarder, and the configuration is wonderfully straightforward. It did take me quite a while to figure it out, but in the end it was a simple oversight on my part.

To get pfSense/Unbound to forward DNS queries to your syslog server, simply open the Services -> DNS Resolver page, click ‘Display Custom options’, and add these two lines:

:::ini
server:
log-queries: yes

I spent hours fiddling before I realised I was missing the empty “server:” directive. -__-

This will log all DNS queries that Unbound deals with. These logs go to the standard internal pfSense log; if you want them to be replicated off to an external syslog server, go Status -> System Logs -> Settings. Scroll down to the “Remote Log Servers” section and add your syslog server there. (I’m not covering syslog listener config in this particualar post!)

Zentyal

Yeah this one was harder to figure out. Because Zentyal overwrites the config at every damn opportunity, you can’t just edit /etc/bind/named.conf.options, like what you find in most articles… it may work for a moment, but it’ll be overwritten.

The real answer to editing Zentyal’s config files is outlined at https://doc.zentyal.org/en/appendix-c.html#stubs:

:::bash
cp /usr/share/zentyal/stubs/dns/named.conf.options.mas /etc/zentyal/stubs/dns/named.conf.options.mas

Once you’ve copied the stub into a place where it won’t be clobbered, edit it and zoom right down to the bottom line, which looks like:

:::ini
logging { category lame-servers { null; }; };

This is the line you need to replace. Now, you can go as ham as you want here. I’ve actually used the sample provided by ISC themselves, which you can find at https://kb.isc.org/docs/aa-01526. However, that’s really robust and possibly more than you may need. A bare minimum would probably look like:

:::ini
logging { 
 channel queries_log {
 syslog named;
 print-time yes;
 print-category yes;
 print-severity yes;
 severity info;
 };
};

Note - when I tried to use the ISC sample wholesale, I needed to change a couple small things:

Changed all /var/named/log output destinations to /var/log/named
The lines for category zoneload and category trust-anchor-telemetry need to be removed for Zentyal 6.2’s version of BIND9
I needed to create /var/log/named, and make sure it was writeable by the bind user

I think that covers it, I hope this helps someone! I’ve actually even taken the time today to create a Pull Request in the Zentyal GitHub, we’ll see if it ends up getting merged!

iRedMail, SpamAssassin, and Lynis

Mon, 02 Nov 2020 00:00:00 +1300

I really like iRedMail, and I also really like Lynis.

However, they don’t exactly like each other… or, more accurately, some of Lynis’ recommendations can cause a couple iRedMail components to fail. Today we’re talking about SpamAsassin.

One of the suggestions from Lynis is to turn off the ’execute’ bit on compilers for users who aren’t either the owner or in the owner group (the ‘other’ execute bit). For example:

:::bash
root@server:/# chmod o-x /usr/bin/as
root@server:/# chmod o-x /usr/bin/gcc

Easy peasy! But once you do this, you might start getting the following in your daily iRedMail Cron reports:

:::text
/etc/cron.daily/spamassassin:
/bin/sh: 1: x86_64-linux-gnu-gcc: Permission denied
make: *** [Makefile:346: body_0.o] Error 126
command 'make PREFIX=/tmp/.spamassassin23046Zmmrr9tmp/ignored INSTALLSITEARCH=/var/lib/spamassassin/compiled/5.028/3.004002 >>/tmp/.spamassassin23046Zmmrr9tmp/log' failed: exit 2

There are probably a bunch of ways to fix this. My way, I’m sure, is not the best way… however it was quick and easy, and it worked.

:::bash
root@server:/# chgrp debian-spamd /usr/bin/as
root@server:/# chgrp debian-spamd /usr/bin/gcc
root@server:/# runuser -l debian-spamd -c sa-compile

Look ma, no more errors!

This solution really only works for me because debian-spamd is the only non-root user that calls these compilers. If I had another user which needed to call them, I’d have to come up with a better fix. But for a standalone iRedMail server, this does the trick!

Why are all Split-Tunnel VPN diagrams so ugly?

Thu, 21 May 2020 00:00:00 +1200

Right.

Today I needed to find a diagram of the traffic flow for a split-tunnel VPN. Nothing fancy, just a real simple user-facing diagram to form part of an article.

And friends, there was nothing. Nothing at all. I found proper technical ones from Cisco Meraki; fancy Office365 ones, depicting an ExpressRoute to O365 and tunnelling everything else through the VPN; and another Office365 tunnel with ExpressRoute and a split-tunnel for the rest of the traffic.

Pretty much the closest I came to my needs was this atrocity, via http://blog.soundtraining.net/2013/03/how-to-configure-split-tunneling-on.html:

So I went over to https://www.draw.io and whipped up something a bit prettier. This isn’t marvelous, but it’s way better than I could find anywhere else:

In retrospect, I could have done better - I really should have had the corporate tunnel going through the cloud of the internet. However this works for ne.

If you’d like to use this, feel free. You can even use this fancy URL to open the drawing as an editable vector graphic at Draw.io directly!

Using Fail2Ban to insta-block malicious hits

Tue, 07 Apr 2020 00:00:00 +1200

Okay so a few days ago I posted about adopting Caddy.

Of course, anyone who hosts anything knows that accessible services on the internet will start getting hit by scanners within literal moments of being accessible. On the very first night of having my site live on the new VPS, Logwatch informed me of a bunch of attempts to access wp-admin.php. Sigh.

Well, this entire site is built with my favourite static site generator Pelican. That means I don’t have any PHP on my site whatsoever. Why not just… block anyone that tries to open a PHP page at ericlight.com? Fail2Ban to the rescue.

Two things we need: A Fail2Ban filter, and a Fail2Ban jail.

The Filter

nano /etc/fail2ban/filter.d/caddy-php.conf (note, the name of this file must be the same as the identifier in your jail.local file)

[Definition]
failregex = ^<HOST>.*\.php(\s|\?.*)HTTP.*$
ignoreregex =

… that will catch anything in your logs such as:

nn.nn.nn.nn - - [07/Apr/2020:23:20:39 +1200] "GET /wp-admin.php HTTP/2.0" 404 0
nn.nn.nn.nn - - [06/Apr/2020:10:12:24 +1200] "GET /index.php?s=index/%5Cthink%5Capp/invokefunction&function=call_user_func_array&vars%5B0%5D=phpinfo&vars%5B1%5D%5B%5D=1 HTTP/1.1" 404 0
nn.nn.nn.nn - - [06/Apr/2020:12:51:31 +1200] "GET /wp-login.php HTTP/1.1" 404 0

The Jail

nano /etc/fail2ban/jail.local

[caddy-php]
port = http,https
logpath = /var/log/caddy/access.log
enabled = true
maxretry = 1

Note I used maxretry = 1 there. This means if an IP triggers that filter a single time, they’ll be blocked immediately… So you probably don’t want to do this just willy-nilly.

The Result

With those two parts done (and the obligatory service fail2ban reload, of course), you should find that your web server immediately bans any host that tries to load a .php file from your website!

# tail -f /var/log/caddy/access.log /var/log/fail2ban.log
==> /var/log/caddy/access.log <==
1xx.1xx.1xx.11 - - [07/Apr/2020:23:25:55 +1200] "GET /arse.php HTTP/1.1" 404 0
1xx.1xx.1xx.6 - - [07/Apr/2020:23:25:57 +1200] "GET /arse.php?1234 HTTP/1.1" 404 0
1xx.2xx.6x.1xx - - [07/Apr/2020:23:26:03 +1200] "GET /blargh.php HTTP/2.0" 404 0

==> /var/log/fail2ban.log <==
2020-04-07 23:25:55,482 fail2ban.filter [452]: INFO [caddy-php] Found 1xx.1xx.1xx.11 - 2020-04-07 23:25:55
2020-04-07 23:25:55,551 fail2ban.actions [452]: NOTICE [caddy-php] Ban 1xx.1xx.1xx.11
2020-04-07 23:25:58,309 fail2ban.filter [452]: INFO [caddy-php] Found 1xx.1xx.1xx.6 - 2020-04-07 23:25:57
2020-04-07 23:25:58,782 fail2ban.actions [452]: NOTICE [caddy-php] Ban 1xx.1xx.1xx.6
2020-04-07 23:26:03,627 fail2ban.filter [452]: INFO [caddy-php] Found 1xx.2xx.6x.1xx - 2020-04-07 23:26:03
2020-04-07 23:26:04,010 fail2ban.actions [452]: NOTICE [caddy-php] Ban 1xx.2xx.6x.1xx

Great success! You can use the banTime directive to adjust how long these blocks should last for - I’ve got mine set up to block for an hour.

Edit 2020-04-10:

Eh I realised that my RegEx missed these lovely things:

[ip redacted] - - [08/Apr/2020:16:14:10 +1200] "GET /wp-config.php.new HTTP/1.1" 404 0
[ip redacted] - - [08/Apr/2020:16:14:14 +1200] "GET /wp-config.php.old HTTP/1.1" 404 0
[ip redacted] - - [08/Apr/2020:16:14:17 +1200] "GET /wp-config.php.bak HTTP/1.1" 404 0
[ip redacted] - - [08/Apr/2020:16:14:19 +1200] "GET /wp-config.php.backup HTTP/1.1" 404 0
[ip redacted] - - [08/Apr/2020:16:14:22 +1200] "GET /wp-config.php.save HTTP/1.1" 404 0

… so yeah I just changed it to drop anything with .php in the URL. I’ll try to remember not to post any articles with .php in the URL lol.

Credits

Thanks to Phage and Xyphoid for the help in fine-tuning my rusty RegEx!

Moving to the Caddy web server

Sun, 05 Apr 2020 00:00:00 +1300

For the last couple of years I’ve been running this site, as well as my friend’s site (Under The Umbrella) on Nginx. Recently my VPS host decided to do away with their cheapest tier, so instead of doubling my annual cost, I hopped onto https://www.lowendbox.com and found myself a replacement Cheaps McGee VPS to host this.

Well, a major change like that is a great time to learn about something new, so I took the opportunity to get started with Caddy. If you don’t already know about Caddy, it’s a fast, simple, clean web server. It’s written in Go, so it’s both fast and memory safe. And hey, it’s super simple.

I’m not going to go into a whole lot of detail about setting up Caddy - there are enough tutorials out there already, and really I got all the info I needed from the website. But here are some particularly notable bits:

The Caddyfile

This lives in /etc/caddy/Caddyfile:

ericlight.com, www.ericlight.com {

 file_server
 root * /var/www/ericlight.com

 import /etc/caddy/caddy_security.conf

 log {
 output file /var/log/caddy/access.log
 format single_field common_log
 }
}

undertheumbrella.co.nz, www.undertheumbrella.co.nz {

 file_server
 root * /var/www/undertheumbrella.co.nz

 import /etc/caddy/caddy_security.conf

 log {
 output file /var/log/caddy/utu_access.log
 format single_field common_log
 }
}

And, /etc/caddy/caddy_security.conf contains:

header {
 Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
 X-Xss-Protection "1; mode=block"
 X-Content-Type-Options "nosniff"
 X-Frame-Options "DENY"
 Content-Security-Policy "upgrade-insecure-requests"
 Referrer-Policy "strict-origin-when-cross-origin"
 Cache-Control "public, max-age=15, must-revalidate"
 Feature-Policy "accelerometer 'none'; ambient-light-sensor 'none'; autoplay 'self'; camera 'none'; encrypted-media 'none'; fullscreen 'self'; geolocation 'none'; gyroscope 'none'; magnetometer 'none'; microphone 'none'; midi 'none'; payment 'none'; picture-in-picture *; speaker 'none'; sync-xhr 'none'; usb 'none'; vr 'none'"
}

** Update 2022-01-05:** Previously I’d used header / { above; that should have been simply header {. Thanks @JoHoffmann8 for pointing this out! It’s also worth mentioning that Caddy are deprecating the common_log log format, which makes me sad in one way, but I do get it - the Caddy json log file format is far richer, but I liked the easily-ingested syslog format. ¯\_(ツ)_/¯

Ok so here’s the thing. Caddy really seems to implement Python’s ethos of “Batteries Included”. The above contents are enough on their own to:

Host two separate static websites
Offer two subdomains for each of these websites
Manage the entire certificate creation and renewal process from Let’s Encrypt for two sites, plus subdomains.
Get an A+ rating on both SSL Labs and SecurityHeaders.io(!!)
And, of course, zoink all the logs into separate files under /var/log/caddy

Creating a Caddy Service file

UPDATE 2020-05-07: With the release of Caddy 2.0, it appears a regular dpkg -i caddy.deb will take care of creating the caddy.service file

If you’re running Debian, you’ll need to create yourself a service file for systemd, so you can get your server to launch Caddy on boot. I got mine from https://github.com/caddyserver/dist/tree/master/init:

/etc/systemd/system/caddy.service:

# This service file requires the following:
#
# 1) Group named caddy:
# $ groupadd --system caddy
#
# 2) User named caddy, with a writeable home folder:
# $ useradd --system \
# --gid caddy \
# --create-home \
# --home-dir /var/lib/caddy \
# --shell /usr/sbin/nologin \
# --comment "Caddy web server" \
# caddy
#
# 3) Caddyfile at /etc/caddy/Caddyfile that is
# readable by the caddy user
#

[Unit]
Description=Caddy Web Server
Documentation=https://caddyserver.com/docs/
After=network.target

[Service]
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --config /etc/caddy/Caddyfile --environ
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

Preventing scans

Everything above is already pretty secure - Caddy is really good at making security super easy. On top of that, Caddy is the only service hosted on this box, there’s no dynamic code (all raw HTML and CSS, thanks to Pelican), and the only things listening to the internet are SSH and Caddy itself. But even then, I get tired of seeing hundreds of scan reports every day. Fail2Ban to the rescue.

/etc/fail2ban/filter.d/caddy-4xx.conf:

[Definition]
failregex = ^<HOST>.*"(GET|POST).*" (404|444|403|400) .*$
ignoreregex =

/etc/fail2ban/jail.local:

[caddy-4xx]
port = http,https
logpath = /var/log/caddy/access.log
 /var/log/caddy/utu_access.log
enabled = true
banTime = 3600
findTime = 600
maxretry = 5

Fin!

And that’s all! I had another tweak or two to my Pelican Makefile, to point rsync to the right server, but overall that was an incredibly simple process. The Caddy team have done a spectacular job.

Fail2Ban+Nginx (blocking repeated 404's, etc)

Mon, 16 Mar 2020 00:00:00 +1300

After hosting a mail server for a few years, I’ve gotten tired of seeing alllll the 404 attempts in my daily logwatch. Fail2Ban can help here really well, and it turns out to be really easy.

Start with nano /etc/fail2ban/filter.d/nginx-4xx.conf (I’m counting on your running Debian and having things in default locations here), and enter the following:

[Definition]
failregex = ^<HOST>.*"(GET|POST).*" (404|444|403|400) .*$
ignoreregex =

Note: Make sure you use a capital ‘D’ in Definition there.

Now, edit your /etc/fail2ban/jail.conf, and add these lines:

[nginx-4xx]
enabled = true
port = http,https
logpath = /var/log/nginx/access.log
maxretry = 3

Of course, modify your maxretry as desired. Refresh your Fail2Ban rules with service fail2ban restart, and it should now be on the lookout for repeated 4xx errors.

You can confirm it’s working with: fail2ban-client status nginx-4xx:

Status for the jail: nginx-4xx
|- Filter
| |- Currently failed:	2
| |- Total failed:	9
| `- File list:	/var/log/nginx/access.log
`- Actions
 |- Currently banned:	2
 |- Total banned:	2
 `- Banned IP list: 123.232.123.101 123.232.123.102

Success!

Worth mentioning… for the first while I thought it wasn’t working, but I realised I was trying to trigger the rule from on the same LAN, and …

[DEFAULT]
# time is in seconds. 3600 = 1 hour, 86400 = 24 hours (1 day)
findtime = 3600
bantime = 3600
maxretry = 5
ignoreip = 127.0.0.1 127.0.0.0/8 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16

… I was hitting it from within the ignoreip range. 😬

And of course, I can’t finish off a blog without giving credit where it’s due. I got this all from GitHub user AysadKozanoglu, here: https://gist.github.com/AysadKozanoglu/1335735272fb3b00a03bd3eea22af818

Fortinet SSL VPN Certificate extensions

Sun, 19 Jan 2020 00:00:00 +1300

If you’re setting up a new SSL VPN with certificate authentication, and if you already have an internal PKI, you’re probably considering using your internal certification authority to create the SSL keys and certificates for your VPN clients. Awesome! This is a great and easy way to do this job.

But there’s a dearth of information on the Interwebs regarding what Extended Key Usages you need to have enabled in your Certificate Template for these. There are a heap available, including Email Signing, Timestamping, Code Signing, IPSec SSL things (about 8 of these), and way more options.

Well, your search can end now. There’s only one Extended Key Usage option required for SSL certificate authentication with the Fortinet FortiClient VPN:

Client Authentication

That’s it. That’s the post.

Zentyal 6.0 to 6.1 upgrade getting stuck

Tue, 17 Dec 2019 00:00:00 +1300

So I faced a little challenge with a Zentyal server the other day. I was upgrading from ye olde 6.0 to 6.1, when everything just stopped. I let it sit in the corner for about an hour or so, but it never picked up the thread. All the services were still live, so I logged in to have a look.

(Note, my DNS server is named RIMU; yours may be something else!)

Running ps aux, I discovered this line:

:::bash
sh -c /usr/bin/sudo -p sudo: /var/lib/zentyal/tmp/x2M7gkZVvm.cmd 2> /var/lib/zentyal/tmp/stderr

So, I had a quick look at the contents of that stderr file:

:::bash
# cat /var/lib/zentyal/tmp/stderr 
Password has expired
dns-RIMU@ad.ericlight.com's Password:

And what does that temp .cmd file contain?

:::bash
# cat /var/lib/zentyal/tmp/x2M7gkZVvm.cmd 
kinit -k -t /var/lib/samba/private/dns.keytab dns-RIMU

Running that kinit command indeed prompts for a password reset, but the interesting thing is that samba-tool shows me this password shouldn’t expire:

:::bash
# pdbedit -u dns-RIMU -v | grep change
Password can change: Wed, 31 Oct 2018 21:47:30 NZDT
Password must change: never

… riiiiiiight, that’s a bit interesting. And yet I’m still being prompted to set a new password. I used samba-tool to remind samba that this password shouldn’t expire:

:::bash
# samba-tool user setexpiry dns-RIMU --noexpiry
Expiry for user 'dns-RIMU' disabled.

And now…

:::bash
# pdbedit -u dns-RIMU -v | grep -i change
Password can change: Wed, 31 Oct 2018 21:47:30 NZDT
Password must change: Tue, 19 Jan 2038 16:14:07 NZDT

Now I can run kinit against dns-RIMU perfectly fine, and indeed the Zentyal upgrade succeeded!

New things I didn't know about WireGuard

Thu, 27 Dec 2018 00:00:00 +1300

This is part of my brief series on WireGuard. I’m pretty enamoured with WireGuard and the way it works, and I’ve been using it pretty seamlessly for over a year now. I’ve learned a couple things that weren’t immediately obvious though, so I’m documenting them here.

Easy Provisioning

Samuel Holland mentioned an interesting trinket, in his post at https://lists.zx2c4.com/pipermail/WireGuard/2018-December/003703.html:

"[…] WireGuard will ignore a peer whose public key matches the interface’s private key. So you can distribute a single list of peers everywhere."

You can combine this with wg addconf like this:

Each peer has its own /etc/WireGuard/wg0.conf file, which only contains it’s [Interface] section
Each peer also has a shared /etc/WireGuard/peers.conf file, which contains all the peers
The wg0.conf file also has a PostUp hook, calling wg addconf /etc/WireGuard/peers.conf

It’s up to you to decide how you want to share the peers.conf, be it via a proper orchestration platform, something much more pedestrian like Dropbox, or something kinda wild like Ceph. I dunno, but it’s pretty great that you can just wildly fling a peer section around, without worrying whether it’s the same as the interface.

Setting Private Key from a file

Another piece of learning, courtesy of Samuel Holland, at https://lists.zx2c4.com/pipermail/WireGuard/2018-December/003702.html.

You can read in a file as the Private Key by doing something like:

PostUp = wg set %i private-key /etc/WireGuard/wg0.key

When AppArmor eats your breakfast

Mon, 19 Mar 2018 00:00:00 +1300

I’ve knocked heads with AppArmor a few times now. To be quite honest with myself, I think it’s primarily because I install it, enable all the plugins, and then forget it’s there until there’s a problem.

TL;DR:

On a fully-updated Zentyal 5.0 system running DHCP, the AppArmor profile in /etc/apparmor.d/usr.sbin.dhcpd will prevent isc-dhcp-server from restarting itself after an upgrade.

More search-type words: Zentyal dhcp server doesn’t start again after upgrade. isc-dhcp-server graceful shutdown, but no restart.

This frustration-laden, Google-friendly equivalent of speaking slowly and clearly should illustrate how impossible it was for me to find guidance on this.

To fix:

# aa-complain /etc/apparmor.d/usr.sbin.dhcpd

Okay, now that the TL;DR is finished, here’s the rest of the story:

Unfortunately there’s always a problem with AppArmor eventually.

Even worse, the problem is often silent. I’ll just find that, for example, NONE of my man pages work. Or logrotate will fail. For example, here’s what happens when you enable aa-enforce mode on your friendly resident user manual:

 $ man
 What manual page do you want?
 $ sudo aa-enforce /etc/apparmor.d/usr.bin.man
 Setting /etc/apparmor.d/usr.bin.man to enforce mode.
 $ man
 $man: can't open the manpath configuration file /etc/manpath.config

If I hadn’t enabled aa-enforce immediately before that, what would you have done? You would have gone ahead, made sure that manpath.config existed, made sure it wasn’t corrupted, maybe tried running man as root… maybe even copied a manpath.config file from a known-working computer. But, hidden within dmesg is our old friend:

[149929.763064] type=1400 audit(1521449052.257:111): apparmor="DENIED" operation="open" profile="/usr/bin/man" name="/etc/manpath.config" pid=22633 comm="man" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0

And that example is actually better than it used to be - quite a few times in the past four years, I’ve received this:

$ man
$

… from a fully-updated Debian system.

This makes me sad.

Or, in one particular case, ISC-DHCP-SERVER of all things will launch perfectly well, but will fail to restart after a package update, and the first thing I’ll notice is “o hai all the things is broken halp plz”. Something like that.

When you’re troubleshooting DHCP, you’ll probably check systemctl, leases table, maybe /var/log … But when you get around to checking dmesg, you may notice the root of the problem:

From dmesg -T:

[Sat Mar 17 03:44:28 2016] audit_printk_skb: 18 callbacks suppressed

Considering the update occurred at 03:44am, this is probably our problem. I can’t see for sure, but I’m pretty certain those 18 suppressed callbacks are filled with “DENIED” log lines.

The fix is simple:

# aa-complain /etc/apparmor.d/usr.sbin.dhcpd 
Setting /etc/apparmor.d/usr.sbin.dhcpd to complain mode.

And your isc-dhcp-server should now restart gracefully after it’s scheduled upgrades!

Making MindTouch's Dekiwiki work on Debian Stretch

Sun, 17 Sep 2017 00:00:00 +1200

We had accumulated a certain amount of technical debt, due to a Wiki solution that was selected a few years ago: Dekiwiki, by MindTouch. Unfortunately a few months after implementation, MindTouch Core (which Dekiwiki builds on) was well-and-truly deprecated back in 2013.

It all happened before my time, but it seems as if Dekiwiki came as a pre-built VMware Appliance, based on Debian Etch (Debian 4.0; released in 2007). After giving our Dekiwiki environment some serious side-eye for a while, I finally decided to get my hands dirty and try to upgrade it.

Jedd, over at Jeddi.org, wrote a really helpful post back in 2015 about his experiences updating the ageing version of Debian that the VM is based on. The whole Dekiwiki image is held together by sticky-tape, but that post is really helpful in getting Etch upgraded to Wheezy.

Here are a couple additional things I’ve had to do (please, please look at Jedd’s post above - he covers a whole lot that I’m not covering here):

Upgrading Etch -> Lenny:

Follow the guide at Jeddi.org above to get to Lenny. I can’t remember much failing for that upgrade step, but I remember something happened.

From Lenny -> Squeeze:

DNS Breaks. You’ll notice that SSH takes aaaages to log into, and other things will start being slow and complaining.

Resolve this by editing the ‘hosts’ line in /etc/nsswitch.conf:

hosts: files dns

Originally this reads hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4

Note I’m not actually sure this happened between Lenny and Squeeze; it may have been from Squeeze to Wheezy. *shrug*

From Squeeze -> Wheezy:

MySQL breaks. When visiting the wiki, you’ll be informed that it couldn’t access the backend, and that the API might be warming up. However, you’ll notice that the MySQL service fails to start. When you run ‘mysqld’ from command line, you’ll see it’s missing a folder.

Resolve this by running mkdir /var/lib/mysql-files ; chown mysql:mysql /var/lib/mysql-files

From Wheezy -> Jessie (upgrade-only):

Apache breaks. You’ll notice Apache starts serving just the contents of /var/www. This is because the Apache config file now looks for *.conf in /etc/apache2/sites-enabled; of course, the dekiwiki config file doesn’t have the .conf ending.

Resolve this by running mv /etc/apache2/sites-enabled/dekiwiki /etc/apache2/sites-enabled/dekiwiki.conf

From Wheezy -> Jessie (dist-upgrade):

Mono breaks. If you upgrade mono, Dekiwiki will fall back into unlicensed mode. The licensing engine will no longer be able to interpret license files correctly. You’ll see the error “Server license validation failed. The license signature is not valid.”

Resolve this by holding back the mono packages before you run a dist-upgrade:

apt-mark hold libapache2-mod-mono libmono-2.0-1 libmono-2.0-dev libmono-accessibility2.0-cil libmono-accessibility4.0-cil libmono-bytefx0.7.6.2-cil libmono-c5-1.1-cil libmono-cairo2.0-cil libmono-cairo4.0-cil libmono-cecil-private-cil libmono-codecontracts4.0-cil libmono-compilerservices-symbolwriter4.0-cil libmono-corlib2.0-cil libmono-corlib4.0-cil libmono-cscompmgd8.0-cil libmono-csharp4.0-cil libmono-custommarshalers4.0-cil libmono-data-tds2.0-cil libmono-data-tds4.0-cil libmono-data2.0-cil libmono-db2-1.0-cil libmono-debugger-soft2.0-cil libmono-debugger-soft4.0-cil libmono-dev libmono-firebirdsql1.7-cil libmono-getoptions2.0-cil libmono-http4.0-cil libmono-i18n-cjk4.0-cil libmono-i18n-mideast4.0-cil libmono-i18n-other4.0-cil libmono-i18n-rare4.0-cil libmono-i18n-west2.0-cil libmono-i18n-west4.0-cil libmono-i18n2.0-cil libmono-i18n4.0-all libmono-i18n4.0-cil libmono-ldap2.0-cil libmono-ldap4.0-cil libmono-management2.0-cil libmono-management4.0-cil libmono-messaging-rabbitmq2.0-cil libmono-messaging-rabbitmq4.0-cil libmono-messaging2.0-cil libmono-messaging4.0-cil libmono-microsoft-build-engine4.0-cil libmono-microsoft-build-framework4.0-cil libmono-microsoft-build-tasks-v4.0-4.0-cil libmono-microsoft-build-utilities-v4.0-4.0-cil libmono-microsoft-build2.0-cil libmono-microsoft-csharp4.0-cil libmono-microsoft-visualc10.0-cil libmono-microsoft-web-infrastructure1.0-cil libmono-microsoft8.0-cil libmono-npgsql2.0-cil libmono-npgsql4.0-cil libmono-opensystem-c4.0-cil libmono-oracle2.0-cil libmono-oracle4.0-cil libmono-peapi2.0-cil libmono-peapi4.0-cil libmono-posix2.0-cil libmono-posix4.0-cil libmono-rabbitmq2.0-cil libmono-rabbitmq4.0-cil libmono-relaxng2.0-cil libmono-relaxng4.0-cil libmono-security2.0-cil libmono-security4.0-cil libmono-sharpzip2.6-cil libmono-sharpzip2.84-cil libmono-sharpzip4.84-cil libmono-simd2.0-cil libmono-simd4.0-cil libmono-sqlite2.0-cil libmono-sqlite4.0-cil libmono-system-componentmodel-composition4.0-cil libmono-system-componentmodel-dataannotations4.0-cil libmono-system-configuration-install4.0-cil libmono-system-configuration4.0-cil libmono-system-core4.0-cil libmono-system-data-datasetextensions4.0-cil libmono-system-data-linq2.0-cil libmono-system-data-linq4.0-cil libmono-system-data-services-client4.0-cil libmono-system-data-services4.0-cil libmono-system-data2.0-cil libmono-system-data4.0-cil libmono-system-design4.0-cil libmono-system-drawing-design4.0-cil libmono-system-drawing4.0-cil libmono-system-dynamic4.0-cil libmono-system-enterpriseservices4.0-cil libmono-system-identitymodel-selectors4.0-cil libmono-system-identitymodel4.0-cil libmono-system-ldap2.0-cil libmono-system-ldap4.0-cil libmono-system-management4.0-cil libmono-system-messaging2.0-cil libmono-system-messaging4.0-cil libmono-system-net4.0-cil libmono-system-numerics4.0-cil libmono-system-runtime-caching4.0-cil libmono-system-runtime-durableinstancing4.0-cil libmono-system-runtime-serialization-formatters-soap4.0-cil libmono-system-runtime-serialization4.0-cil libmono-system-runtime2.0-cil libmono-system-runtime4.0-cil libmono-system-security4.0-cil libmono-system-servicemodel-discovery4.0-cil libmono-system-servicemodel-routing4.0-cil libmono-system-servicemodel-web4.0-cil libmono-system-servicemodel4.0-cil libmono-system-serviceprocess4.0-cil libmono-system-transactions4.0-cil libmono-system-web-abstractions4.0-cil libmono-system-web-applicationservices4.0-cil libmono-system-web-dynamicdata4.0-cil libmono-system-web-extensions-design4.0-cil libmono-system-web-extensions4.0-cil libmono-system-web-mvc1.0-cil libmono-system-web-mvc2.0-cil libmono-system-web-routing4.0-cil libmono-system-web-services4.0-cil libmono-system-web2.0-cil libmono-system-web4.0-cil libmono-system-windows-forms-datavisualization4.0-cil libmono-system-windows-forms4.0-cil libmono-system-xaml4.0-cil libmono-system-xml-linq4.0-cil libmono-system-xml4.0-cil libmono-system2.0-cil libmono-system4.0-cil libmono-tasklets2.0-cil libmono-tasklets4.0-cil libmono-wcf3.0-cil libmono-web4.0-cil libmono-webbrowser2.0-cil libmono-webbrowser4.0-cil libmono-webmatrix-data4.0-cil libmono-windowsbase3.0-cil libmono-windowsbase4.0-cil libmono-winforms2.0-cil libmono0 libmono2.0-cil mono-2.0-devel mono-2.0-gac mono-4.0-gac mono-apache-server2 mono-common mono-csharp-shell mono-dmcs mono-gac mono-gmcs mono-mcs mono-runtime mono-utils mono-xbuild

Note: Some of these can certainly be unheld. I haven’t yet gone through and identified exactly which component is breaking on upgrade. If you put in the effort to determine the culprit, please let me know.

That’s all! You should now have Dekiwiki running on Debian Stretch! I’m planning to upgrade it to Debian Buster in the next week or two.

Respecting Amavis' "Banned Extensions" setting

Sun, 13 Aug 2017 00:00:00 +1200

I’ve been dabbling a little bit with iRedMail, mostly just to have a play with a mail server, but also to see what’s involved in mail security. iRedMail is a package that pulls together Postfix as an MTA, Dovecot as a POP3 & IMAP server, SOGo for ActiveSync, Roundcube for Webmail, SpamAssassin for spam protection, and ClamAV for virus scanning.

Okay I have no idea why I have to write this, but apparently it’s a thing.

Amavis has a list of banned file extensions. In Debian, they live in /etc/amavis/conf.d/20-debian_defaults, and /etc/amavis/conf.d/50-user, and are set in the $banned_filename_re variable. THIS MAKES PERFECT SENSE.

But of course, there’s always something that doesn’t make sense, and that is the fact that there is a SQL backend (at least in the environment created by iRedMail), and settings in here take precedence over the Amavis config files somehow.

And even more bizarrely, there exists in this SQL environment, a policy setting entitled “banned_files_lover”, which was set to “Y”. I shit you not. My only hope is that this only defaults to “Y” for postmaster.

To fix this, you need to hop into the database, and update the appropriate column in the policy table:

$ mariadb
MariaDB [none]> \u amavis
MariaDB [amavisd]> update policy set banned_files_lover="N";

And now, Amavis will obey your file extension filters!

Thankfully, I found this information at http://www.iredmail.org/forum/topic13147-iredmail-support-amavisd-passed-but-setup-at-ddiscard.html – I never would have found it otherwise!

Making Amavis work with ESET Antivirus

Sat, 12 Aug 2017 00:00:00 +1200

But of course, ClamAV has shown disappointing performance, and it would be really nice to use something more… commercially suitable.

To tie together mail receipt and scanning, iRedMail uses Amavis (strictly speaking, ‘amavisd-new’). Amavis uses ClamAV by default, but it comes with a bunch of configuration blocks to bring together other antivirus applications.

But although amavisd-new is stable and still maintained, some parts of it are really old. In particular, many of these av-scanner config blocks are… uhh… “deprecated”. There’s one particular entry for ESET that is dated 2002 - things have changed a lot in the last fifteen years. *shudder*

So, with the help of some documentation, I managed to piece together a code block that works:

['ESET File Security for Linux',
 ['/opt/eset/esets/sbin/esets_scan','esets_scan'],
 '--subdir --unsafe --unwanted --clean-mode=strict {}',
 [0,10,100],[1,50],
 qr/threat="([^"]+)"/m
],

That would be the end of the story, except I also had to hunt around to find out the right place to put my beautiful codeblock into. It turns out that Debian’s Amavis config structure is quite different to the CentOS config that is most-frequently mentioned in the iRedMail forums. I spent a lot of time playing with /etc/amavis/conf.d/15-av_scanners, and nothing seemed to work. Eventually I found out that Debian features a /etc/amavis/conf.d/50-users file that overwrites the settings from 15-av_scanners. Finally I had progress!

Somewhere around line 154 in /etc/amavis/conf.d/50-users, you’ll find an @av_scanners codeblock. I deleted the ClamAV section in there, and replaced it with the ESET codeblock above. I left the ClamAV settings in the @av_scanners_backup section, because Amavis will fall back to that if ESET fails.

That seems to be all! At least, it works with the EICAR anti-malware test file.

RingZer0team CTF - Challenges 86, 87, and 88

Thu, 13 Jul 2017 00:00:00 +1200

There are a bunch of fantastic Capture The Flag security challenges on RingZer0Team.com. I’ve been working through some of these for a wee while now, and with the New Zealand Cyber Security Challenge coming up again soon, I thought I’d get back into some of them.

Challenge 86 (“1/3 Do not waste the environment”, under the Forsensic Challenges) is one of a series of challenges where you need to dig through some provided data to find the flag.

I started by downloading the ‘forensic bundle’, which was just a large zip file. The first challenge was to figure out the contents of that zip file. The file name was just a jumble of characters, and there was no extension. Running file on it just returned the file type “Data”… not very enlightening, but head uncovered the string “VBOX” in the file… okay, it’s VirtualBox file.

Because I didn’t have Virtualbox installed, I spent a bit of time digging through the actual Vbox file itself, trying to see if there’s a flag already in there. I used strings | egrep -i flag-...., to try to find any flags embedded in the file itself. I didn’t find any flags, but I did discover that the computer name itself is “FLAG-PC”. Very clever. :-|

Update, many hours later:

Okay this totally cooked my bacon. After hours of playing with VirtualBox and playing with files, I finally gave up and Googled the name of the downloaded file.

Turns out I was REALLY close. The flag IS embedded in the actual file itself, but the format is different for the first time. The correct grep string would have been egrep -i '.?f.?l.?a.?g.?-'.

Real frustrated by how close I was, I’ll have to expand my searches in the future. Hat-tip to @professormahi and his GitHub page.

Once I understood that, I also discovered the flag for challenge 87 (“2/3 Did you see my desktop?”), in the same manner.

I haven’t managed to get the flag for Challenge 88 (“3/3 Suspicious Account Password”) yet, but I do know what it is. I can see (by grepping for “Visited”) a bunch of visits to http://www.forensicswiki.org/wiki/Tools:Memory_Imaging… this thing is a memory dump. That explains now why it doesn’t work as a VirtualBox disk.

Next time I sit down here, I’ll find visit Forensics Wiki, and start digging through the memory dump for the passwords.

Update two

I decided not to put this up as a separate post, because once I knew what I was looking at, it was pretty easy. Again though, I relied on @professormahi’s work - but now thanks to him I have a little bit of experience with Volatility!

First, we get Volatility to scan the image and see what it’s dealing with:

$ volatility -f vm.vmdk imageinfo
Volatility Foundation Volatility Framework 2.6
INFO : volatility.debug : Determining profile based on KDBG search...
 Suggested Profile(s) : Win7SP1x86_23418, Win7SP0x86, Win7SP1x86
 AS Layer1 : IA32PagedMemory (Kernel AS)
 AS Layer2 : FileAddressSpace (/home/eric/Downloads/86/vm.vmdk)
 PAE type : No PAE
 DTB : 0x185000L
 KDBG : 0x82920be8L
 Number of Processors : 1
 Image Type (Service Pack) : 0
 KPCR for CPU 0 : 0x82921c00L
 KUSER_SHARED_DATA : 0xffdf0000L
 Image date and time : 2014-03-09 20:57:55 UTC+0000
 Image local date and time : 2014-03-09 13:57:55 -0700

Now, knowing the profile that we need to use, we can use it to just… dump the password table:

$ volatility -f vm.vmdk --profile Win7SP1x86 hashdump
Volatility Foundation Volatility Framework 2.6
Administrator:500:aad3b435b51404eeaad3b435b51404ee:31d6cfe0d16ae931b73c59d7e0c089c0:::
Guest:501:aad3b435b51404eeaad3b435b51404ee:31d6cfe0d16ae931b73c59d7e0c089c0:::
flag:1000:aad3b435b51404eeaad3b435b51404ee:3008c87294511142799dca1191e69a0f:::

And then, we grab that NTLM hash and pop it into an online NTLM decryptor service, and we’re away!

Staying Secure in Business

Wed, 28 Jun 2017 00:00:00 +1200

A friend asked a question today on Facebook. I started writing a reply, and it turned into a 700-word essay. Hate it when that happens…

The question was:

“If large multi-international companies are getting hacked in Europe who have millions if not billions of $$ and capacity to protect their IT systems - how can small/micro businesses protect their IT platforms and systems?”

Firstly - nobody is safe from an APT (“Advanced Persistent Threat”). The recent NotPetya outbreak was an APT - malicious actors hacked a Ukrainian firm that produced accounting software, and used their software’s update facility to literally deliver malicious code specifically to the users of this particular software package (“M.E. Doc”). Nothing’s keeping folks safe from that level of sophistication.

However, the exploit by which NotPetya spreads was actually patched by Microsoft in March. If people had updated their internet-connected devices, this virus would have been limited to only users of M.E. Doc.

Aside from that though, there’s actually a disappointingly simple answer to this. Security is actually relatively within reach, if we all follow these simple steps:

1) Update. Everything. At every opportunity.

That includes Windows Updates, but also Adobe Reader (annoying!), Java (double annoying!), anti-virus engines, etc. This is absolutely the top priority, and the cybersecurity industry has been trotting this line out for years, but people just don’t. Because annoying.

Real life incident: June 22, 2017 - Honda falls prey to the WannaCry worm, 37 days after release (four months after the patch was released), due to poor patching hygiene.

See http://thehackernews.com/2017/06/honda-wannacry-attack.html

2) Backup everything important.

The more important it is, the more places you should keep it. You should have at least three copies of every important file. THIS IS ANNOYING, but if you don’t do it, data loss will hit you. Something like Crashplan (a low-cost service that sends backups to the cloud), plus a regular (weekly?) copy to a read-only media (e.g. a DVD), should be enough for a small business.

Real life incident: Feb 1, 2017 - GitLabs suffers a major outage after data was accidentally deleted, followed by a sequence of discoveries that none of their five layers of backup or replication techniques “are working reliably or set up in the first place”.

See https://techcrunch.com/2017/02/01/gitlab-suffers-major-backup-failure-after-data-deletion-incident/

3) Use next-generation anti-virus software.

Many of the AV products these days include very clever ways of analysing the behaviour of unknown software, and literally just rolling back all the changes if the AV decides it’s not trustworthy. Webroot is a good example of this, at a very affordable price.

Real life incident: April 23, 2017 - the Adylkuzz virus started quietly spreading around the internet, and setting up Monero cryptocurrency mining operations on infected PC’s. Because the payload wasn’t destructive, it flew under the radar and was missed by many antiviruses for weeks.

See https://www.proofpoint.com/us/threat-insight/post/adylkuzz-cryptocurrency-mining-malware-spreading-for-weeks-via-eternalblue-doublepulsar

4) Be alert, cautious, and sufficiently paranoid.

The last - and hardest - step is to maintain a healthy sense of distrust; sometimes referred to as “sufficient paranoia”. When you receive an email, ask yourself: is this email in-character for this person? Is the spelling what I’d expect? Is it appropriate? Is it asking me to take an action (even if that action is just opening a file)? If so, is it expected, or is it out of the blue? If it’s out of the blue, why? What’s the story behind that?

Real life incident: Sept 30, 2011 - I received an email from a lady I’d once interviewed for a job. It was a distraught email saying she was stuck in Wales, had been mugged, lost all her money, and needed a brief loan until she got home. I wrote her an email (not a reply), telling her email had been hacked; lo and behold, I got a reply! “It’s me… this is for real, i have checked with the consulate but there is nothing really working out, most important is i don’t have enough money on please, please i need you to loan me some, i can pay you back once i get home.”

I ended up finding her cellphone number and sending her a message - she was at home, safe and sound.

See https://blog.malwarebytes.com/cybercrime/2014/08/email-hijack-leads-to-i-was-robbed-send-me-money-scam/ – this was exactly the wording I received.

Geez, that was a long Facebook post Eric.

It really was, and a few minutes afterwards, this tweet came to my attention:

https://twitter.com/josephfcox/status/879982828418719744

It’s fun to laugh, but it’s a good reminder that there’s always going to be a weakness. You need to structure your business disaster recovery in a way that you can quickly and efficiently deal with a complete wipe-out like this.

And we’re done.

So, to summarise this essay, you need to:

Always install updates
Always take backups
Use AV software with full journaling and rollback capabilities
Practice sufficient paranoia

With these four steps, you will be as safe as the biggest corporate. (roughly speaking)

RingZer0team CTF - Challenge 44

Sat, 24 Jun 2017 00:00:00 +1200

Challenge 56 (“Hey Chuck where is the flag?”, under the Forensic Challenges) started by only offering a .pcap file. I downloaded Wireshark and had a quick dig. The packet capture consisted of a brief browse of a “Chuck Norris Facts” website.

After quickly flicking through the pcap itself looking for low-hanging fruit, I used File -> Export Objects -> HTTP. This takes all the HTTP traffic, recombines the packets into files, and saves them.

Once I’d done that, I opened up the files and had a quick flick through. I checked the images for metadata etc but didn’t find anything. However I did fine a picture of Chuck Norris, appearing to urge the reader to keep searching files – in French.

When I didn’t find anything super-obvious in the image contents, I hopped into Bash and just ran grep -i flag *. It quickly returned the flag, stored in a php file!

Nice and easy hunt, great way to finish the day.

RingZer0team CTF - Challenge 65

Sat, 24 Jun 2017 00:00:00 +1200

This is a continuation of my series on RingZer0Team.com.

Challenge 65 (“Hide my ass in my home”, under the Forensic challenges) is a nice quick little forensic challenge. You get to download a .tar file, and do a bit of digging to uncover the flag.

This was a nice easy few minutes. After downloading the archive, I discovered a few files inside - including a particularly groovy electronica swing track, which I’m listening to again right now. Yeaaaaahhhhh.

Apart from a groovy mp3, the archive contains some dot-files such as .bash_history, a .jpg picture, an index.html, and some other bits.

I found the Flag in one of the dot-files; specifically, it was inside a Vim swap file that seemed to be leftover from a power failure or similar.

WireGuard - Part Three (Troubleshooting)

Mon, 12 Jun 2017 00:00:00 +1200

This is part of my brief series on WireGuard. I’m pretty enamoured with WireGuard and the way it works, but there were a couple slightly curly bits that I needed to get my head around. This troubleshooting guide is a rough dump of the issues I had, and how I fixed them.

Gotten Stuck?

At this stage, there are actually a few ways that this can go wrong, even though we haven’t done much. Think through all the bits:

Installed WireGuard at both ends
Set up your NAT rule on the remote side
Created a private and public key on each side
Put each public key in the opposite side’s [Peer] config
Put a suitable IP address on each side

Troubleshooting

There are, actually, a myriad of ways this can fail:

Broken routing on the client PC
Broken NAT on the remote router
Broken routing on the remote PC
Broken wg0 configuration on one side
Lack of TCP forwarding on the remote computer
Lack of Proxy ARP on the remote computer

If you can’t ping the remote server yet, don’t panic. Run tcpdump to find out what you’re missing.

On the remote server: tcpdump -i wg0
On your local machine: ping -c1 10.20.40.1

That will tell you whether your packets are reaching the remote server, or if they’re not getting through the tunnel.

If they’re not making it through the tunnel at all, you’ll probably be seeing error messages in the ping. For example:

PING 10.20.10.1 (10.20.10.1) 56(84) bytes of data.
From 10.20.40.2 icmp_seq=1 Destination Host Unreachable
ping: sendmsg: Required key not available

That’s the error message I got when I set AllowedIPs too strictly. Because I was trying to ping something that was routable, but wasn’t within the AllowedIPs range, there was no applicable key for the packet.

Thanks

Huge gratitude to Jason Donenfeld (aka zx2c4) for spending his time not only reading this post, but also for sending me some fantastic feedback! I’d made some bungles in my original post on this topic, and he vastly helped my understanding.

Also, huge gratitude to another Jason (@rendition) who has helped me develop from a junior network admin into a … ‘moderate’ network admin. I’ve learned more in the last year than I ever thought possible. He’s taught me nearly everything I know about managed networking, reviewed this post for me, and is actually the guy who introduced me to WireGuard originally!

WireGuard - Part Two (VPN routing)

Sun, 11 Jun 2017 00:00:00 +1200

This is a continuation of my brief series on the new WireGuard VPN. Part One was about the simple building-blocks to get WireGuard working between two endpoints. Now that we’ve got a couple machines able to ping each other by IP address, we can carry on a bit deeper into the inter-LAN routing stuff.

Extending on from the IP addresses in Part One, instead of JUST connecting to the remote machine, I want to actually have access to everything on the whole 10.20.0.0/16 network; even the non-WireGuard devices. I want it to be like I’m there on-site. The idea here is roughly:

Remote machine: wg0: 10.20.40.1 (behind public IP 163.172.161.0)
Local machine: wg0: 10.20.40.2 (with eth0 on 192.168.88.207)
Machines on remote LAN: 10.20.0.0/16
Remote LAN Watchguard range: 10.20.40.0/16 – note this is within the remote 10.20.0.0/16 range

From my local machine, with minimal interaction, I want to be able to ping something like 10.20.10.30, and get a response.

Happily, now that the two endpoints are talking with each other, there’s really not much that has to happen to get things working the way I want.

Remote Config

Many Linux distros disable IPv4 packet forwarding by default. But in this case, we very much want this enabled. Edit your /etc/sysctl.conf file, and look for the line: net.ipv4.ip_forward, and set the value to 1. You may need to add this line manually, or uncomment it.
You’ll also need to enable ProxyARP. This is another kernel setting that’s usually disabled, so edit /etc/sysctl.conf, and add the following line: net.ipv4.conf.all.proxy_arp = 1.
Reload your kernel settings. A reboot will do the trick, or you could just run sudo sysctl -p /etc/sysctl.conf.

Starting WireGuard on Boot

There are a variety options to do this, and if you’re a seasoned sysadmin you probably already have a favourite way.

One option is to add post-up wg-quick up wg0 to the tail of your eth0 block (or appropriate interface) in /etc/network/interfaces.

A second option (for systemd users) is to simply run systemctl enable wg-quick@wg0. This will tell systemd to bring the wg0 interface up once a network connection is established.

I only do this on the remote machine, because I don’t want my local machine to be forever spinning up it’s WireGuard connection; however I always want the remote machine listening for my packets. That said, there’s probably no reason to avoid this; I just haven’t done it yet.

Local Config

Here’s the really cool bit. Since your remote machine is now set up to perform IPv4 forwaring and ProxyARP, the only thing you need to change on the local side is a single number. Or rather, a couple numbers.

In your /etc/WireGuard/wg0.conf file, just expand the network range of your interface to include the entire remote LAN:
```
 [Interface]
 ...
 Address = 10.20.40.2/16

 [Peer]
 ...
 AllowedIPs = 10.20.0.0/16
```
Now reload your WireGuard config, either by rebooting, or running sudo wg-quick down wg0 && sudo wg-quick up wg0.

That’s all! When you make the above changes, wg-quick will modify your routing table so that any IP within 10.20.x.y will be sent over the wg0 interface. The remote host will dutifully forward the packet out into the rest of the network, and ProxyARP will take care of the rest:

$ ping -c1 10.20.10.31
PING 10.20.10.31 (10.20.10.31) 56(84) bytes of data.
64 bytes from 10.20.10.31: icmp_seq=1 ttl=63 time=91.8 ms

Troubleshooting

Okay so by now there’s a really good chance that you’ve bumped into trouble. I’ve started on a basic WireGuard troubleshooting guide, but it only covers the issues that I bumped into. If you’re still stuck after reading through that, ask a question on the WireGuard Mailing List, or reach out via IRC (#WireGuard on Freenode) - this is all on the WireGuard website.

Wrap-up

That should be all! When you reboot the remote server (I specify reboot, because you want to test that your solution can survive an outage at the remote site), you should be able to ping other things in the remote LAN without any additional interaction. I still manually run wg-quick up wg0 on my local laptop, because I don’t want to be connected remotely all the time.

Thanks

If you do end up using WireGuard, go forth and donate! Seriously, at very least, send Jason the cost of a local cup of coffee or a beer for his efforts.

WireGuard - Part One (Installation)

Sat, 03 Jun 2017 00:00:00 +1200

WireGuard is the most excellent VPN stack around. It’s really fast, the concept of Cryptokey Routing is awesome, and I love the speed and simplicity benefits that come from opionionated cryptography. The protocol is so simple - expressed in a mere 4k lines of code - that it’s auditable by anyone.

But.

With my initial naive approach, I found myself using HTTPS, over ports forwarded over SSH tunnels, connected over WireGuard. Although it was straightforward to get WireGuard working between two endpoints, I ended up in nested-crypto hell.

So, this brief series is about making WireGuard work as a VPN onto a different site. We’ll start by making it work between two endpoints, and we’ll go forward from there. My end goal is to have access to all the resources on a remote site, just by running wg-quick up wg0.

The endpoints I’m going to set up are at:

Remote machine: wg0: 10.20.40.1 (behind public IP 163.172.161.0)
Local machine: wg0: 10.20.40.2

NAT Setup

Chances are, your remote endpoint is behind a firewall of some sort. Pick a high port, and configure your firewall to forward UDP packets on that port through to your remote WireGuard endpoint. You don’t need to do this on your local side, because reply traffic from the remote side will generally be handled by the stateful session part of your firewall.

Many routers and firewalls offer port address translation (also known as PAT) as part of port forwarding or NAT. This is when a packet hits the firewall on (say) port 57432, and the firewall puts it on the LAN to port 22, for example. You don’t want this. I think it’s possible with WireGuard, but it adds complexity without benefit.

Config - Remote Site

Install WireGuard

Generate your keys. The following will create a public key and a stub config in /etc/WireGuard/:

 cd /etc/WireGuard
 umask 077
 printf "[Interface]\nPrivateKey = " > wg0.conf
 wg genkey | tee -a wg0.conf | wg pubkey > publickey

Edit your config to match:

 [Interface]
 PrivateKey = WhAt3v3R= (this is the private key generated on this machine)
 ListenPort = 12345 (this is the UDP port you've forwarded from your firewall)
 Address = 10.20.40.1/24 (this will be the IP given to the wg0 interface)

 [Peer]
 PublicKey = (leave this blank for now; you'll paste in your local public key here soon)
 AllowedIPs = 10.20.40.0/24 (this is the range of WireGuard IP addresses that this Peer's key can be used from)

That should be all you need to configure on the remote side for now. Save your wg0.conf file, and bring the interface up:
```
 wg-quick up wg0
 ping -c1 10.20.40.1
```
You should see a bunch of actions performed by wg-quick, and a reply packet from your ping. Now onto the local side.

Config - Local Machine

Install WireGuard and generate your keys, as per the first two steps above.

Edit your configuration again:

 [Interface]
 PrivateKey = WhAt3v3R+PaRt-tw0= (this is the private key generated on this machine)
 ListenPort = 12345 (this is the UDP port again; I don't think they have to be the same)
 Address = 10.20.40.2/24 (note - different IP address, but in the same range)

 [Peer]
 PublicKey = ?????????? (copy the public key from the REMOTE server here)
 Endpoint = 163.172.161.0:12345 (enter the PUBLIC IP address of the remote site, plus the forwarded port)
 AllowedIPs = 10.20.40.0/24 (specifying that packets using this key must come from within 10.20.40.x)

Now that you’ve got a local public key, take a second to paste it into the Peer public key section on the remote server.
Once you’re finished you should be able to bring the interface up and ping it:
```
 wg-quick up wg0
 ping -c1 10.20.40.2
```
… and you should even be able to ping the remote server as well:
```
 ping -c1 10.20.40.1
```

Gotten Stuck?

At this stage, there are actually a few ways that this can go wrong, even though we haven’t done much. Here’s a quick summary of everything we’ve done:

Installed WireGuard at both ends
Set up your NAT rule on the remote side
Created a private and public key on each side
Put each public key in the opposite side’s [Peer] config
Put a suitable IP address on each side

If you’ve nailed each of those and you’re still having trouble, you can have a quick look at the brief Troubleshooting guide I’ve put together as Part Three.

Onto Part Two

That should be all you need to get WireGuard working between two machines on two different sites. So far we haven’t done anything either interesting or uncommon - this is all the basic stuff you’ll find on the WireGuard Quick Start page, although expressed slightly differently. The next article will be a bit more about intra-site routing.

Thanks

If you do end up using WireGuard, go forth and donate! Seriously, at very least, send Jason the cost of a local cup of coffee or a beer for his efforts.

RingZer0team CTF - Linux Sysadmin challenges

Sun, 28 May 2017 00:00:00 +1200

The Sysadmin Linux series of challenges is where you’re trying to breach the security of a Linux system. I actually finished most of these last year, but I wanted to finish my last two. Of course, to get to the last two stages, you need to use the flags from the previous stages. So I’m revisiting them.

Level 1 - Morpheus -> Trinity

We start by SSH’ing into a particular user account on the ringzer0team server:

You have mail.
Last login: Thu Apr 27 02:52:40 2017 from <somewhere>
morpheus@forensics:~$

There’s only one file in the home folder, and I can’t read it. There’s a /home/trinity folder with full read access, but also nothing legible.

The flag for level 1 is found by running ps aux, which reveals what appears to be Trinity’s password:

root 3241 0.0 0.0 4188 572 ? S Jan14 1:44 /bin/sh /root/backup.sh -u trinity -p Flag-<redacted>

Level 2 - Morpheus -> Architect

Aha, and the flag for level two is in /etc/fstab, which contains what appears to be The Architect’s password, in base64:

/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0
/dev/fd0 /media/floppy0 auto rw,user,noauto 0 0
#//TheMAtrix/phone /media/Matrix cifs username=architect,password=$(base64 -d "<redacted>"),iocharset=utf8,sec=ntlm 0 0

Level 3 - The Architect

Level 3 requires you to log on as architect, using the password we discovered earlier. The only clue is “dig for password”.

This bit got a bit harder. There are a couple possible hints in architect’s mail file, including a bunch of attempts to mount a cifs volume, and reference to a file in /backup/.

Digging through the files in /backup left me at a loss. There are references to /tmp/Gathering.py, but that’s a challenge for later (cypher’s challenge). After some digging I located /var/tmp/.swl, which contained some MySQL credentials!

mysql> show tables;
+----------------+
| Tables_in_arch |
+----------------+
| arch |
| flag |
+----------------+
2 rows in set (0.00 sec)

mysql> select * from arch;
+------+-----------------------+
| id | arch |
+------+-----------------------+
| 1 | The one |
| 1 | The null one |
| 1 | The mother of the one |
| 1 | The father of the one |
| 1 | The flag of the one |
| 1 | The null one |
+------+-----------------------+
6 rows in set (0.00 sec) 

mysql> select * from flag;
+---------------------------------+
| flag |
+---------------------------------+
| FLAG-<redacted> |
+---------------------------------+
1 row in set (0.00 sec)

Yasss after a solid hour of digging, I’ve found the flag for the third level!

Level Four - Morpheus -> Oracle

Again we’re logging on as morpheus, but this time the goal is to access the oracle account.

Lots of tasty stuff under /var/tmp/.viminfo, now that I’ve found it. References to files in /backup/, to /tmp/Gathering.py, /tmp/mail_cypher

Don’t make the mistake of trying to cat .swo; it’ll eat your console session.

Found this interesting thing in mail, but it’s not helpful: forensics.localdomain : Apr 14 21:14:33 : morpheus : user NOT in sudoers ; TTY=pts/4 ; PWD=/home/trinity ; USER=root ; COMMAND=/usr/sbin/usermod -aG neo morpheus

In one of the /backup files, I found the remains of an SSH key, and it worked! Access to the oracle account achieved. In the home folder, we’ve got a handy-dandy Base64-encoded flag in flag.txt.

Level Five - Oracle Encrypted File

There’s a file in the oracle home folder named encflag.txt.enc:

U2FsdGVkX1+dCl4WEHNJKBqA8a4fQeheOgA7oiNmjwlJQvGaQAgqcIsGRIcbdHKF
heSs51JRSEmOLqVyGvoxDA--

Decoding the base64 returns a binary blob, prefixed with the string “Salted”. This is one of the levels that I never completed originally, so maybe it’s time to give it another crack.

A quick google of “salted base64” reveals an interesting comment on StackExchange. Apparently I’m looking at a file encrypted with the OpenSSL ’enc’ command, so I first copy the file to my own computer to work.

I discover that the command openssl enc -in testfile -d -a will un-base64 it, then load the file in. I need to pass it a password to decrypt with though, so now I need to find that.

Oh my god I just did cat .* in the oracle home folder and literally found an alias that reads and decrypts the file. I can’t believe that took me almost an hour.

There are no words.

Level Six - Trinity -> Neo

Okay getting serious now. Using Trinity’s account, I need to find the password for the neo account.

It’s a bit noisy on login:

You have mail.
Last login: Tue Apr 25 16:29:29 2017 from <le blah>
-bash: hello: command not found
Sup Neo!
ls: cannot open directory /home/neo: Permission denied
cat: phonebook: Permission denied

A vague tickle in my memory prompted me to run sudo -l right off the bat. Rewarded with:

User trinity may run the following commands on this host:
 (neo) /bin/cat /home/trinity/*

So of course I immediately run sudo -u neo /bin/cat /home/trinity/*

This gives us a ‘phone book’ containing:

The Oracle 1800-133-7133
Persephone 345-555-1244





copy made by Cypher copy utility on /home/neo/phonebook

Trinity’s .bashrc contains a bit of kruft:

hello neo
echo "Sup Neo!"
$(ls -lart /home/neo)
cat phonebook

Really not useful. I remember finding Trinity’s password with ps aux:

root 3241 0.0 0.0 4188 572 ? S Jan14 1:47 /bin/sh /root/backup.sh -u trinity -p Flag-08grILsn3ekqhDK7cKBV6ka8B

It looks a little bit like a MySQL connection string, so I try that but no dice. Nothing under /var/tmp this time, and grep doesn’t find anything useful under the /backup goldmine.

There’s some weirdness under /etc/passwd. I don’t know what this means, but saving it for later:

trinity:x:1002:1002:trinity,%,lsdf(940998+(n.~,3):/home/trinity:/bin/bash

I looked through the mail file, but nothing jumped out at me there. Also trawled again through /backup, as well as /var/backups. Then /etc and /var. Argh.

Finally I start looking for Persephone. Nothing helpful, but it reminds me of the last line in the phonebook.

copy made by Cypher copy utility on /home/neo/phonebook

And then I remember that the sudo command allows a * at the end of the cat command, which might allow me to do path traversal…

sudo -u neo /bin/cat /home/trinity/../neo/phonebook

Finally, after a whole hour of digging, I’m awarded Neo’s password!

Level 7 - Neo is Not Alone

I don’t know what this clue means, but I bet it has something to do with Persephone. Persephone comes from Greek mythology as one of Zeus’ daughters, and the goddess of the underworld. Does that help me here? I don’t know, it’s getting late…

I run cat .* in Neo’s home directory to see if there’s anything hiding there, but no luck this time. Also nothing for sudo -l.

There’s an unreadable file (owned by root) called ‘result.stat’, that’s probably related to something. I browse the mail history and see:

forensics.localdomain : Dec 7 14:08:08 : neo : user NOT in sudoers ; TTY=pts/2 ; PWD=/home/neo ; USER=morpheus ; COMMAND=/home/morpheus/egrep /home/morpheus/../neo/result.stat

That could be handy. I do remember an egrep in /home/neo, will have a look. Lots of reference to /bin/monitor, too.

I’m only in the neo group, so nothing to see there.

Running ps aux | grep neo shows a heap of /bin/monitor processes started by root, but running under neo. There’s also a “SCREEN” instance there, but running screen -D -RR doesn’t reclaim it, so something’s up there. I’m pretty sure there’s something there in /bin/monitor.

In all honesty this one had me beat. It was half-past midnight and I was exhausted. I knew that there was something special about /bin/monitor, and I knew there was something special about the fact that it was running as neo. But I took the expedient route and had a quick google. I found the answer at http://blog.dornea.nu/2016/10/30/ringzer0-ctf-sysadmin-linux/ - without this I could have been looking for years. However, as is so often the case, this tutorial was the basis of some great learning for me.

Because the process is owned by neo, and because /bin/monitor could be read by neo, it meant I could run a stack trace. I have never done this before - no false impressions here, I totally relied on that dornea.ru page to tip me off to the usage of strace. But once I knew that, I had the flag.

I was very well misled by this one, actually. The Persephone comment in the phonebook totally led me down the wrong path, and I spent quite a long time searching for her details on the system. It was the reddest herring possible.

Level 8 - Morpheus -> Cypher

Okay it’s late now so I’m going to try to thrash this one quick. Nothing in ps aux for cypher. Grepping /backup found a python file and a dump showing a cron job runs this file every minute. The contents of the python file are:

import os
os.system('ps aux > /tmp/28JNvE05KBltE8S7o2xu')

I can’t read the output file, but I can edit the python script! This is gonna be easy.

After a little bit of trial and error, I edited the python script to first list the contents of the /home/cypher folder, and after that, to output the contents of /home/cypher/flag.txt

It’s important to note that the python file is recreated every three minutes when it’s run, so you need to redo your changes for every iteration.

Fin

That’s the end! I’m quite pleased I’ve managed to finish all of the Sysadmin Linux section. Of course that’s also scary, because it means I need to break into the other sections. That’s OK though… it’s a good time to do it!

RingZer0team CTF - Challenge 148

Sat, 27 May 2017 00:00:00 +1200

Challenge 148 (“Sysadmin Linux Level 2”) is one of a series of challenges where you’re trying to breach the security of a Linux system. I actually finished most of these last year, but I wanted to finish my last two. Of course, to get to the last two stages, you need to use the flags from the previous stages. So I’m revisiting them.

We start by SSH’ing into a particular user account on the ringzer0team server:

You have mail. Last login: Thu Apr 27 02:52:40 2017 from morpheus@forensics:~$

There’s only one file in the home folder, and I can’t read it. There’s a /home/trinity folder with full read access, but also nothing legible.

Running ps aux reveals what appears to be Trinity’s password:

root 3241 0.0 0.0 4188 572 ? S Jan14 1:44 /bin/sh /root/backup.sh -u trinity -p Flag-

Aha, and /etc/fstab contains what appears to be The Architect’s password, in base64:

/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0 /dev/fd0 /media/floppy0 auto rw,user,noauto 0 0 #//TheMAtrix/phone /media/Matrix cifs username=architect,password=$(base64 -d “”),iocharset=utf8,sec=ntlm 0 0

Great! Now I can carry on to my actual goal, which was to nail challenge 91 (“Sysadmin Linux Level 7”)!

RingZer0team CTF - Challenge 57

Fri, 05 May 2017 00:00:00 +1200

This is a continuation of my series on RingZer0Team.com.

Challenge 57 (“Hash Breaker Reloaded”, under the Coding Challenges) is one of a series of challenges where you’re simply presented with a hash - you need to return the plaintext value to the page within 3 seconds.

In contrast to Challenge 56, this challenge also includes a salt:

You have 3 seconds to break this hash
Send the answer back using https://ringzer0team.com/challenges/57/[clear_text]


----- BEGIN HASH -----
ab9507edbb2501b3c02e47c51af0178d68655980
----- END HASH -----

----- BEGIN SALT -----
c2ac9d8d004b4011d0864e76c7ebaaccfd18464bb8ff66bdbf19a703eb95a944
----- END SALT -----

The hash looks like another SHA-1, but of course the online hash reversers don’t have the now-salted hash in their rainbow tables. I figured that the challenge was a simple continuation of the previous challenge, so I crossed my fingers and guessed the hash was simply a number (from last time), concatenated with the plain-text salt.

… And I was lucky! The below code got me the flag:

#!/usr/bin/python3
from requests import get
from hashlib import sha1
from re import search

url='https://ringzer0team.com/challenges/57'
session=dict(PHPSESSID='<blahblah>')
resp=get(url,cookies=session)
hash=resp.text.split("-----")[2].split("\t")[2].split("<")[0]
salt=resp.text.split("-----")[6].split("\t")[2].split("<")[0]
newhash=''
salt=salt.encode('utf-8')
x=0
while newhash != hash:
		newhash=sha1(str(x).encode('utf-8')+salt).hexdigest()
		print("%s = %s" % (newhash, x))
		x+=1
	
resp=get(url+'/'+str(x-1),cookies=session)
flag=search("FLAG-.{24}",resp.text).group()
print(flag)

As last time, there’s no error handling in that code, so if it runs for more than a few seconds it’s probably toast.

RingZer0team CTF - Challenge 56

Tue, 02 May 2017 00:00:00 +1200

Challenge 56 (“Hash Breaker”, under the Coding Challenges) is one of a series of challenges where you’re simply presented with a hash - you need to return the plaintext value to the page within 3 seconds.

Of course, this is impossible to do manually, so it’s a programming challenge.

The hash looks like a simple SHA-1, so I put it into an online hash reverser and discovered it’s just a SHA1 hash of a random number. SHA1 is really fast, so this should be simple.

Here’s how I tackled the problem:

#!python
# It's in Python3, because of reasons
from requests import get
from hashlib import sha1
from re import search
calculated_hash=''
x=0

# Your session cookie needs to go here
session=dict(PHPSESSID='<cookie contents>')

# Load the web page
url='https://ringzer0team.com/challenges/56'
resp=get(url,cookies=session)

# Extract the hash from the page contents
target_hash=resp.text.split("-----")[2].split("\t")[2].split("<")[0]

# Iterate hashes from zero until the calculated hash matches the target hash
while calculated_hash != target_hash:
		calculated_hash=sha1(str(x).encode('utf-8')).hexdigest()
		print("%s = %s" % (calculated_hash, x))
		x+=1

# If we got here, yay! We found the key. Now submit it back to the site, and print our magic flag
resp=get(url+'/'+str(x-1),cookies=session)
flag=search("FLAG-.{24}",resp.text).group()
print(flag)

There’s no error handling in that code, so if it runs for more than a few seconds it’s probably toast.

Making Lektor work with grsecurity

Sat, 29 Oct 2016 00:00:00 +1300

I started using grsecurity on my servers in 2015, and there’s always a bit of tuning required.

I was recently playing with Lektor (before I swapped to Pelican), and I had a bit of trouble with my grsec kernel. In particular, Lektor and Pelican are both run within a virtualenv Python environment, and grsec eats it like popcorn in two different ways:

TPE (Trusted Path Execution) throws a wobbly:

[253241.370019] grsec: From {ssh-origin-ip}: denied untrusted exec (due to file in world-writable directory) of /tmp/#50 by /usr/local/lib/lektor/bin/lektor[lektor:60593] uid/euid:1000/1000 gid/egid:1000/1000, parent /bin/bash[bash:60581] uid/euid:1000/1000 gid/egid:1000/1000

This block occurs because the virtualenv violates grsecurity’s Trusted Path Execution protection - the Python executable isn’t under a trusted path, and the user isn’t in the TPE-bypass group.

To resolve this one, add your user to the TPE group. This will allow the user to execute binaries which aren’t in trusted locations:

sudo usermod -aG grsec-tpe $USER

RWX protection. This is a common problem with Python-based apps, even though Python has an exception (Edit 2020-04-12 - Issue 6 from https://github.com/thestinger/paxd/ has been deleted) by default.

[ 207.534876] grsec: From {ssh-source-ip}: denied RWX mmap of <anonymous mapping> by /usr/local/lib/lektor/bin/lektor[lektor:534] uid/euid:1000/1000 gid/egid:1000/1000, parent /bin/bash[bash:477] uid/euid:1000/1000 gid/egid:1000/1000

First, create your PaX ELF headers, and then disable grsec’s MEMPROTECT extension on the included python2 binary, while you enable EMUTRAMP:

sudo paxctl /usr/local/lib/lektor/bin/python2 -c
sudo paxctl /usr/local/lib/lektor/bin/python2 -Em

Result!

Updating the modem and radio firmware on a Samsung Galaxy 5 (G900I)

Thu, 27 Oct 2016 00:00:00 +1300

A couple months ago, Check Point revealed their discovery of the Quadrooter vulnerability affecting the Qualcomm chipsets in oodles of Android phones.

I use –CyanogenMod– LineageOS on a Samsung Galaxy S5, so thankfully I received patches for three of the four vulnerabilities in only a few days.

However, that last vulnerability was part of a proprietary binary blob for controlling the Qualcomm LTE chipset, only patchable by Samsung themselves. It took a while for the Samsung updates to roll out, and then I got distracted for a while, but I finally downloaded my firmware. The package I’ve used is “G900IDVU1CPH3”, which seems to be compatible at least with the three NZ-based carriers, as well as all the Australian carriers.

There are a couple little issues with the firmware upgrade, easily fixed if you know how. I’m going to assume you know exactly how to do everything without help. I’m sure you’re pleased.

In particular, the version of Heimdall in the Debian Sid repositories (1.4.1-2) doesn’t quite work with the SGS5. It appears other people have had similar problems.

The beginning of fix for me came from turboyz on Github, at the bottom of this post: (Edit 2020-04-12: GitHub Issue 348 from https://github.com/Benjamin-Dobell/Heimdall/ has been deleted); however, he’s manually made a couple changes to BridgeManager.cpp, which appear to be no longer necessary. We can simply build it from source:

sudo apt-get install build-essential cmake zlib1g-dev libusb-1.0-0-dev git
git clone https://github.com/Benjamin-Dobell/Heimdall.git
cd Heimdall

# OPTIONAL: Remove the `if(NOT DISABLE_FRONTEND)` codeblock at the end of Heimdall/CMakeLists.txt
# I believe this just saves time compiling the GUI

mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make
cd bin

Now you’ve got a nice new Heimdall installation, with the approproate updates to allow it to talk nicely to the new Samsung bootloader.

I copied my new baseband files into the Heimdall/build/bin folder, just to get them all together. The last step is to flash your NON-HLOS.bin and modem.bin files:

./heimdall flash --APNHLOS NON-HLOS.bin --MODEM modem.bin

I happen to have an issue on my phone currently where, even though the modem and LTE drivers have both been successfully installed, and even thought the Quadrooter tester is no longer showing any vulnerabilities, my Baseband version on my phone is still shown as the old one. I haven’t been able to figure out why yet, but I don’t think I’ll worry about it for now. There have been reports that you should first do this with –no-reboot, wait for the update to complete, then pull your battery, then when you power back on, immediately go back into download mode and re-flash. Apparently this helps update the recognised Baseband version, but to be honest it didn’t help me.

If you’re completely 100% lucky and absolutely nothing goes wrong, you win! You should now have the most recent Samsung modem and LTE chipset drivers available for your phone… without having to roll back to the stock Samsung firmware.

Incidentally, if you want to save gigabytes and time downloading said firmware, and if your phone is precisely a Samsung Galaxy S5 (G900I), and if you definitely want firmware G900IDVU1CPH3, I’ve extracted the important files (modem.bin and NON-HLOS.bin) and stored them here:

https://www.ericlight.com/files/G900IDVU1CPH3_modem_LTE.zip
SHA256 hash = a2ab13063583f6e83a3c2d8b79521a59ba103dda30a24bddf9248dd5a25bff3c

I promise I haven’t intentionally backdoored them, but I make no warranties of any sort. They might just be pictures of my foot.

Good luck!

Using ECDSA certificates with Let's Encrypt

Thu, 27 Oct 2016 00:00:00 +1300

Let’s Encrypt’s Certbot will generate an RSA key by default. But we want to step into the new and exciting world of elliptic curve cryptography! Unfortunately Certbot doesn’t really roll that way, so there are a couple hoops to jump through first.

A word of caution: this post has been written in retrospect, some time after I actually got ECDSA working. That means there are bound to be squiggly little steps that I’ve missed, and I certainly should have provided screenshots or snippets that I’ve missed. Sorry.

This post assumes you’ve already installed Certbot. I had a working regular certificate from Certbot before I changed to ECDSA, so if you have problems following this from scratch, I do recommend trying that first.

For future reference, I’m running the latest certbot available in Debian unstable, which is version 0.8.1-3.

I got most of this information from Scott Helme’s website, which has been awesome.

In brief:

Generate yourself an ECDSA private key:

openssl ecparam -genkey -name secp384r1 | openssl ec -out ec.key

You can change the curve that you use, if you feel a bit wiggly about the controversy around the NSA & NIST degrading the quality of the curves. I don’t feel particularly wiggly about that, myself.

Generate a Certificate Signing Request (CSR) with your shiny new key:

openssl req -new -sha256 -key ec.key -nodes -out ec.csr -outform pem

That will give you ec.key (your private key), and ec.csr (your certificate signing request). Time to get Let’s Encrypt involved.

Create your certificate:

certbot certonly -w /var/www/html/ -d {your_domain} –email “{your_email}” –csr ./ec.csr –agree-tos

If everything goes perfectly, that should leave you with a new shiny set of certificates – quite possibly named something clumsy like 0000-cert.pem and 0001-fullchain.pem, or similar. Throw those into your nginx config and give it a test to see if it’s working.

Schedule your certificate renewals:

I had particular trouble with the renewal process of ECDSA certificates, because certbot renew isn’t compatible with custom CSR’s. You need to run certbot certonly to pass the –csr argument, and then you need to deal with the output yourself.

Even more irksome, the certonly function will fail if you ask it to renew certificates which already exist:

An unexpected error occurred:
OSError: [Errno 17] File exists: '/etc/letsencrypt/live/{your_domain}/cert_ecdsa.pem'
Please see the logfiles in /var/log/letsencrypt for more details.

There doesn’t seem to be any way to tell certbot to overwrite the old certificates automatically, so I created a /etc/letsencrypt/temp folder, and wrote up a really yuck cron job for it. If I were working on a production system I’d do something better, but this works for my lowly domain:

# Recreate certs under /etc/letsencrypt/temp
30 2 24 * * certbot certonly -w /var/www/html/ -d {your_domain} --email "{your_email}" --csr /path/to/your/ec.csr --agree-tos --non-interactive --webroot --cert-path /etc/letsencrypt/temp/cert_ecdsa.pem --fullchain-path /etc/letsencrypt/temp/fullchain_ecdsa.pem

# Backup current certs to home dir
31 2 24 * * mv /etc/letsencrypt/live/{your_domain}/cert_ecdsa.pem ~/certbackup/
31 2 24 * * mv /etc/letsencrypt/live/{your_domain}/fullchain_ecdsa.pem ~/certbackup/

# Move new certs to live folder
32 2 24 * * mv /etc/letsencrypt/temp/* /etc/letsencrypt/live/{your_domain}/

# Restart nginx
33 2 24 * * service nginx restart

And that’s all! It seems to be working so far, but I’m sure something is going to fail at some point – maybe I’ll hit my request limit, or the Let’s Encrypt service will be down, and I’ll lose my certs. If it happens too often I’ll come up with a nicer cronjob and update this post.