Let’s be honest, jokes are normally funny because they are at somebody’s expense. Whether it’s about a mother-in-law, a profession or ‘an Englishman, Irishman and a Scotsman’ – somebody is always on the end of it and nobody like to be made fun of.
Now, these jokes are all well and good as long and you are not the subject – that said I’m a Cornishman and I get some stick over that but in general it’s just banter and I personally don’t find it offensive or oppressive.
Some of this I do find over the top and some people will find offence in just about anything (including this blog post I expect) but I do understand that we are more enlightened and have more appreciation of how these jokes can have a serious impact on others and we need to be mindful of that.
All that said, I saw a tweet yesterday which made me put my head in my hands;
Many of us know that when we send an SMS (aka text message) it is sent in plain text and can be read by anyone with sufficient access.
This is normally limited to your cell provider but there are hackers out there using readily available hardware to act as a cell tower and initiate a man-in-the-middle attack.
Now I don’t know about you but I seldom send a regular SMS message – I use WhatsApp most of the time. Not only does it allow group chats (very handy for communicating with the family – especially during Covid-19 Lockdown) but it also provides end-to-end encryption. This means that only the intended recipient(s) and myself can read the message content.
It’s not that I’m doing anything illegal of course, it’s all about privacy.
Now, before you start down the ‘what about terrorists and paedophiles’ route I’ve already covered my thoughts about that argument, normally spouted by Government officials when trying to justify an erosion of our privacy and freedoms, in a previous blog post.
The UK Government (and they are not alone by any means) are continually using this argument to attempt to force technology companies to weaken their encrypted messaging systems. They are calling for backdoors to be put in to allow ‘authorised’ agencies to access the data to aid with criminal investigations.
My argument is that should the tech companies relent and add these backdoors then the bad people will just use something else or, crucially, write their own. This would leave the rest of us on hobbled, insecure system with ‘authorised agencies’ trawling around peoples private messages attempting to justify their hard won access.
So what is involved in writing your own secure messaging system? Surely it’s not just an app on the phone. Surely there needs to be servers and things to receive and forward these messages to the correct recipients. How would a regular user set these things up?
Well, yes there does need to be some form of delivery system but that doesn’t mean we need to write it – there’s already one out there, it’s on just about every phone out there and it’s been tried and tested for years. I am of course talking about the humble SMS text message.
Hang on, SMS isn’t secure!
True, the way SMS works doesn’t add any encryption to the process – plain text goes in and plain text comes out.
But this doesn’t mean that we can’t encrypt the message before we send it and for the recipient to decrypt it at the other end.
Now, I consider myself as a pretty competent developer – but I’m not what I would think of as a “rockstar developer”. I won’t have any of the big tech companies banging on my door offering me a massive salary and stock options to work for them. But surely I could write a mobile app which would allow the user to send and receive encrypted messages over SMS. As it happens – I can, and did just that.
As a proof of concept (you’re going to hear that term a lot in this post) I wrote an Android application, using Xamarin, which handles Key Pair generation, Key Exchange via QR code, Encryption, Decryption and integration with the devices SMS functionality.
Encryption is handled by the Open Source Sodium.Core library which is a fork of libsodium.net (also Open Source) which is itself a wrapper around the well regarded libsodium library – yep, Open Source all the way down.
In the image below Bob is sending a message to Alice, they have already exchanged their public keys using the app to display and scan a QR code containing the required data.
Lets break this down a bit and explain, at a high level, what is happening here;
Inside the Shhh.SMS app Bob selects Alice as the message recipient and enters his message (he can add emojis if he likes – that all works too!)
After clicking Send devices SMS app is opened with the encrypted message ready to send
Bob will need to specify the recipient from his contacts
Alice receives the SMS just like any other and uses the SMS apps ‘Share with’ functionality to send the encrypted message content with the Shhh.SMS app
Shhh.SMS opens and if it can verify the message it displays the decrypted text
After passing the message over to the devices SMS application is only exists on Bob’s phone in it’s encrypted form.
When Alice receives the message it is only decrypted for viewing within the Shhh.SMS application.
Moreover, while the message was being sent it was encrypted using keys that only exist, securely, on the sender and recipients devices. The cell carriers involved in the delivery of the message have no way of decrypting it.
So what’s the point?
Let’s get this absolutely straight – I did not write this app so that bad people could communicate with each other about bad things. That’s not the point I’m trying to make here.
The point is that the encryption genie is out of the bottle and it cannot be put back in. It’s just math!
“…it’s a proof of concept and lacks polish but that’s not the point”
If the world Governments think that outlawing the use of encrypted messaging applications is going to stop bad people from using them then they are frankly deluded.
This app took me a couple of weeks of evenings to put together – sure, it’s a proof of concept and lacks polish but that’s not the point. The point is that if I can write this in a couple of weeks as an investigation, what could others with a more sinister mindset achieve?
The general public are, generally, good people. So why should we all be treated as if we are criminals?
When privacy is criminalized, only criminals will have privacy.
If you want to submit pull requests to fix problems or improve the app then please feel free to do so – but do bear in mind that it will never be my intention to release this as a production app via the Google Play Store.
By the way – did I mention that Shhh.SMS is only a Proof of Concept? Good, just checking 🙂
The police chief was forced to u-turn in his threat while the forces involved with the other two incidents say that the officers were ‘well intentioned but over zealous’ – but to my mind, that’s not the point.
The point is that there will always be people in a position of authority or power who overstep their remit – and when it come to our privacy that’s not a good thing.
So, I’ve had an idea for another privacy-focused application, this time aimed at mobile devices – Android in particular (I know that Apple are a little touchy about encryption apps – maybe I’ll venture into iOS at a later date).
Notwithstanding my desire to keep my skills up to date I knew that the project I have in mind would require a lot of platform specific logic. While Xamarin Forms can handle this I prefer to take the hit, roll my sleeves up and I opted for a native Android project instead – and that’s where the trouble/fun started.
If you go through the ‘New Project’ process below you will end up with an application which will look something like the one above;
Yep -just what I needed, an application with a slide out menu. Now all I need to do is to replace the default options with my own and then open the appropriate views when they are clicked – what could be easier?
While we all know that Test Driven Development (TDD) is a good idea, in practice it’s not always viable. It could be a time constraint, a resource issue or the project just doesn’t warrant it.
While TDD may sometimes be an option, unit tests themselves should really be considered to be a must. They will save you a lot of time in the long run and while they may not prevent you from going grey, ask me how I know, they will reduce your stress levels when bugs raise their ugly heads.
So, whether you write your tests before you write the code or vice-versa, if you are developing production code you really should write tests to cover it.
Now, one of the requirements for unit testing is the ability to mock out a components dependencies so that you are only testing the component itself.
Normally you would use the Dependancy Injection pattern to help develop loosely coupled systems but Xamarin.Forms has a few components which can be a fly in the ointment – one of these is the MessagingCenter.
If you have used ASP.NET in recent years you will probably be familiar with the appSettings.jsonfile and it’s associated, build-specific transformations, e.g. appSettings.development.json and appSettings.release.json.
Essentially these allow developers to define multiple configuration settings which will be swapped out based on the build configuration in play. Common settings are stored in the main appSettings.json file while, for instance, API Endpoint Urls for development and production deployments are stored in the development and release files.
At compile-time any values specified in the deployment version of the file overwrite those in the common version.
Simple – works well and we use it all the time without thinking about it. But what about Xamarin.Forms – it doesn’t have such a mechanism out of the box so how do we achieve this and prevent accidentally publishing an app to the App/Play Stores which are pointing to your development/staging servers?
Update: Well, there’s certainly more to Dynamic DNS than meets the eye – who knew.
Investigations led to the decision that I should put my hand in my pocket and spend my time better elsewhere.
Not that I opted for the overpriced offering from Oracle, signing up with NoIp instead.
Like many devs I have been known to host websites and services behind my home broadband router and therefore needed a Dynamic DNS resolver service of some description. But in my recent moves to limit my reliance on third-party services – including those provided by Google – I wanted to see what would be involved in creating my own service.
Why would I want to roll by own?
Over the last few years I’ve moved my hosted websites outside of my home network and onto services offered by Digital Ocean so I was only really using my DDNS provider for a single resource – my Synology NAS.
Now, in the past I’ve used DynDNS (an Oracle product) and while I’ve had no issues with the service it’s not what you could call cheap – currently starting at $55 a year. When a previous renewal came through, and after reviewing what I was using it for, I decided to let it expire and do without the access to the NAS from outside my network.
Have I changed my mind about withdrawing support for this app – no, I haven’t. Essentially my hand has been forced by Google’s recent decision to deprecate some of the functionality I was using to fetch nearby places as part of the ‘Add New Station’ wizard as well as requiring 64 bit support – the latter being little more than a checkbox and nothing that most users will ever notice.
Removal of the Places Picker
Prior to the update, when adding a new station a user could specify the location in one of two ways;
Select from a list of locations provided by the Google Places API and flagged as ‘Gas Stations’
Open a ‘Place Picker’ in the form of a map and drop a pin on the desired location
It is the second option which is now going away – and there is nothing I can do about it. Google are pulling it and that’s that.
When I was at NDC London in January I watched a demonstration of the OzCode extension for Visual Studio. Not only was it well presented but it highlighted some of the pinch points we all have to tolerate while debugging.
In return for a scan of my conference pass, i.e. my contact details, I received a whopping 35% discount off a licence and without completing the 30 day trial I was so impressed that I pulled out my wallet (actually the company wallet!).
While I don’t use all of the features every day there are a few that I use all the time – the first one is called ‘Reveal‘.
Consider the following situation:
At this breakpoint I’m looking at collection of View Models – but I knew that already what value am I getting from this window? There are over 600 records here – do I have to expand each one to find what I’m looking for? What if one has a null value that is causing my problem – how will I find it?
So, the last developers conference (if you could call it that) I went to was way back in 2001 when we had WAP phones and if you had a Pentium 4 computer you were some super-techno hacker.
Well, time have changed somewhat, we now have smart phones with more memory than we had hard drive space back then and I’m writing this post on a workstation with 8 cores and 32GB RAM (and this isn’t even the cutting edge). Add to that the cloud, driverless cars and social networks with billions of users (for better or worse) and we have come a hell of a long way.
Well, I decided to bite the bullet and shell out for a super early-bird ticket and get myself up to London for a few days to Geek-Out.
TLDR; The FillLPG for Android app has reached end of life and no further development or maintenance will be undertaken. It will remain in the Play Store for as long as the third-party FillLPG website remains online but I have no control over that.
The FillLPG for Android app was initially developed to scratch an itch – I had an LPG powered vehicle (I don’t anymore) and had stumbled on to the FillLPG website. I responded to a request from the website owner for someone to write a mobile and thought it would be a good little project to get started in this arena.
It has proved itself to be just that, initially written in Java and then migrated to C# using Xamarin this little app has helped me keep my skill levels up – or at least it did.
I receive a lot of SPAM regarding the FillLPG mobile app, mainly offering to increase my install numbers and ratings, but today new approach appeared in my Inbox.
I was reviewing your app FillLPG – LPG Station Finder, and it looks great. But it appears the app does not support new Android Pie. Without it, more than half of the users cannot use the app properly. I am an app developer and I can update your app in a couple of days if you want.
It may not be obvious but this is just SPAM, sent by a bot and I doubt that anyone called ‘Jon’ is actually involved in the process, let alone reviewing my app.
Well, I’m happy to say that I’ve found some time to do this and moved to .NET Standard at the same time – yep, the future is here.
The initial project was really a quick and dirty exercise to demonstrate the ease with which these applications can be developed. It didn’t really lend itself well for extension – it was only meant to be a proof of concept after all.
In the new project I have created a class library to handle the mechanics of the encryption and a separate project for the CLI. There is also a skeleton project for a UWP desktop application which I’m currently working on (and committed in error – but hey, I’m only human). The UWP application will be initially aimed at Windows 10 but the ultimate aim to for a version of this project to be installable on just about any platform, e.g. Mac, Linux, iOS and Android.
I was recently approached by a website owner, who had seen the FillLPG for Android application and wanted a similar mobile application, i.e. an app which displayed a number of points of interest on a map and allowed it’s users to select a location and have further details displayed.
With my experience with FillLPG I was happy enough that I would be able to create applications to render the map with the points displayed in the appropriate location. The big question is – where is the data coming from?
The current situation
The website in question is fairly dated, in the order of 10 years old, written from scratch in PHP (not something like Drupal or Joomla) and backed up by a MySQL database.
The existing database is somewhat badly structured (in terms of todays programming practices) with no foreign keys, endless bit fields for HasThis and HasThat properties and a disjointed table structure (properties which you would think would reside in one table are in fact in another – manually linked by an id value).
There is no API that I can access from the mobile application – it’s just not there and there is no resource to create one.
The way forward
So, how do I access the data return it in a sensible format?
After a bit of thought I decided that the best option would be for me to create a WebApi project for the mobile apps to access. This project will access a separate MySQL database (with a structure more in line with what I need) which will be populated & updated by a utility program that I will also develop and execute on a regular basis (this is not highly volatile information that changes frequently).
So why .NET Core? You don’t need that to develop a WebApi project!
Glad you asked and my initial reply would probably be – ‘Why not?’. As a contractor I feel it is vital to keep your axe sharp and your skills up to date. I’m starting to see a few contracts now for .NET Core so it makes sense to keep my skills relevant.
After come careful analysis I decided that the most cost effective hosting for this solution was on a Linux based server. Yes, I know I can do that on Microsoft Azure but there are far cheaper services out there offering Linux hosting, so I’m going to use one of those.
Now, the only reason I can even consider using a Linux host is because of .NET Core. This allows me to develop using .NET technologies and C# – my development stack of choice.
But would it allow me to do what I intended to do? Could I create a WebAPI application to allow the mobile applications to access the data? What about the ‘Data Shuttle’ utility that will populate and maintain the data between the website database and the new WebAPI one?
Well, I’m happy to say that the answer to that question is yes, it will – and it did.
I’m writing this post after developing the initial, server side components, i.e. the Data Shuttle and WebAPI, and everything it working well – you would not know from the outside that the endpoints are not hanging off an Azure instance.
There were some pain points along the way and I’ve not answered all of my questions quite yet, things like logging and error notification, but everything I need for a Minimum Viable Project (MVP) are now in place from a server side perspective.
I have a handful of posts drafted at the moment which will dive deeper into the development for this project but here are a handful of links that you may find helpful in getting started:
Following on from my post about the UK Governments campaign to erode our privacy by demanding that tech companies put back doors in their encrypted products, I have created a simple utility to demonstrate how easy it is for a reasonably competent developer to create their own using standard development tools and libraries.
Now, I’m not expecting the UK Government to take a blind bit of notice but the fact is that encryption is out there, it’s only mathematics after all, and it’s not going away. You cannot feasibly make maths illegal – although the US did classify encryption as a weapon until 2000 (and to some degree still does).
The course was a minute under four hours and took me a couple of evenings to get through, Cryptography is not the most stimulating subject but Stephen did his best to key the information flowing. At times I did feel frustrated at how he seemed to labour some points but the upshot is that by doing this the information did seem to get through and stick. During the course he slowly increased the complexity, developing and enhancing C# code to demonstrate the principles.
It is this code which I have used as a base to create the ‘Personal Encryptor’ (hereafter referred to as PE) – a command line application that can be used to generate encryption keys, encrypt and, of course, decrypt data into files that can be safely sent over the Internet. Only someone with the required public and private keys will be able to decrypt the file and view the original data.
I’ll probably put another post together shortly diving a bit deeper into the functionality and explain the contents of the output file – but I highly recommend you watch the above course as Stephen know the subject inside out and does a great job of explaining it.
Why would I need/want to encrypt a file?
Imagine the following scenario;
Alice and Bob want to exchange private messages with each other; maybe they are planning a surprise birthday party or sharing ideas about a new business venture. Whatever the messages contain, they are Alice and Bobs business and nobody elses.
Alice and Bob both download the PE application and copy it to a location on their Windows PC (Mac version coming soon).
They then use the utility to generate a Public and Private key pair – which will create two XML files.
They each send each other their PUBLIC keys (this is just an XML file and can be freely sent over the Internet or via Email).
Both Alice and Bob copy their PRIVATE keys to a safe location (maybe a secure USB key – or a normal USB key which is stored in a safe)
Now Alice wants to encrypt a file, a PowerPoint presentation for their new product, and send it to Bob
Alice uses the PE application to encrypt the file using Bobs PUBLIC key.
The PE application Digitally Signs the encrypted data using Alices PRIVATE key.
A text file is created containing the encrypted data and everything needed to verify the contents has not been tampered with and to confirm that Alice encrypted it.
Alice then emails the file to Bob as she normally would if she was sending a photo of her cat!
Bob receives the message and downloads the encrypted file to his computer.
Bob uses PE to decrypt the file by specifying the location of his PRIVATE key and Alice’s PUBLIC key.
The PE utility will check the digital signature using Alice’s PUBLIC key to confirm that it was signed with her PRIVATE key.
It will then check the integrity of the package to ensure that it has not been tampered with in transit
If all is well then the PE application will decrypt the file and write the contents out to a location that Bob has specified.
Bob can now endure enjoy Alice’s PowerPoint presentation.
Of course if Alice (or Bob) just wanted to encrypt a file for their own personal use and not for sharing it is perfectly feasibly to provide their own Private AND Public keys to encrypt the data. These keys will be required to decrypt the data.
And that’s it, privacy restored/reclaimed.
I can now safely download my Lastpass vault in plain text, encrypt it and save it to any cloud drive I like, secure in the knowledge that, as long as my private key remains under my control, nobody can decrypt it to access it’s contents. Nothing illegal there – these are passwords to legitimate sites (Amazon, Pluralsight, Microsoft, Apple etc) and need to be protected. A valid use of The Personal Encryptor.
Yes, at the moment it requires the users to have some familiarity with the Command Line but this project was always intended to be a proof of concept. The original aim was to explore encryption to enable me to implement it in an existing mobile Chat application.
Creating a simple GUI would certainly be possible – a simple Winforms or WPF application to collect file paths and call out to the command line utility shouldn’t take too long for a competent developer to write. That said, I’m probably going to focus my attention elsewhere.
While using the Microsoft libraries is perfectly valid in my opinion, I am aware that many people will wince just a little bit. With this in mind I intend to investigate using the libSodium Crypto Library which Steve Gibson is such a fan of – so that’s good enough for me.