The Virtual Raven

When the King needs to send a message to His allies, He relies on a network of ravens. Recently, a new apparatus has been created that allows Him to draft a secret letter without the use of birds. This mystical device is called a virtual raven and will only work when used by the King.

When the King gazes into the device, it opens up a secret window that lets Him enter a secret message, assign a messenger and a recipient.

The message is safely hidden away in a magical cloud that is inaccessible to anyone but the intended recipient. It cannot be read while in transit, and so it cannot fall into the wrong hands.

The code itself is displayed on the apparatus, and I can assure you that it is unreadable to human eyes. Only by using a magical device entrusted to the rightful Kings can the message be deciphered.

What follows is a technical description of how the messaging system is created. It is of no particular interest to anyone but the practitioners of the arcane arts. It is included here only by the insistence of the device maker.

– King Sven

Technical description

The application is developed as an HTML5 / Javascript app using a Koa server and the React framework. The source code is published to github at the following address: https://github.com/svenanders/aspc2018.

When deployed it creates a docker container using an Ubuntu image that I maintain and use for most of my web applications. The container is stored in Azure on the container registry.

In order to be able to easily deploy and scale the application, I set up a Kubernetes cluster using the Azure Cloud Shell and deployed it with two replicas. I also used a loadBalancer to expose the application with an external IP.

Setting up a Kubernetes cluster on Azure is amazingly straightforward. I used this guide to quickly get up to speed on the Azure specifics: https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough. Using the Azure Container Registry required me to create a secret to hold my credentials:

kubectl create secret docker-registry <SECRET_NAME> \
--docker-server <REGISTRY_NAME>.azurecr.io \
--docker-email <YOUR_EMAIL> \
--docker-username=<SERVICE_PRINCIPAL_ID> \
--docker-password <YOUR_PASSWORD>

In my deployment.yaml I expose the secret like this:

spec:
 containers:
 - name: virtualraven
 image: <REGISTRY_NAME>.azurecr.io/virtualraven
 imagePullSecrets:
 - name: <SECRET_NAME>

When building my docker image I tag and upload it like this:

PKG_VERSION=`node -p "require('./package.json').version"`
docker tag svena/virtualraven:$PKG_VERSION svenscontainers.azurecr.io/svena/virtualraven:$PKG_VERSION
docker push svenscontainers.azurecr.io/svena/virtualraven:$PKG_VERSION

On the node backend I use Koa to connect to a set of Azure functions that retrieves a set of valid Kings and messengers and encrypts any outgoing messages in combination with the face recognition API.

For instance, the message upload function looks like this (some parts of the receiver address is obfuscated intentionally):

A lot of work also went into the UX design to make it work efficiently on a cellphone. I wanted to use a serif font to fit with the theme and ended up using the free Cormorant Garamond font. I wanted to use a professional TypeKit font, but the licensing costs proved too extreme. The color scheme is carefully selected to project an image of power, while still being pleasant to look at.

Up and running!

The first message

Westeros has finally seen the sending of the first message between 2 kingdoms. Kings are happy with the security validation and magic behind the magic mirror “Surface” and the magic crown “Hololens”.

King Sven Anders is ready to send a message using the magic mirror. It is blocked, only a king can unlock it.

He looks directly to the magic mirror and it recognizes the king! It then asks for instructions from the user, and he writes “Send a message”. The strange device answers with another question “What message do you want to send?”, so the king types it down.

Finally, the mirror asks for the destination and the messenger who will deliver.

What is behind?

The first step is to athenticate the user, the device takes a picture every 5 seconds to check if a face is present, and if it is then it validates if it is a king’s face.

Once authenticated, a bot starts the conversation, asking for simple questions. The user supplies the answer by using handwritten text, which is sent to Computer Vision services in order to convert it to data that the bot can manage. Once the text is written and the king and messengers are selected, the bot sends the info to an Azure function that receives the parameters and starts the encryption process which goes like this:

  1. A new record in an azure table is created that holds: The unique key from the sender (King A), The unique ID of the messenger, the unique ID of the destination (King B).
  2. Encrypts the original text using RFC 2898 and generates a QR code image that goes to printing. (The original message and encrypted message are never saved in any place, only printed in paper, if the paper is lost, the message is lost).

Lot of trial and error

In addition to handwritting, we created also a mobile app that runs the same logic but without a bot. It runs under NodeJS in a docker container. The QR code can also be sent this way without paper.

The first receiving

King Carlos receives the messenger in his realm, he proceeds to wear his magic crown and looks at the messenger’s face while saying “Validate messenger”. Magically he receives information from a ghost called “The cloud” than can only be seen using this magic crown.

This info confirms the sending from Sven Anders (King A) and also confirms the messenger’s name.

What is behind?

By using the hololens the user can send voice commands to call certain functions in the device, some of the commands will make holograms react while other commands execute a HTTP POST to an azure function.

The first validation is the messenger’s face. Once the person is scanned, the request is sent to the defined endpoint and it returns the info about who send him and who is he.

The second validation is executed by sending the voice command “Show me the message”. It sends a second HTTP POST, but this time asking for the decrypted text. The process goes like this:

  1. The message is sent to an azure function, it receives the byte array (picture) and the receiver key (Kings B unique ID).
  2. It proceeds to decrypt the message by using the receiver ID (which is unique for each hololens, it is linked to the devicename). It means that only that specific device (King) kan get the decrypted text.
  3. Once the info is returned, it is shown in a ghostly hologram that we call “The cloud”.

As a cool addition, we added some voice commands to animate a couple of dragons if the messenger is not validated.

The game has started!

The kings are impatient, there have been a lot reports of many spies in the seven kingdoms. The cloud has started to show its magic to the different realms, The kings were able to try a very strange device that the elders call: “Surface”. It looks like a mirror, and it shows lines magically as a person puts fingers on it.

1.- Messaging

This “Surface” is Alive!, It is able to recognize who is watching it and magically asks question to know what the king wants to do.

First it asks for a message to be sent, once the message is written, it asks for the destination (the king who will read the message), and finally it asks for the name of the messenger who will personally deliver it.

 

2.- Start the ride

This “Cloud” knows magically what has been written in that weird mirror, it generates a very strange scroll with a ciphered message that is given to the messenger. He starts the ride to the destination kingdom. We hope he can make it!

3.- Validation

The message has arrived, the king has been provided with a strange crown that the elders call “Hololens”, it shows images that nobody can see except the one wearing it. The king is able to talk to it and he asks it to validate the messenger who has brought this letter. It is confirmed! the message comes from Winterfell and the text is magically deciphered and shown in a strange image that can only be seen by the king.

4.- Spying

8 days later, another messenger was sent to deliver the message to a different real. This time unfortunately he was intercepted by a king’s landing spy. He was killed, and the spy took the letter to deliver it himself and that way he would be able to know what the content was when the king deciphered it.

Khaleesi used the magic crown to validate the messenger, but this time the message was clear: He is a spy, kill him!. She asked Dracarys to take care of the spy and.

What is behind it?

1.- Messaging.

Microsoft Ink does the trick! it is an app that runs in the surface that will automatically take pictures every 5 seconds to see if there is someone in front of the screen, it works as the authentication mechanism. Of course, the information about the the messengers and kings have been previously saved using a power app. A unique key is automatically generated for every new user.

Once the person is authenticated, the surface asks for a text to be sent, the king is going to manually write this message, we really hope that his handwritting is good :).

It asks then for a destination and the messenger. So the king writes their names and they are validated against the info saved in Face API database.

There is an alternative way to provide messages, and it is via voice commands, we are still working on it 🙂

To make all of this work we are using:

  1. Power app (Initialization UI to register messengers and kings)
  2. Azure table (Storing)
  3. Key vault (security)
  4. Azure functions (Logic)
  5. Microsoft Ink (handwritting)
  6. Bot framework (for communication with user in surface)
  7. Cognition services
    1. Face API (authentication before writting message)
    2. Computer vision (translation from handwritting to data)
  8. Encryption utilities
  9. Kubernetes cluster
  10. Docker as container
  11. NodeJS
  12. Azure WebApp

2.- Validation

To validate the message and messenger we use an interface developed with Unity3D to be shown in the hololens. It is configured to activate the following features:

  1. Internet client
  2. WebCam
  3. Microphone
  4. Picture library

At the start, we show a hologram with the form of a scroll that will act as the communication system between the user and the system. Using voice commands we execute some methods to be able to call the azure functions published in Azure. The holograms react when commands like “Hello Cloud” or “Validate Messenger” are submitted.

The first action executed by the user is the validation of the messenger, The hololens executes a HTTP POST including the picture of the messenger and his unique key. This way the system can validate if that messenger was actually sent to that king. The magic behind the AI service (Face API) works very well.

If the messenger is validated then the king proceeds to read the text in the scroll. This is of course encrypted, so we execute another HTTP POST with the picture of the text to an Azure function that gets the text, sends it to Computer Vision OCR and decrypts the text obtained. It is then returned to the hololens and shown in the Text hologram, so only the king can see it.

Finally, we will be showing dragons that respond to voice commands from the user, like attack, or fly.

 

Westeros secure messaging

The three-eyed raven

The kingdoms of westeros are in conflict, there is conspiracy and treachery between them and it has been a big challenge to rely on the messages they receive from other kingdoms. How can they trust the messenger? How can they trust that the message has been actually sent by an specific realm?

An unknown source of power has been discovered by the kingdoms, it behaves like magic! They call it “The cloud”. The kings have been provided with a special device that is able to recognize spies and also decipher hidden messages in weird languages (encrypted).

Is there a spy? Is there a hidden message in a scroll?
 
We will soon find out…