The kings are impatient, there have been a lot reports of many spies in the seven kingdoms. The cloud has started to show its magic to the different realms, The kings were able to try a very strange device that the elders call: “Surface”. It looks like a mirror, and it shows lines magically as a person puts fingers on it.
This “Surface” is Alive!, It is able to recognize who is watching it and magically asks question to know what the king wants to do.
First it asks for a message to be sent, once the message is written, it asks for the destination (the king who will read the message), and finally it asks for the name of the messenger who will personally deliver it.
2.- Start the ride
This “Cloud” knows magically what has been written in that weird mirror, it generates a very strange scroll with a ciphered message that is given to the messenger. He starts the ride to the destination kingdom. We hope he can make it!
The message has arrived, the king has been provided with a strange crown that the elders call “Hololens”, it shows images that nobody can see except the one wearing it. The king is able to talk to it and he asks it to validate the messenger who has brought this letter. It is confirmed! the message comes from Winterfell and the text is magically deciphered and shown in a strange image that can only be seen by the king.
8 days later, another messenger was sent to deliver the message to a different real. This time unfortunately he was intercepted by a king’s landing spy. He was killed, and the spy took the letter to deliver it himself and that way he would be able to know what the content was when the king deciphered it.
Khaleesi used the magic crown to validate the messenger, but this time the message was clear: He is a spy, kill him!. She asked Dracarys to take care of the spy and.
What is behind it?
Microsoft Ink does the trick! it is an app that runs in the surface that will automatically take pictures every 5 seconds to see if there is someone in front of the screen, it works as the authentication mechanism. Of course, the information about the the messengers and kings have been previously saved using a power app. A unique key is automatically generated for every new user.
Once the person is authenticated, the surface asks for a text to be sent, the king is going to manually write this message, we really hope that his handwritting is good :).
It asks then for a destination and the messenger. So the king writes their names and they are validated against the info saved in Face API database.
There is an alternative way to provide messages, and it is via voice commands, we are still working on it 🙂
To make all of this work we are using:
- Power app (Initialization UI to register messengers and kings)
- Azure table (Storing)
- Key vault (security)
- Azure functions (Logic)
- Microsoft Ink (handwritting)
- Bot framework (for communication with user in surface)
- Cognition services
- Face API (authentication before writting message)
- Computer vision (translation from handwritting to data)
- Encryption utilities
- Kubernetes cluster
- Docker as container
- Azure WebApp
To validate the message and messenger we use an interface developed with Unity3D to be shown in the hololens. It is configured to activate the following features:
- Internet client
- Picture library
At the start, we show a hologram with the form of a scroll that will act as the communication system between the user and the system. Using voice commands we execute some methods to be able to call the azure functions published in Azure. The holograms react when commands like “Hello Cloud” or “Validate Messenger” are submitted.
The first action executed by the user is the validation of the messenger, The hololens executes a HTTP POST including the picture of the messenger and his unique key. This way the system can validate if that messenger was actually sent to that king. The magic behind the AI service (Face API) works very well.
If the messenger is validated then the king proceeds to read the text in the scroll. This is of course encrypted, so we execute another HTTP POST with the picture of the text to an Azure function that gets the text, sends it to Computer Vision OCR and decrypts the text obtained. It is then returned to the hololens and shown in the Text hologram, so only the king can see it.
Finally, we will be showing dragons that respond to voice commands from the user, like attack, or fly.