The Dothraki translator is ready for production!

New site collections can now be provisioned with the Dothraki translator web part. The provisioned site will also be mobile friendly with Dothraki branding.

As this blog is being composed the Dothraki translation service has already been used by another team: Acando. So, feel free to use it 🙂

For future purposes this concept can also be used to translate other fictive languages such as Elvish, Valyrian, Hi Valyran and Klingon.

 

Here a  screenshot from the mobile phone. When you press the button and start speaking to the phone in English, the English output will be shown as text in real time on the phone. So will the Dothraki translation, which then also will be spoken out  through the speaker.

 

 

 

 

Here is a picture explaining the architecture of how this was developed:

The technologies used to develop this solution are:

– Office Dev PnP
– React
– Azure
– Azure functions
– Bing Speech API

 

 

 

 

How is this possible you may ask:

  1. We use the the dothraki bing speech api from our SPFx webpart for reading the users microphone and finding out what the user says and translate that to text.
  2. We send the text we get from the bing-speech api to the dothraki azure function that does the actual translation. The azure function then sends back the right Dothraki text to our SPFx webpart.
  3. The SPFx webpart uses browser native synteziser to speek the dothraki text you get back to the user.

Please contact Bouvet Penguins if you need access to the Dothraki translate api, we are happy to let you integrate the azure function in your app. (You need API key) Dont worry ! Your data is never stored any where, your voice is safe 🙂 (GDPR)

Click here to see the source control on GitHub:
https://github.com/paalnym/ACDC2018

The Virtual Raven

When the King needs to send a message to His allies, He relies on a network of ravens. Recently, a new apparatus has been created that allows Him to draft a secret letter without the use of birds. This mystical device is called a virtual raven and will only work when used by the King.

When the King gazes into the device, it opens up a secret window that lets Him enter a secret message, assign a messenger and a recipient.

The message is safely hidden away in a magical cloud that is inaccessible to anyone but the intended recipient. It cannot be read while in transit, and so it cannot fall into the wrong hands.

The code itself is displayed on the apparatus, and I can assure you that it is unreadable to human eyes. Only by using a magical device entrusted to the rightful Kings can the message be deciphered.

What follows is a technical description of how the messaging system is created. It is of no particular interest to anyone but the practitioners of the arcane arts. It is included here only by the insistence of the device maker.

– King Sven

Technical description

The application is developed as an HTML5 / Javascript app using a Koa server and the React framework. The source code is published to github at the following address: https://github.com/svenanders/aspc2018.

When deployed it creates a docker container using an Ubuntu image that I maintain and use for most of my web applications. The container is stored in Azure on the container registry.

In order to be able to easily deploy and scale the application, I set up a Kubernetes cluster using the Azure Cloud Shell and deployed it with two replicas. I also used a loadBalancer to expose the application with an external IP.

Setting up a Kubernetes cluster on Azure is amazingly straightforward. I used this guide to quickly get up to speed on the Azure specifics: https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough. Using the Azure Container Registry required me to create a secret to hold my credentials:

kubectl create secret docker-registry <SECRET_NAME> \
--docker-server <REGISTRY_NAME>.azurecr.io \
--docker-email <YOUR_EMAIL> \
--docker-username=<SERVICE_PRINCIPAL_ID> \
--docker-password <YOUR_PASSWORD>

In my deployment.yaml I expose the secret like this:

spec:
 containers:
 - name: virtualraven
 image: <REGISTRY_NAME>.azurecr.io/virtualraven
 imagePullSecrets:
 - name: <SECRET_NAME>

When building my docker image I tag and upload it like this:

PKG_VERSION=`node -p "require('./package.json').version"`
docker tag svena/virtualraven:$PKG_VERSION svenscontainers.azurecr.io/svena/virtualraven:$PKG_VERSION
docker push svenscontainers.azurecr.io/svena/virtualraven:$PKG_VERSION

On the node backend I use Koa to connect to a set of Azure functions that retrieves a set of valid Kings and messengers and encrypts any outgoing messages in combination with the face recognition API.

For instance, the message upload function looks like this (some parts of the receiver address is obfuscated intentionally):

A lot of work also went into the UX design to make it work efficiently on a cellphone. I wanted to use a serif font to fit with the theme and ended up using the free Cormorant Garamond font. I wanted to use a professional TypeKit font, but the licensing costs proved too extreme. The color scheme is carefully selected to project an image of power, while still being pleasant to look at.

Having a tight schedule? Fear not!

Hei bloggen!

In our last post, we presented our provisioning engine. Sadly, you don’t always have your loyal servant Podric Payne at hand, or a computer for that matter. Luckely, house Coudborne has a solution for those who cannot wait to wage more war!

The PowerApp is pretty simple: it adds new items to a list. This action triggers a flow, which sends a http – request to the flow discussed in our previous blog post, which again provisions a group.

 

The old flow is updated to check for an email address in the payload, and sends an email to said adress with the relevant information.

 

 

 

House Cloudborne (first, second and third of the Andals).

Up and running!

The first message

Westeros has finally seen the sending of the first message between 2 kingdoms. Kings are happy with the security validation and magic behind the magic mirror “Surface” and the magic crown “Hololens”.

King Sven Anders is ready to send a message using the magic mirror. It is blocked, only a king can unlock it.

He looks directly to the magic mirror and it recognizes the king! It then asks for instructions from the user, and he writes “Send a message”. The strange device answers with another question “What message do you want to send?”, so the king types it down.

Finally, the mirror asks for the destination and the messenger who will deliver.

What is behind?

The first step is to athenticate the user, the device takes a picture every 5 seconds to check if a face is present, and if it is then it validates if it is a king’s face.

Once authenticated, a bot starts the conversation, asking for simple questions. The user supplies the answer by using handwritten text, which is sent to Computer Vision services in order to convert it to data that the bot can manage. Once the text is written and the king and messengers are selected, the bot sends the info to an Azure function that receives the parameters and starts the encryption process which goes like this:

  1. A new record in an azure table is created that holds: The unique key from the sender (King A), The unique ID of the messenger, the unique ID of the destination (King B).
  2. Encrypts the original text using RFC 2898 and generates a QR code image that goes to printing. (The original message and encrypted message are never saved in any place, only printed in paper, if the paper is lost, the message is lost).

Lot of trial and error

In addition to handwritting, we created also a mobile app that runs the same logic but without a bot. It runs under NodeJS in a docker container. The QR code can also be sent this way without paper.

The first receiving

King Carlos receives the messenger in his realm, he proceeds to wear his magic crown and looks at the messenger’s face while saying “Validate messenger”. Magically he receives information from a ghost called “The cloud” than can only be seen using this magic crown.

This info confirms the sending from Sven Anders (King A) and also confirms the messenger’s name.

What is behind?

By using the hololens the user can send voice commands to call certain functions in the device, some of the commands will make holograms react while other commands execute a HTTP POST to an azure function.

The first validation is the messenger’s face. Once the person is scanned, the request is sent to the defined endpoint and it returns the info about who send him and who is he.

The second validation is executed by sending the voice command “Show me the message”. It sends a second HTTP POST, but this time asking for the decrypted text. The process goes like this:

  1. The message is sent to an azure function, it receives the byte array (picture) and the receiver key (Kings B unique ID).
  2. It proceeds to decrypt the message by using the receiver ID (which is unique for each hololens, it is linked to the devicename). It means that only that specific device (King) kan get the decrypted text.
  3. Once the info is returned, it is shown in a ghostly hologram that we call “The cloud”.

As a cool addition, we added some voice commands to animate a couple of dragons if the messenger is not validated.

Increasing sales with the three-eyed raven

Understanding your customer, so you can provide the best matching offers, and increase sales is always a top priority. Therefor we have deployed three-eyed raven technology® on our customer facing website isitwinter.today.

The third eye is watching, identifies the visitor(s) and suggest the best offers matching offer(s), targeted to the audience.

Technology wise we are leveraging cognitive service and client side scripting to provide an efficient targeted marketing solution.

And as a footnote, since typescript, react, vue, etc. now is mainstream, everything is implemented in Vanilla JS (God we miss the hipster badge)

Are your house GDRP Compliant?

As a house of Westeros it is important to know if your house is GDPR compliant, and if you have control over your data.

We have created a Microsoft Form survey for you to take, to be able to score your GDPR Readyness.

The completing of the form will trigger a Microsoft Flow :The Flow will send your info to a custom CRM Entity, which we then can use to calculate your GDPR Readyness, and start a sales process specially targeted to help you reach full GDPR Compliance!

PowerApp – GDPR

GDPR states that users should be able to see what data we have stored about them, and to choose to delete or anonymize it. Unfortunatly, anonym is not an option in Westeros. You are either a person or a living dead.

Anyway, we have created a Powerapp listing all the contacts we have in our system:

 

The app is available for all customers and has a list with name and title. (OK, we know this should only be you!) If you want to see all the information about a contact click the contact for the detailed view:

The detailed view gives you the option to edit information:

Where you as a user could choose to be forgotten:

And since you are either dead or alive, you then actually became dead.. Shame..

The status “Forget me” Will trigger a CRM Workflow that will anonymize the user.

And tada, you exist no more:

This is then reflected all through our systems.

Technologies that the House of CGI uses

These are the technologies , the House of CGI is currently working with to help our clients, be protected against all threats 😀

  • Azure IoT-HUB
  • Stream Analytics
  • Azure Service Bus
  • Logic Apps
  • Azure Functions
  • Dynamics 365 Service
  • Dynamics 365 Field Service
  • Flow
  • PowerApps
  • Sharepoint
  • Azure Cognitive services – Face api to detect persons and Azure Video Index
  • Azure Functions
  • Azure App Services
  • Augumented Reality
  • Amazon Alexa
  • Arduino for IoT
  • Azure Service Bus
  • Azure Queue
  • Azure Cosmos DB
  • Azure blob storage
  • Azure Key Vault
  • WebAPI
  • Angular2 Web application (Nodejs and angular-cli)
  • Github for version control
  • CI/CD integrated with Github

 

Westeros Battle Simulator (WBS)

Hei Bloggen 👋

We are pleased to announce the brand spanking new ⚔️ Westeros Battle Simulator (WBS)  – which enables you to easily command your armies against your opponents.
This system is one of the key components of the Westeros Battle Command System.

In the WBS you have an SPFx web part that renders your current army size and battle stats (graph extension data). If you have available XP you can spend it on leveling up your foot soldiers and dragons.

The WBS makes it easy to level up you army by selecting the units that you would like to level up and then click on the level up list command (also an SPFx extension).

The WBS has a system for battle commands.

Currently it supports “Go to war”. The command sends your army to wage war against your selected opponent – if you win you gain experience points that can be used to level up your army. The gold you loot is used for conscripting new soldiers and dragons.

The business logic that calculates victory/defeat is currently in development – we use the Microsoft Graph to update Group extensions for Experience Points and Gold that are gained after a battle is won.

More on the business logic will come in our upcoming posts.

 

⚔️ HOUSE CLOUDBORNE ⚔️

The game has started!

The kings are impatient, there have been a lot reports of many spies in the seven kingdoms. The cloud has started to show its magic to the different realms, The kings were able to try a very strange device that the elders call: “Surface”. It looks like a mirror, and it shows lines magically as a person puts fingers on it.

1.- Messaging

This “Surface” is Alive!, It is able to recognize who is watching it and magically asks question to know what the king wants to do.

First it asks for a message to be sent, once the message is written, it asks for the destination (the king who will read the message), and finally it asks for the name of the messenger who will personally deliver it.

 

2.- Start the ride

This “Cloud” knows magically what has been written in that weird mirror, it generates a very strange scroll with a ciphered message that is given to the messenger. He starts the ride to the destination kingdom. We hope he can make it!

3.- Validation

The message has arrived, the king has been provided with a strange crown that the elders call “Hololens”, it shows images that nobody can see except the one wearing it. The king is able to talk to it and he asks it to validate the messenger who has brought this letter. It is confirmed! the message comes from Winterfell and the text is magically deciphered and shown in a strange image that can only be seen by the king.

4.- Spying

8 days later, another messenger was sent to deliver the message to a different real. This time unfortunately he was intercepted by a king’s landing spy. He was killed, and the spy took the letter to deliver it himself and that way he would be able to know what the content was when the king deciphered it.

Khaleesi used the magic crown to validate the messenger, but this time the message was clear: He is a spy, kill him!. She asked Dracarys to take care of the spy and.

What is behind it?

1.- Messaging.

Microsoft Ink does the trick! it is an app that runs in the surface that will automatically take pictures every 5 seconds to see if there is someone in front of the screen, it works as the authentication mechanism. Of course, the information about the the messengers and kings have been previously saved using a power app. A unique key is automatically generated for every new user.

Once the person is authenticated, the surface asks for a text to be sent, the king is going to manually write this message, we really hope that his handwritting is good :).

It asks then for a destination and the messenger. So the king writes their names and they are validated against the info saved in Face API database.

There is an alternative way to provide messages, and it is via voice commands, we are still working on it 🙂

To make all of this work we are using:

  1. Power app (Initialization UI to register messengers and kings)
  2. Azure table (Storing)
  3. Key vault (security)
  4. Azure functions (Logic)
  5. Microsoft Ink (handwritting)
  6. Bot framework (for communication with user in surface)
  7. Cognition services
    1. Face API (authentication before writting message)
    2. Computer vision (translation from handwritting to data)
  8. Encryption utilities
  9. Kubernetes cluster
  10. Docker as container
  11. NodeJS
  12. Azure WebApp

2.- Validation

To validate the message and messenger we use an interface developed with Unity3D to be shown in the hololens. It is configured to activate the following features:

  1. Internet client
  2. WebCam
  3. Microphone
  4. Picture library

At the start, we show a hologram with the form of a scroll that will act as the communication system between the user and the system. Using voice commands we execute some methods to be able to call the azure functions published in Azure. The holograms react when commands like “Hello Cloud” or “Validate Messenger” are submitted.

The first action executed by the user is the validation of the messenger, The hololens executes a HTTP POST including the picture of the messenger and his unique key. This way the system can validate if that messenger was actually sent to that king. The magic behind the AI service (Face API) works very well.

If the messenger is validated then the king proceeds to read the text in the scroll. This is of course encrypted, so we execute another HTTP POST with the picture of the text to an Azure function that gets the text, sends it to Computer Vision OCR and decrypts the text obtained. It is then returned to the hololens and shown in the Text hologram, so only the king can see it.

Finally, we will be showing dragons that respond to voice commands from the user, like attack, or fly.