Deep Dive into the CreatorCon IoT Lab
How it Works, Behind the Scenes, Build Your Own
13 minute read
If you attended Knowledge 17 (or CreatorCon on the road in India or San Diego) and happened to be walking past certain breakout rooms, you might have mistaken the high pitched buzzing you heard for a fire or security alarm, and no one would blame you…that’s certainly what it sounded like. But nope, you just happened to walk past the IoT Intelligence and Automation with ServiceNow lab in progress.
If you didn’t attend Knowledge 17 or CreatorCon, here’s what you would have heard:
This is the sound of IoT devices going critical. The fuel tanks are empty, the plant/vehicle/something-really-important is shutting down and alarms are making sure that everyone is aware of just how dire the circumstances really are.
What happens next in this lab is the awesome part. Each device is configured to send event data to a service called Thinger.io which in turn sends information to ServiceNow. First, lab participants configure an event table to automatically open Incidents and assign them to a technician. This reduces the time to resolution but still requires a technician to get involved when the tank is empty. Next, teams of participants fully automate the process by building a workflow that sends commands back to the device when it runs out of virtual fuel. No technician required.
Take a look at this short highlight reel from CreatorCon San Diego.
We also gave away devices to the first teams that successfully automated the remediation process! Here are the teams from CreatorCon San Diego ‘17.
Throughout the rest of this post, I’m going to explore the details behind how this lab came into being. I’ll start by taking a closer look at how these devices communicate with ServiceNow and how ServiceNow sends commands back to the devices. Then I’ll get into the process of building the devices themselves (and how you can build your own!).
A Bit of Background
Let’s take a step back. You’ve almost definitely heard of IoT, or the Internet of Things. IoT devices range form Internet connected toasters and refrigerators to industrial equipment with dozens of components and sensors, each with its own connection to the Internet. But the real question is: what do we do with that data? How can we use the massive volume of information available to us to reduce outages and improve service levels? How do we automate the diagnosis and remediation of issues?
The IoT lab lays out a scenario where Internet connected devices provide sensor data about a fuel tank. There’s no actual fuel tank in the lab, but the conditions the lab simulates are similar to those you might encounter in the real world. As the virtual fuel tank drains, it sends status information to ServiceNow and it’s up to the attendee to build business logic to act on this information.
ServiceNow’s Role
It’s important to understand ServiceNow’s role in this IoT scenario. ServiceNow excels at being a system of action. In the lab scenario, storing the historical fuel level for a dozen virtual fuel tanks is not useful. It might be tempting to start dumping buckets of IoT data into your ServiceNow instance, but this is not the best use of ServiceNow resources and there are other services that are better suited for this use case. Instead, we want to be informed when a threshold is crossed, indicating that some action needs to be taken, and then leverage the power of the platform to perform some kind of remediation.
How the Device Communicates
The lab was focused on letting attendees leverage ServiceNow and all of the infrastructure was already pre-configured for you. There wasn’t time to focus on every detail behind how communication happens so let’s demystify this a bit.
During the lab, there were four major communication points.
- The device itself. Connects to the Internet via WiFi.
- IoT Master instance. When the device starts up, it communicates with the master instance to get configuration settings.
- Thinger.io. Thinger is an IoT platform that facilitates device management and bi-directional communication.
- Lab instance. The instance each team actually used during the lab to build business logic that refills the virtual fuel tank.
Let’s take a closer look at each step in the communication path.
Device to WiFi
Each device connects to a common WiFi SSID. Pretty straight-forward.
Device to Master Instance
After the device starts up and establishes a WiFi connection, it then communicates with the Master IoT instance.
- The device sends its own Mac Address to a Scripted REST API in ServiceNow.
- The Scripted REST Resource looks up the Mac Address in a device table and returns:
- Device ID, e.g. esp826623
- Rate of Drain
- Mute enabled/disabled
- Fuel warning %
- These values are used by the code running on the device to control how it operates.
This allows us to dynamically configure the devices without having to re-flash code every time we want to make parameter changes.
Device to Thinger.io to ServiceNow
After retrieving settings from the master instance, the device communicates exclusively with Thinger.io. Thinger.io then handles all communication with the ServiceNow instance.
When the fuel level reaches the defined warning threshold, the device calls a Thinger endpoint via the Client Library. Thinger then forwards this request on to ServiceNow via a simple HTTP Post to the REST Table API.
ServiceNow to Thinger.io to Device
In the final scenario, ServiceNow sends commands to the device. This is accomplished by sending a REST API call to the Thinger endpoint associated with this device. Thinger then handles the process of sending a command to the device.
Pulling it all together
Now that we’ve looked at each piece individually, here’s a look at everything working together.
- Device runs out of fuel and sends message to Thinger.io
- Thinger.io forwards information to ServiceNow via Table API
- IoT Incident is generated and kicks off Incident Workflow
- Incident workflow posts “refill” command to Thinger API endpoint for the specific device
- Thinger.io issues the “refill” command via the Client Library running on the device
The IoT Device
To help demonstrate a use case that is similar to a real world scenario, it was necessary to design a device from scratch. The device had to be:
- Inexpensive. There are 30+ devices in the room. We also want you to feel empowered to build your own without spending too much money.
- Simple. It needs to be capable enough to perform the functions required by the lab but not so complex that it’s difficult to troubleshoot and understand.
- Visually interesting. During the lab, attendees need to see the device “draining” from across the room.
- Wifi-enabled.
Fortunately, there are quite a few great options for prototyping devices exactly like this and the original creators of the lab were able to build a device with off-the-shelf components that are relatively easy to obtain. Here’s what it looks like:
The Parts List:
- Bread board.
- ESP8266 / NodeMCU. Arduino-compatible wifi-enabled MCU.
- Push Button.
- Piezo buzzer.
- Potentiometer.
- Servo.
- 3 LEDs - green, yellow, red.
- 3 150-Ohm resistors.
Building the Devices
Building one device is one thing, but building 30+ requires teamwork. I’ll let the pictures speak for themselves.
Geek Christmas. Boxes started arriving at the San Diego offices a few weeks before the CreatorCon event.
_
Spending some time experimenting by duplicating one of the original devices from K17._
_
Taking an incremental approach by testing LEDs before introducing more components._
Moving to a bigger space and enlisting some help.
Build fully under way.
That’s a lot of devices…
Getting Data from Devices
One of the challenges when working with IoT devices is dealing with connectivity and managing dozens or hundreds of devices (or more). Most IoT devices aren’t running full-blown operating systems so it’s not as easy as installing your favorite scripting engine and writing a shell script.
Instead of having to roll your own low level communication libraries, there are a number of options available for simplifying connectivity. We chose to use Thinger.io. Instead of getting into detail about Thinger here, I’ll cover the various Thinger touchpoints throughout the rest of this post.
Sending Commands To the Devices with Thinger.io
When the device fully “drains”, we need a way to send commands down to the device to refill it. This is simple in concept, but potentially challenging to accomplishb b b how do you actually reach an IoT device that may or may not be publicly routable on the Internet? This is where Thinger.io and similar services really shine. Thinger provides libraries for a number of platforms like Arduino and these libraries give developers a convenient way to handle our use cases:
- Send data to Thinger.io (and forward that data to ServiceNow).
- Send commands to the device.
The second scenario is what we’ll focus on. We want to send a command to the device to “refill” the virtual fuel tank.
How does this work?
Once the device starts up and initializes the Thinger client, the Thinger client library keeps the device in constant communication with the Thinger service. We can then send commands to the Thinger REST API and Thinger in turn sends the command to the device. The device detects the command and executes the code associated with that command.
Example:
Here’s the code that runs on the IoT device that registers a “refill” command:
(*thing)[“refill”] << [](pson & in) {
B if (in.is_empty()) {
B B B in = false;
B } else {
B B B if (in) {
B B B B B // Execute the command
B B B B B doRefill();
B B B }
B }
};
On the Thinger side, each registered device has its own REST API endpoint. When the device starts up and connects to Thinger, each command is added to the device’s REST API as a resource. Using the refill command as an example and assuming our device ID is “esp8266_josh”, we can then issue a command by making a REST call to an endpoint that looks like this:
POST https://api.thinger.co/v2/users/joshnerius/devices/esp8266_josh/refill
This is just a simple REST call. We don’t have to worry about where the device is or how to reach it, we just call the Thinger API and Thinger handles the process of sending the command down to the device.
Everything You Need to Build your Own
The best part is that you can build your own! Here’s what you need to know.
Parts List
Here are links to specific product pages for the parts we used. You can purchase these from your retailer of choice (you probably don’t need 650 LEDs), but this will give you a place to start.
Fritzing Diagram
Fritzing is an open-source application that can be used to design and document electronics prototypes. You can use the Fritzing diagram as a reference when building a device.
Download Fritzing here. Download the IoT Device Fritzing diagram here.
ServiceNow Configuration
The device is configured to create records in an IoT Event table. I’ve create a special version of the application used during the lab specifically for this blog post and for those of you that have a device at home you wish to interact with. You can download the update set from GitHub here: CreatorCon IoT App.
Once you’ve installed the update set, you will find a new application menu called CreatorCon IoT App.
Use the Thinger.io Properties module to configure the Thinger.io username and the access token to use when calling the Thinger API. The access token is not your Thinger.io password and must be created from the Access Tokens menu on the Thinger console.
The IoT Devices module takes you to a list of IoT devices that you can interact with. By default, there’s just a single device: esp8266_sn_iot. Opening one of these device records gives you the ability to interact with the device. You can Refill, Mute and Unmute the device using the provided UI Actions.
Thinger.io Configuration
In order to accept events from the IoT device, you’ll need to configure a Device and an Endpoint on the Thinger.io side.
First, get a free Thinger.io account. You can register two devices and four endpoints on a free account - plenty to get you started with your own version of this lab.
Create Device
You must create a device on the Thinger.io side before the device can communicate.
Set the Device Id to esp8266_sn_iot.
If you wish to call this something else, be sure to update the Arduino sketch accordingly.
Set a description of your choosing and click the Generate Random Credential button. Copy the generated value - you’ll need it to finalize the Arduino sketch before uploading it to your device.
Create Endpoint
An Endpoint in Thinger allows you to handle an event from the device in some way. In order to forward events from the Device to your ServiceNow instance, you’ll want to create an HTTP Request endpoint.
- Endpoint Identifier. Set this to esp8266_sn_iot. This identifier is used in the Arduino sketch when making endpoint calls.
- Endpoint Description. Set this to a value of your choosing.
- Endpoint Type: HTTP Request
- Request URL: POST https://your-instance.service-now.com/api/now/table/your_event_table
Click the Add Header button. Set Key to Authorization. Set Value to. Basic Base64EncodedCredential. You can use an online tool or combine “username:password” (username/password separated by a colon) and base64 encode the value.
Arduino Sketch
We’ve adapted the Arduino sketch we used in the lab for standalone operation. The core drain/refill logic is identical to what we used in the lab, but the startup/initialization code has been modified to remove the dependency on a master instance.
Instead, you can control the initialization settings by modifying the code in the sketch directly before uploading to your device.
GitHub: CreatorCon IoT Arduino Sketch.
Considerations when starting a new IoT project
The CreatorCon lab and device is meant to provide inspiration and give you an idea of how you might start leveraging ServiceNow for your IoT use cases. If you want to use ServiceNow in a real scenario, this lab and blog post can serve as a starting point.
There are a number of things to consider if you embark on an IoT project of your own.
Data volume
If you’re running an array of weather stations, each providing multiple updates per minute per sensor, it’s probably worth investigating an IoT platform optimized for handling Time Series data. Pushing all of that data into a simple table in ServiceNow will not scale.
Bi-directional requirements
Be sure to choose the IoT services that make sense for your use case. If you have devices that will check in only occasionally, it might be perfectly acceptable to communicate directly with ServiceNow. But even if data volume is not a concern, make sure you consider bi-directional requirements. Do you need to send commands back to the device? If so, how will that communication happen? A service like Thinger.io or AWS IoT may make things easier.
In Conclusion
Hopefully this post gives you a better understanding of how the IoT lab was put together and how you might undertake a similar project. This isn’t meant to be a demonstration of “the one right way” to handle IoT devices, but should be used for inspiration and as a starting point.
The device we used doesn’t have real sensors in it but you could easily adapt it by adding any number of real sensors to collect real data. Even if you don’t plan to start an IoT project at your company any time soon, this is all just a ton of fun and can be a good learning experience on its own.
Go forth and IoT.
Share this post
Twitter
Facebook
Reddit
LinkedIn
Email