Photo of Dan
Dan
  • Thu Dec 17 2015

These aren’t the droids you’re looking for

As a web developer, wouldn't you love to take control of your devices using Web Technologies? Imagine having a heartbeat monitor that you could access from your browser. Being able to connect to it, get battery levels and heartbeat statistics. We all know how easy it is to deploy websites compared to apps; why wouldn't you take advantage of being able to release and move faster. Or maybe you just love making cool shit with HTML and JavaScript? I know I do. Heck, we all love making cool shit.

What if I told you all of this is possible, and more! It's possible via technology called Web Bluetooth and you can use it today! The great thing is that it's not just heartbeat monitors that are using the Bluetooth Low Energy (BLE). Heartbeat monitors are cool and they serve a purpose, but they're not necessarily as fun as flying a drone or even making a BB-8 roll around your floor. Yes, these devices are BLE enabled, which means Chrome can connect to them.

By utilising the great work done by the Web Bluetooth Community Group, we're able to use JavaScript in the browser to talk to, and control, a BB-8 drone. Connecting physical devices directly to a browser and being able to do things with it; that's freaking awesome! We've been writing apps using Node.js for years now, by using Johnny Five, other programming languages, and even using Chrome Apps that have access to the full Bluetooth Stack, but the fact we can do it in the browser using pure JavaScript is so exciting.

Enough talk, check it out!

This BB-8 dances and changes colour to the Imperial March by using the Web Audio APIs to analyse the music as it plays, and using the Web Bluetooth APIs to communicate with the device itself. On every G note, we change the colour between red and blue, as well as change its heading between 90 and 180 degrees.

Web Bluetooth

Today, Web Bluetooth is only available in Chrome with the enable-web-bluetooth flag enabled. Depending on your OS, you'll get varying levels of support.

OSX

Partially supported, you can find and connect to devices but that's about it - i.e. not useful for active development right now.

Android

Android devices get further than OSX; you're able to connect to "services" and get "characteristics" from the devices, but you can't listen for events. These events could be vital for your device and therefore you again may not be able to develop using an Android device. If you do use Android Marshmallow (6), make sure to use the Chrome Dev version from the Play Store; if you're using Android Lollipop (5) you'll need to use Chromium.

Chrome OS (Chromebooks)

Chrome OS is the furthest along, and is the most supported of the platforms. In Chrome OS you can connect to the device, connect to services and get characteristics, as well as accessing event data. For my recent work with the Web Bluetooth APIs, I went and bought myself a £200 Chromebook; an added bonus is that it's great for testing WebRTC calling.

As this demo doesn't rely on events, you should be able to do this from your Android device if you're running pure Lollipop or Marshmallow.

Go give the fork a try from github pages - you'll need to make sure the website is being delivered over HTTPS otherwise it won't work - the Web Bluetooth APIs require what is called a "Secure Context" or "Trusted Origin" to be able to be used, like so many other APIs.

I'm not going to re-write what's in this excellent guide by François Beaufort in the Google Chrome team on how to interact with BLE devices. His guide covers the basics, how the APIs are Promise based, development tips and what's coming next. However, I will say that if you don't have a Chromebook, the nRF Master Control Panel Android App is excellent in giving you information that you wouldn't normally be able to see. It enables you to scan for all BLE devices, connect to one, and list all of it's services and the characteristics within each service.

If you don't know what a service is, it's the thing that ties features together, such as battery information or heart rates. Within a service you have characteristics, these could be battery temperature or charge level, or your heart rate. These are standard services, you can find some information on them at developer.bluetooth.org, or look at a specific service such as the heart rate service, where you can find the standard characteristics included. The great thing about this is that one website using the Web Bluetooth APIs would be able to talk to any heart rate tracking BLE devices no matter what the brand is, for example; that's pretty neat.

Unfortunately, the BB-8 doesn't use any of these standard services, and so you can only interact with it using services and characteristics with special UUIDs - figuring out what each service and characteristic does is part of the fun though. I was lucky enough that the work was already done by the Web Bluetooth Community Group on Github. With the help of the Sphero JS SDK released by Orbotix (the people that make Sphero) and their developer manual, I was able to add a few more commands to the JS API and understand how to make the BB-8 rotate without moving off the spot.

Take a look at the beginning of the communication with the BB-8 using JavaScript:

navigator.bluetooth.requestDevice({
  filters: [{
    //services: ['22bb746f-2bb0-7554-2d6f-726568705327']
    namePrefix: ['BB']
  }]
})
.then(device => {
  console.log('> Found ' + device.name);
  console.log('Connecting to GATT Server...');
  return device.connectGATT();
})
.then(server => {
  gattServer = server;
  // Get radio service
  return gattServer.getPrimaryService("22bb746f-2bb0-7554-2d6f-726568705327");
})
.then(service => {
  // Developer mode sequence is sent to the radio service
  radioService = service;
  // Get Anti DOS characteristic
  return radioService.getCharacteristic("22bb746f-2bbd-7554-2d6f-726568705327");
})
.then(characteristic => {
  console.log('> Found Anti DOS characteristic');
  // Send special string
  let bytes = new Uint8Array('011i3'.split('').map(c => c.charCodeAt()));
  return characteristic.writeValue(bytes).then(() => {
    console.log('Anti DOS write done.');
  })
})

You should be able to see a really nice promise based API. You can see in the requestDevice call that you have to specify a way of limiting devices around you; no, you can't just search for all BLE devices. However, you can limit by services which would enable you to limit your search to all heart rate monitors for example, or you can limit by name or a name prefix. In this case, I'm limiting by a name prefix of BB, which then returns a promise with the found device (on Android you'll get a picker UI, but on OSX and Chrome OS you won't yet). Using the device, you can connect to the GATT service (GATT stands for Generic Attribute Profile, which is used for BLE), after which you can connect to an advertised service that is then available for you to inherently use to get and set values and event listeners on characteristics.

I would encourage you to have a read of the Community Group's Draft Specification if you're interested in more information; it'll help you considerably in understanding the Bluetooth APIs that are available and coming soon.

Web Audio

For the Web Audio portion, I used the Pitch Detection sample from Chris Wilson to be able to figure out what note is currently being played in the audio playing from the <audio> tag; Chris has a load of other Web Audio Demos available on appspot. Originally, I used the Imperial March orchestrated by John Williams, but SME didn't really approve of its use on YouTube so it's now a variation on the Imperial March by Orchestra Cinema Paradise.

If you're not familiar with Web Audio, you can do remarkable things with the Web Audio APIs, from creating beats and instruments, to entire songs. You can even take an audio file and apply affects to that audio stream before its output to your speakers. But that's not how this demo works, this demo works by playing an audio file and analysing it as it's played. Then you're able to take the audio stream that's being played, using the <audio> tag, and connect it to an Audio Context Analyser.

var audioContext = new window.AudioContext();

//load audio file from html tag
var audioElement = document.getElementById('soundFile');
//creating source node
var source = audioContext.createMediaElementSource(audioElement);

//create analyser node
analyser = audioContext.createAnalyser();
//set size of how many bits we analyse on
analyser.fftSize = 2048;

//connect to source
source.connect(analyser);
//pipe to speakers
analyser.connect(audioContext.destination);

Once the Analyser has been created, you're able to call getFloatTimeDomainData on it to be able to get data about the currently playing track; this is done on a loop utilising requestIdleCallback, which enables us to know when the browser is available to be doing things internally. Utilising the results from getFloatTimeDomainData, the data is then passed into the autoCorrelate function from Chris' Pitch Detection example; the result of this is knowing which note and what pitch, and with what certainty the application has about the information.

This enabled the change of colour and the rotation of the BB-8 in time to whenever the note G was played. At best guess, G is the all important note.

The web is such an exciting place to be developing right now. Throughout this post I've only talked about Chrome, but Firefox OS also has BLE features which allow us to use JavaScript to interact with BLE devices. The API is a little different and generally is a little more advanced than what's in Chrome today, but is restricted to Firefox OS devices - which have recently taken a bit of a knock in where they're going. I'm sure the Web Bluetooth APIs will get pushed and added into other browsers over the coming months. Connecting the browser and BLE devices shows us how the way we interact with devices from the Web is changing and it's changing fast.

I have to say thanks to Paul Kinlan and François Beaufort from Google, for helping with questions while getting up to speed with BLE on the Web. Jan Jongboom has also been a considerable help - he wrote a great post about controlling a drone from Firefox OS, utilising the Physical Web to access a URL for the drone. Finally, thanks to Ruth John for showing me what's possible with Web Audio - check out her recent post about her VJing setup on 24ways, where she also talks about Web Audio and Web MIDI.

What are you going to do with Bluetooth Low Energy in your browser? Let me know in the comments.

web-audioweb-bluetoothbb-8spherostar-wars