Raspberry Pi Headless setup

Door Bene op zaterdag 2 januari 2016 09:30 - Reacties (3)
Categorie: -, Views: 4.373

Best thing of 2015? The Raspberry Pi Zero! Slower, less memory, too few interfaces for practical use and overal so much of a hassle to install I'd rather watch reruns of four month old golf matches. It's amazing and it's also only 5 euro.

But that installation. No ethernet and a mini HDMI. That's a real pain. I did the first few by setting up a fresh image on a RPI B (using Ethernet), then cloning that, but that was not a pretty method. There are some excellent blogs out there that explained to me how to open the image on the SD card, but they required a Linux machine with a card reader. The best solution was to configure the image file itself, so any new Raspberry Pi I load with said image can join my network without having to attach a monitor or an ethernet cable.

To enjoy such a pain-free installation on the RPI Zero or any other Raspberry Pi, you'll need the following:
• Any flavour Raspberry Pi. And the common stuff like power and a MicroSD card.
• The most recent Raspbian image. I use 2015-11-21-raspbian-jessie-lite.img
• The Raspberry Pi should have an USB stick based on the RTL8188CUS chipset. For example, the PiHUT 802.11n WIFI dongle, the EDUP N8508 or N8508GS WiFi stick or any of those other tiny, tiny WIFI USB stubs costing about 5 euro's. This should also work with anything based on the 8192CU, but I did not test those.
• Any Linux machine for editing the image file and running NMAP. Another RPI already running or a VM will work just fine.

Editting the image
Unzip the downloaded Raspbian image so you have the basis 1,5 gb .img file. Copy that to your Linux machine. We are going to mount the file like a filesystem, but before we can do that there is a small hurdle, this file actually contains two images: a small system for booting, and the actual Raspbian installation that we would want to modify. If we don't tell the mount command where our disk starts in the image file, it will fail with the "mount: wrong fs type, bad option, bad superblock" error. So first, we will determine at which byte offset mount will open the file. We use fdisk to show us the partition information found in the image file:


code:
1
2
3
4
5
6
7
8
9
10
11
12
root@linux:~# fdisk -l 2015-11-21-raspbian-jessie-lite.img

Disk 2015-11-21-raspbian-jessie-lite.img: 1.4 GiB, 1458569216 bytes, 2848768 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xb3c5e39a

Device                               Boot  Start     End Sectors  Size Id Type
2015-11-21-raspbian-jessie-lite.img1        8192  131071  122880   60M  c W95 FAT32 (LBA)
2015-11-21-raspbian-jessie-lite.img2      131072 2848767 2717696  1.3G 83 Linux


Okay. So every sector contains 512 bytes. The second disk starts at sector offset 131072 (second column on the last line). This means mount will have to open our image file from byte 512 * 131072 = 67108864 and up.


code:
1
2
3
4
root@linux:~# mount -o loop,offset=67108864 2015-11-21-raspbian-jessie-lite.img /mnt/
root@linux:~# ls /mnt
bin   dev  home  lost+found  mnt  proc  run   srv  tmp  var
boot  etc  lib   media       opt  root  sbin  sys  usr


Cake for everybody! That's half the job already done. Assuming you have a RTL8188CUS card, edit /etc/wpa_supplicant/wpa_supplicant.conf and replace it's content with the following:

code:
1
2
3
4
5
6
7
8
9
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
network={
    ssid="YOUR_SSID"
    psk="YOUR_WIFI_PASSWORD"
    proto=RSN
    key_mgmt=WPA-PSK
    pairwise=CCMP TKIP
}


You could also configure a static IP address, but for me that contradicts the purpose of being able to load the image onto multiple RPI's.

Since the RTL8188CUS has some erratic behaviour with power saving mode, we'll disable that in two seperate ways:

code:
1
2
echo "wireless-power off" >> /mnt/etc/network/interfaces;
echo "options 8192cu rtw_power_mgnt=0 rtw_enusbss=0" >> /mnt/etc/modprobe.d/8192cu.conf


There. Now leave the /mnt directory and "umount /mnt". "Burn" the image file to the SD card with whatever tool you favor - I usually copy it back to my Windows machine and use win32disk imager.


Booting..
Power it up. Check that the light on the WiFi sticks blinks like a disco. Now from the Linux machine, run an NMAP scan for new devices on port 22 on your subnet:

code:
1
2
3
4
5
6
7
8
9
root@linux:~# nmap -p 22 --open  192.168.1.0/24

...

Nmap scan report for 192.168.1.235
Host is up (0.0090s latency).
PORT   STATE SERVICE
22/tcp open  ssh
MAC Address: E8:7E:06:32:33:95 (Edup International (hk) CO.)


There! A fresh Raspberry Pi Zero, on a clean image.

Additionally, since we are headless, you should consider the following:
- Lower the amount of memory allocated to graphics: using raspi-config, set the memory split value to 16 mb.
- Disable the HDMI port and save power: add "/usr/bin/tvservice -o" to /etc/rc.local

Connect a Bluetooth lightbulb to Philips HUE

Door Bene op dinsdag 15 december 2015 23:45 - Reacties (4)
Categorie: -, Views: 4.793

Anyone who regularly buys gadgets online has seen them: cheap Bluetooth enabled RGB led light bulbs. Different fittings, different packages, different manufacturers. What they have in common is that they all cost about $15 each. They also all share the same problem: you can only control them with a dedicated iOS / Android app. But no API, no alternatives. That turns them from “supercool!” to “meh.”

In this blog I’ll attempt to link them to my Philips Hue system one way or the other. I’ll write up my research, implementation and code and I’ll probably end up with a situation that’s just about useable. Useable means good enough for my wife not to get annoyed.

First, let’s take a look at what we have on the table. On the left, there’s a bulb that has a Bluetooth interface. On the right is the HUE system. HUE uses Zigbee. They share the same frequency and the friendship stops there. There is simply no way in day to day ordinary hell that they could ever communicate. Whatever our solution, we will have to do what is basically done in any IT problem: we introduce a middleman. In this case, I’ll use the Raspberry Pi for a platform. It’s cheap, fast and as versatile as a roll of quality duct tape.

Approach

The BT bulb is the easy part here. I’ll start with a Bluetooth adapter (3 dollars!). I think I can sniff traffic and see how the bulb and the app communicate. I should be able to replay any traffic I find. HUE / Zigbee however is not cheap. I don’t think the whole Hue bulb -> Bridge communication setup has been fully documented and there are no alternatives for the official bulbs, so I would be the first one in. I thought I should at least be able to wire(less) something up that sniffs network queries out of the air (similar to the dash button approach) but the Zigbee sniffers cost up to 50 for a complete USB package or 20 dollars for just a chip. I know my limits and soldering up a whole board around a single controller is one of them and even then, I am not so sure I can isolate HUE bridge commands, let alone emulate a HUE lamp.

Anyone who has ever fiddled with HUE bulbs and code has run into the excellent JSON API. As a lousy alternative, I can poll and track changes in that. That might be a much more reasonable goal, to query on a specific lamp and copy that to my Bluetooth bulbs.

So, my shopping list is as follows:
  • One cheap Bluetooth bulb. I bought the Tanbaby “Magicblue” lamp, fully known as “Magic Blue UU E27 Bulb Bluetooth 4.0” on the well-known Chinese webshops. E 10 .
  • The cheapest Bluetooth USB dongle I could find. E 2.
  • I use the Raspberry Pi Zero for the extremely low power draw, but any linux machine will do. E 5.
  • Additional USB knickknacks: a 2 dollar USB hub, a 5 euro wifi stick and a 1 euro micro USB -> USB converter (which required some shearing).
That's about 25 euro's total.

zero_setup

The "Magicblue" bluetooth bulb

http://static.tweakers.net/ext/f/iMU5b2MQxdzfzJaBQdkLnDnC/full.png

Let’s see if we can make some sense of the protocol used between the bulb and the app first. If you have an android phone, in the hidden developer options you can “enable Bluetooth HCI snoop log”. This will log all Bluetooth traffic to btsnoop_hci.log, which in turn you can open in tools such as Wireshark. Don’t forget to disable this option once you are done to prevent your phone from filling up.

I set about a number of actions. First, I tried turning the light on and off a number of times.
http://static.tweakers.net/ext/f/AayqdTYz0BcB6XJNjQMZ9x2W/full.png

The above screenshot is from opening the log file in Wireshark, then sorting on the source. I'm assuming they took the cheapest approach possible so the bulb is probably a load of fire and forget messages. In the above screenshot, you can see the value being written: “cc2433”, to handle 0x000C. If you have no knowledge of Bluetooth working (like me), just consider a handle similar to a memory location. The value alternated between CC2433 (turning off) and CC2333 (turning on).

After that, I moved the lamp from red to green and back, also dozen times.

http://static.tweakers.net/ext/f/ySRzxsSEZda2PDd39mdrTjk6/full.png

You can see that the write command is being sent to the same handle as the on/off commands. The values all looked similar to this “56------00F0AA”, with the dashes being replaced by a 6 digit hex code. And what else is a 6 digit hex code? Exactly, web colors. The closer to a fully saturated color the hex code is, the brighter the lamp. Not as sophisticated as HUE, but very easy to use.

Some googling into these codes led me to a github page on the "saberlight". This light has almost similar handles and mode of operation, which adds to the suspicion that these lights use a common Bluetooth controller probably sold as a package to all these budget light makers. If you want to know more outside of the scop of this blog, the control message for the Magicblue light can be read by first writing the message “ef0177” to handle 0x000c, which tells the controller to update the values at handle 0x000f which can then be read. The status request layout is almost the same as in the saberlight link.

Prepping the Pi

Okay. So now I the knowledge I need to reproduce app commands. Next step is to configure the RPI to actually replay them.

I’ll be starting out with a clean Raspberry Pi, using the latest raspbian Jessie lite image. I’ll skip the part where I set up networking, apt-get update, run raspi-config and disable some things for lower power use.

After inserting the Bluetooth stick, I first check if Raspbian can successfully connect it:

code:
1
2
root@raspberrypi:~# lsusb
Bus 001 Device 004: ID 0a12:0001 Cambridge Silicon Radio, Ltd Bluetooth Dongle (HCI mode)



Good! Let’s install the required tooling first:

code:
1
apt-get install bluez



Reboot after installation (or, do a “hciconfig hci0 up”) and you should be able to run a scan for Bluetooth Low Energy devices:

code:
1
2
3
4
5
root@raspberrypi:~# hcitool lescan
LE Scan ...
FB:6F:13:A3:C1:98 LEDBLE-13A3C198
FB:6F:13:A3:C1:98 (unknown)
FB:6F:13:A3:C1:98 LEDBLE-13A3C198



LEDBLE-somenumber is how the Magicblue bulb identifies itself. You should recognize this value from the splash screen of the magicblue app on your phone. We’re more interested in the Bluetooth MAC value, which in my case is FB:6F:13:A3:C1:98

Now I try to turn the light on by sending the cc2333 (“turn on”) message:

code:
1
root@raspberrypi:~# gatttool -t random -b FB:6F:13:A3:C1:98 --char-write --handle=0x000c --value=cc2333



Great success! I could have added a picture but it’s like you imagine: the light merrily switched on to it's last known state.

Note that using the gatttool "char-write" command will not return a response, at most an error if something went entirely wrong during the Bluetooth connection. If you get no response, either in an error message or in actual changing of the light state, it could be that the last color set was “000000” or something very dark.

Code!

How to get this discovery into Python? I looked into a Bluetooth python library (Gattlib), but couldn’t get it stable. Every other request I would get “device is busy” notifications so I dumped that in favor of fire-and-forget GATTOOL commands. I could have tried bluepy which seems a little more alive (https://github.com/IanHar...lob/master/bluepy/btle.py).

First The HUE part. There are a lot of Python HUE libraries (http://www.developers.meethue.com/tools-and-sdks has a good list), but we’ll only need some basic functionality. I’m going with the one that claims little overhead: https://github.com/quentinsf/qhue/. It’s just some wrapper code, very simple. We'll require the Python requests library later.

code:
1
root@raspberrypi:~# pip install requests



Some research into polling the Hue bridge shows that you can easily poll 30 times per second. That’s a lot more then needed – the HUE lamps write their state into the bridge only every second or so, so the gain is little in relation to the stress we will be putting on the network, pi, etcetera.

Polling every two seconds seems sane. This means that if I’m setting up the Bluetooth bulbs to follow the HUE light, at most we will have to wait 2 seconds before the Bluetooth bulb follows suit.
HUE uses a different color scheme then HEX colors, a so called CIELAB map. The RGB converter class can help us convert CIELAB colors to HEX, but it’s not perfect.

All components are there. Now to tie them together in a python script. Basically, it polls the state of a single given HUE light. If the power state has changed, the Magicblue bulb follows suit. If the color has changed, the conversion code approximates an RGB color and has the Magicblue bulb change to that.

Results and conclusion

The proof is in the pudding..

What you see is one bluetooth bulb on the left and one HUE bulb on the right. Not shown in the video is the raspberry pi setup and the HUE remote control. First, I switch the HUE light on. Then, I change it to a red color. After that, I switch off. In all three situations the Bluetooth bulb follows after a few seconds.

As you can see, the bulb is slow to follow. This is due to HUE lights lagging in sending their updated status. Also, the color conversion isn't very good at red-heavy colors. In daily use, it's better to just configure a color and have them

My final code can be found here: https://github.com/b0tting/magicbluehue. It's just a proof of concept, all of the settings can be found in the first 20 lines or so. If you want to use it but can't manage to start it, drop me a message and I'll add some actual documentation.