Augmented vision project

Some of you may have seen this before on my wearable technologies class blog but I thought I would put it up here too. I had so much fun working on this project and would love to take it much further.

The story behind the…vision.

Like thousands being born without 3d vision, I want to experience distance. I thought I have a hearing aid so why not a seeing aid?

What would it look like if I were to make a device that allowed me, or the user, to experience visual information in a new way? Distance and heat seemed to be likely candidates. Safe from oncoming objects and fires or heat sources like machinery. I thought an infrared heat sensor is accurate enough and an ultrasonic distance sensor that would be relevant at a personal body distance of about 5 feet.

The components

-Arduino, Lilypad micro controller

-Arduino, battery holder. 1.5v to 5v, with surge protector circuit

-Lilypad, Tri-light (RGB led)

-Lilypad, green LED

-Ultrasonic sensor

-Infrared, heat sensor

Putting it together

I began by using the wiring guides I found on the following web sites to solder it together. I didn’t proto-board it or alligator clips as I was confident in my soldering skills and wiring map to go for it! (cocky bastard!) This was the tutorial and sample code for the Maxbotix Sonar LV EZ1 ultrasonic device. I found the sample in out sensor code and used the tutorials on the blog and meetings to get it working and map out the flashing to be appropriate to the distance it represented. This site got me wired in no time for the thermal sensor. The little infrared device is accurate to .02 degrees F! So if it mattered you can have a wicked degree of accuracy. These are commonly used in cars for the size and reliability and robotics. It is a very nice non-contact sensor indeed.

This site was a wealth of information but it leaves out the very important detail of pin association. In a nutshell, one must wire it according to the convention provided due to the internal components of the microcontroller. I had them wired on different pins and couldn’t get a reading. Thanks to Kate for getting me on track! I would have never figured that out. This was the only hardware hurdle I had. Once re-soldered it worked instantly.

The merging of the codes

That is a movie title for sure! I struggled quite a bit with tutorials and the like online and found they all had one thing in common, that they make assumptions as to the level of your knowledge going into this and that you have some basics… I do not. I had never been exposed to this before and was way out of my comfort zone. I understood the basics of code modifying, setup and loop (sort of) but anything I did with regards to following the wickedly confusing amount of information about coding didn’t work out and only lead to frustration and distain for the process. I had two functioning codes but couldn’t for the life of me figure out how to merge them.

I met with a classmate for an hour and we hammered it out. Borxu showed me how setup creates the conditions for the loop to happen. We transferred the commands and tried out samples along the way till it worked right. I assigned the appropriate pins and values to the sensors. It works like a charm! (I knew it would)

Wearing the device

There are two considerations for this section, first being the personal aspects and the second being the social ramifications of wearing a Borg-like device.

I am personally comfortable wearing the device as my vision allows for it. I found others who looked through the device found the light to be too bright and distracting. The light can be further defused but more importantly what happens is something the military has coined “helmet fire.” Essentially helicopter pilots who use a wearable computer screen with inflight information being broadcast in one eye and real world vision in the other get confused easily in the heat of combat with the brains ability to process two images at once and can briefly shut down causing the helicopter to crash. It is a real problem. My limited vision lends me well to this technology but the signals need to be very deliberate hence the choice and sequence of colours listed bellow.

Blue light is connected to the thermal sensor and is on when a temperature of 80f or lower is detected. The blue light turns red when above 80f is detected. I used this number, as room temperature won’t set it off. It loves the stove and after some time you can map out the heat in the air with it! It will pick up a hot cup of coffee or body heat at close range of 2-3 feet.

The green LED flashes according to distance so the closer the object the quicker the flashing, simple enough but effective.

Socially wearing the device is another story. Of course here at OCAD-U it I easily accepted and met with great curiosity and fanfare but in public people don’t understand what it is. Hearing aids are well accepted for their sleek, tight form factor so that I am sure is a huge part of why people are uncomfortable with it. I have been asked most of all if the device is recording them. It has no camera or the capability to record any of the incoming information. But just like a hearing aid it provides what is missing and gives the wearer new information in a usable way.

Further development

For the visually impaired the uses are obvious but for the completely blind new outputs could be used like a hat with vibration motors in key locations. For the fully sighted individual this could act as a real time safety device in hazardous areas of work like monitoring air conditions or detecting land mines. A tighter form factor would make this more user friendly and acceptable. Batteries could be integrated, rechargeable and be made smaller, components could be better mapped out and tightened up, wires and traces could be strategically embedded and hidden and waterproofing the device would be greatly beneficial. The IR sensor can see through rain but the ultra sonic wont like driving rain, too much interference.

Closing remarks

This was a very challenging and personal project for me. The narrative has been a lifetime in the making and the skill set I learned to assemble and code the device is stronger than ever before. The seeing aid is more than the sum of its parts. The abilities it gives me makes my vision more complete but also acts as a window into our very complex social behaviors and notions of personal space. This is an ongoing experiment and has only begun with the creation of the device. I wish I did record for the best things it will see are the reactions of the people it encounters as I wear it but it will never know them.

Here are some pictures.

me wearing my seeing aid

me wearing my seeing aid

the device

Green flashes for distance and blue under 80F and red above

Green flashes for distance and blue under 80F and red above

This entry was posted in Uncategorized and tagged , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s