For my February project, I’ve decided to continue with a project that I’ve already started, but neglected over the last month or two. I don’t want to give everything away, but it’s about taking values from nature using an accelerometer hooked up to an arduino and using the data to create something visually interesting using processing. This is actually a collaboration with an artist that’s also super psyched about this project. He came to me with this idea and I’m here to help him pull it off.

Currently, this project exists as a sketch on an arduino board and 3 different processing sketches that each serve a very specific purpose. One for recording data, one for calibrating the accelerometer, and one for reading and displaying the data. This of course is not an ideal set up, but the processing environment has been great for splitting up the different functions and figuring out each piece separately.

At the end of the month, I hope to have some cool sketches and hopefully something worthy of creating a nice print of. Since this is a collaboration, it might take a few months to get to that point, but this should at least be a huge step towards being able to create some beautiful stuff with this data.

To get there, this is what I need to do:

• Combine the previous processing sketches into one project/sketch.
• Create an accurate representation of the incoming data from the arduino so that we better understand the data coming in as well as how it relates to the actual accelerometer movements.
• Explore ways to showcase the data in a visually interesting way. I hope this will be a series of experiments. I’ll try to post these on my tumblr blog.

My project for January 2012 is complete for now. The result is below:

The result is virtual pin art which takes the depth values from a kinect camera and translates them into a depth which is projected for each pin. This was put together using cinder and the kinect cinder block which is the freenect kinect library configured to work with cinder.

There were a few things that I learned, some of them super obvious, but will nonetheless help me make better decisions next time around:

- The kinect has a lot of inconsistencies, especially when it comes to depth data. There are some ways to make things smoother, but you’ll notice that there aren’t too many examples out there that rely on the precise kinect depth values.

- Processing is great for prototyping. Cinder is great for the real thing. Processing helped me figure out what was possible and helped get me there relatively quick, but once I had a lot of particles on screen, things quickly began to slow down and make my processor chug. Once I moved into c++ and cinder, those same processes began to run much smoother.

- Having side projects is hard to keep up with when you have a 1 year old at home and projects to do at work. The is obvious, but not so much when you’re thinking about all the things you want to learn and explore. That being said, I’m still committed to learning and experimenting as much as I can once my first two priorities are taken care of. Who needs to watch crappy reality TV anyway?

I don’t consider this a final piece, but more of a really good proof of concept. Since this was my first kinect, cinder, and C++ project, a lot of the time was learning the capabilities and workflow. I’m obviously not a C++ expert and I know I have a lot to learn, but this was a great way to learn it. For the second phase of this project, I’d like to bring in some 3d textures and shading. Really give it a metal pin art look. I’d also like to smooth out some of the depth map noise. I know I won’t be able to nail that down perfectly, but I’ve read about some methods to smooth that out a bit better within the limitations of the current kinect’s resolution and camera positioning issues. The hope for this is to throw it up somewhere that it can be part of an installation that people can walk up to and interact with and cycle through to see the imprint of previous visitors as well.

To see some of the work in progress, check out my tumblr. Once I clean up the code, I’ll post that somewhere too.

Related Posts:
Project 1 – January 2012
January Kinect Project – Update 1
January Kinect Project – Update 2

So far, this month’s project is not moving along as quickly as I had hoped it would, but it’s ok. With the time I have had to work on stuff, I’ve already learned a lot. And because of what I learned, I’ve concluded that I have to modified my goal a bit.

To get going with this project, I’ve read a good chunk of Making Things See, which is an amazing starting point for anyone new to the Kinect, especially because it uses processing as the teaching environment. Through some of the examples in the book, I was able to piece together a quick little program that enables me to only grab image data that is within x distance and disregard the rest. It wasn’t a very complex program to put together, but once you have it working, the kinect’s limitations are obvious. At this point, it’s not perfect. I had in mind that you’d be able to easily mask out anything in the foreground, but there’s so much noise that it’s impossible to get any sort of clear defining outline. I feel like somehow averaging a person’s outline along with doing some image color comparison processing, you could get to a good point. The other problem is that the kinect’s rgb camera is also not very high-res, so even if you get close, the image isn’t going to look great.

My hope in the future is that there will be a kinect update with higher res and more depth data. In fact, I heard that an updated kinect is coming out very soon, though we’ll have to see what kind of improvements it really has. In addition to an updated kinect, I’d like to figure out if there’s a way to capture a photo with a decent DSLR and match up the data from the kinect with that to do be able to get some decent photos.

So after coming to these conclusions, I’ve decided that there were two directions I could move in. I could either continue exploring this project knowing that I have some pretty high technical fences to jump or I can take what I’ve learned so far and come up with a new direction to move in that’s more feasible. I’ve decided to pursue the second route, which entails using the kinect depth data in a way that could be visually cool and fun to play with, while also having the potential to be a nice little installation. This route also would include the use of a little openGL, which I’ve wanted to learn more about too.

More to come soon.

See Project 1 – January 2012

As mentioned in my previous post, my plan for 2012 is to produce a side project every month. For January, I’m deciding to create something with the XBox Kinect. It’s something that I’ve wanted to do since I saw some of the amazing stuff that people have been doing all over the internet over the past year or so since the Kinect was released. I started reading the book Making Things See last month and have done a few tutorials and will use it as a starting point for this little project.

What I plan to do is use processing to create a program that takes a photo, removes the background and replaces it with another background. Pretty much what a green screen does, without the green screen. What is replaced and what happens with that photo is gonna be a surprise. If it works out the way I want it to, I’d like to develop it more and make it into some sort of installation that can be used at work or where ever anyone wants to set it up.

Here’s my breakdown of weekly goals:

Week 1: Research and feasability scoping

Week 2: Image capture and testing

Week 3: Image saving and uploading

Week 4: Clean up and blog post

So here we go…

A few months ago, I started having this little fascination with spider webs. I was just wondering how they work and how they were built. At the same time, I befriended a little spider that made its home on the driver’s side mirror on my car, called Charlie. Charlie was great and he hung around for a few weeks and taught me a good bit about spider webs. For one thing, they’re very strong. Each strand that’s created is reinforced by the next strand. Sometimes strands double up. Sometimes he even misses a connection. Sometimes he creates this y-shaped connection. Unfortunately Charlie is dead. Or he ran away.

To show what I learned, I started this little processing sketch to just see if I could make a simple spider web. The math for making some sort of simple perfect spider web isn’t hard, but it’s also not that interesting to look at either. As perfect as some spiderwebs look, there is a actually a lot of variation to them. They are an example of what a lot of generative artists always look to replicate from nature, the balance between the predictable and the random.

Once I got the web working in processing, I had an idea to use my webs in the context of an html site, which meant I would have to move it to Javascript. I’ve kind of avoided javascript for a while, but more recently wanted to explore it a bit, so this served as the perfect first javascript project. I originally started by just exporting the processing sketch as a processing js sketch. It didn’t wuite work since I was using the toxiclibs library for the physics environment. Even if it did work as I thought it should, I feel like it would have been cheating. So instead, I went through some javascript canvas tutorials by Keith Peters and restructured the project as pure javascript. I still ended up using processing js for the canvas drawing and I also ended up finding a version toxiclibs that was ported to javascript. The major difference with that was that I had to target the specific toxi namepsace when using any part of the toxiclibs library. No biggie.

Go here and play with it a bit. It’s not perfect, but I’ve got some other things I want to explore and need to move on. I don’t plan on abandoning it altogether though. I’d like to go in and add some dimension to it and bring it into the more efficient 3d environment of webgl. Also, I’d like to explore and push the webs farther in a more interesting direction visually. It’s also begging to have some more interactivity brought to it. Hopefully this will end up making it into some other project in the future.

As a disclaimer, I’m not much of a web front end kind of guy. I’ve from the world of flash and just like to make pretty looking stuff. From what I’ve found, this is slow as hell in firefox, but plays reasonably well in chrome. Let me know if you have any comments on how to improve this. All comments are welcome. I think.

So I figured out what I was going for…








Now that I got what I was thinking out of my head. Now it’s time to evolve it into something new. With where I hope this will go, there should be a video up for the next post.

I’ve been trying to re-create a style of doodling that I used to do in highschool (which I admit I got from one of my actually artistic friends). Since I kind of feel bad about stealing his style, I decided I would take it as a base and build on it and make my own. Besides, everything’s a remix anyway.

attempt #1

attempt #2

It’s not there yet, but I’m happy that it’s turning into something that I had in my head.