I’d like to share a nifty little project that we just completed for our first roofies party of the summer here at TBG. We’re calling it InstaScope. InstaScope is an installation that was made to entice partygoers to take photos with instagram so that they can feed the live kaleidoscope that was being projected right there at the party. All they had to do was tag their photo with “#TBGRoofies”.
What it does is simplifies and returns a smoothed out, simplified Path2d object based on a vector of Vec2fs that you pass to it. It uses the same algorithm that paperjs uses, but c++-ified. It’s a slightly modified version of An Algorithm for Automatically Fitting Digitized Curves from “Graphic Gems.”
There are 2 samples in there.
PathSimplification: Drag to draw a line on the screen. Once the line is finished, it analyzes and smooths out the line, letting you know how many lines you saved. It’s based on this paperjs example.
ContinuousPath: Analyzes and smooths a path as you draw it. Could be good for making a drawing application.
My project for January 2012 is complete for now. The result is below:
The result is virtual pin art which takes the depth values from a kinect camera and translates them into a depth which is projected for each pin. This was put together using cinder and the kinect cinder block which is the freenect kinect library configured to work with cinder.
There were a few things that I learned, some of them super obvious, but will nonetheless help me make better decisions next time around:
– The kinect has a lot of inconsistencies, especially when it comes to depth data. There are some ways to make things smoother, but you’ll notice that there aren’t too many examples out there that rely on the precise kinect depth values.
– Processing is great for prototyping. Cinder is great for the real thing. Processing helped me figure out what was possible and helped get me there relatively quick, but once I had a lot of particles on screen, things quickly began to slow down and make my processor chug. Once I moved into c++ and cinder, those same processes began to run much smoother.
– Having side projects is hard to keep up with when you have a 1 year old at home and projects to do at work. The is obvious, but not so much when you’re thinking about all the things you want to learn and explore. That being said, I’m still committed to learning and experimenting as much as I can once my first two priorities are taken care of. Who needs to watch crappy reality TV anyway?
I don’t consider this a final piece, but more of a really good proof of concept. Since this was my first kinect, cinder, and C++ project, a lot of the time was learning the capabilities and workflow. I’m obviously not a C++ expert and I know I have a lot to learn, but this was a great way to learn it. For the second phase of this project, I’d like to bring in some 3d textures and shading. Really give it a metal pin art look. I’d also like to smooth out some of the depth map noise. I know I won’t be able to nail that down perfectly, but I’ve read about some methods to smooth that out a bit better within the limitations of the current kinect’s resolution and camera positioning issues. The hope for this is to throw it up somewhere that it can be part of an installation that people can walk up to and interact with and cycle through to see the imprint of previous visitors as well.
To see some of the work in progress, check out my tumblr. Once I clean up the code, I’ll post that somewhere too.
So this week, I actually made some good progress and am close to reaching my goal for this month. After jumping from using processing, to using cinder, I learned a ton. The biggest challenge was the jump into cinder and C++ in general. What took me a few hours to come up with in processing, took 3 or 4 days in cinder since I don’t really know C++, haven’t touched cinder in over a year, upgraded xcode and had to learn my way around it and have come across a lot of other weird errors and stupid mistakes that I’ve been making.
Luckily, I have the author of cinder, Andrew Bell, now working with me at The Barbarian Group again and has been a good sport with answering my silly questions and helping me though some of the learning process. If he wasn’t around to help me out, I’d probably be another week behind where I want to be.
So as far as progress goes, here’s a sample of some screenshots from the processing and cinder experiments.
I’ve been able to come up with a good proof of concept that what I had in mind will work and I just need to tidy it up and make it look good so that I can video capture it and write up my learnings. I’ve realized that with this project, I still have more to do, but there’s enough that it can be another month’s worth of side project, so that’s what I plan on doing. But first, I need to read up more on c++ and get more familiar with cinder. So then next blog post about this project will be the last for the month and will summarize what I’ve learned and showcase the output for this stage of the overall project.
So far, this month’s project is not moving along as quickly as I had hoped it would, but it’s ok. With the time I have had to work on stuff, I’ve already learned a lot. And because of what I learned, I’ve concluded that I have to modified my goal a bit.
To get going with this project, I’ve read a good chunk of Making Things See, which is an amazing starting point for anyone new to the Kinect, especially because it uses processing as the teaching environment. Through some of the examples in the book, I was able to piece together a quick little program that enables me to only grab image data that is within x distance and disregard the rest. It wasn’t a very complex program to put together, but once you have it working, the kinect’s limitations are obvious. At this point, it’s not perfect. I had in mind that you’d be able to easily mask out anything in the foreground, but there’s so much noise that it’s impossible to get any sort of clear defining outline. I feel like somehow averaging a person’s outline along with doing some image color comparison processing, you could get to a good point. The other problem is that the kinect’s rgb camera is also not very high-res, so even if you get close, the image isn’t going to look great.
My hope in the future is that there will be a kinect update with higher res and more depth data. In fact, I heard that an updated kinect is coming out very soon, though we’ll have to see what kind of improvements it really has. In addition to an updated kinect, I’d like to figure out if there’s a way to capture a photo with a decent DSLR and match up the data from the kinect with that to do be able to get some decent photos.
So after coming to these conclusions, I’ve decided that there were two directions I could move in. I could either continue exploring this project knowing that I have some pretty high technical fences to jump or I can take what I’ve learned so far and come up with a new direction to move in that’s more feasible. I’ve decided to pursue the second route, which entails using the kinect depth data in a way that could be visually cool and fun to play with, while also having the potential to be a nice little installation. This route also would include the use of a little openGL, which I’ve wanted to learn more about too.