Monday, August 28, 2017

First shot: City nightlights

First shot of the city at night.  Bloom is too high right now and that will eventually be controlled by the audio but I've got buildings growing and reacting to music.  Video of all this in motion is coming soon.

Saturday, August 5, 2017

New audio analysis engine 3.0 complete

Version 3.0 of my audio analysis system is finally complete!  This is essentially a complete rebuild of my previous visualization system with additional flexibility added.  Now I can control individual frequency ranges across the audio spectrum.

The previous version had all that stuff hard-coded.  This latest version allows someone to set different frequency ranges for each of the different audio channels.  Some pics of the newest system are below:

A new and improved interface with larger text and more accessible sliders. This new system will also make use of Max/MSP's Mira which allows you to control the interface from a website!  Custom interfaces will be faster to implement versus having to use TouchOSC in previous versions.

Interface for controlling different frequency ranges.  Currently this is set to 32 bands. It can be increased if additional frequency resolution is required.

Simplified and condensed audio compression interface. It looks like a lot of knobs in there but ideally you would only need to set these once and then forget about it afterwards.

Example of some of the data being extracted from each channel. This gets sent to Unity3D via OSC.

Now that I've wrapped this part up, it's time to move back into seeing how all this plays out in 3D.  More updates will be coming soon with a city reacting to music.

Thursday, May 18, 2017

A City and Night - visualization progress #1

Development continues.  I realize it's been awhile.  Life just seems to get busy.  So here is a line-up of buildings I've got.  I figured I would go with all the letters of the alphabet so there are 26 different buildings here!  Check them out.

Building Z is obviously the biggest and coolest one while building A is the shortest and smallest.  The idea will be to procedurally increase building type based on amplitude analysis for different frequency ranges.  I'm also exploring the possibility of higher frequencies having more of a spire look to them while lower frequencies will be thicker/heavier shaped buildings.  Height would still be tied to amplitude.

Getting all these buildings built was the easy part.  The next stage will be tracking amplitude as it accumulates across various frequencies.  My previous audio analysis system was somewhat limited. I had 8 bands of frequencies to pull from but I wanted more flexibility. The frequency ranges were limited to 8 distinct bands which were hard-coded.

For example, one band went from 20Hz to 120Hz.  The next band might have been from 150Hz to 350Hz. You can see the original 8 bands in blue here:

The new system I've got will allow almost unlimited flexibility with which frequency bins I collect. There are a great series of tutorials on the web about FFT analysis for Max/MSP.  I highly recommend checking them out below.

Using the help of the work above, I've constructed a new equalizer with extremely fine controls over which frequencies will be analyzed.  Check out the image below:
The light blue and gray curves at the top control which frequency ranges will be analyzed.  You can see the result in the lower spectrograms.  The one on the left is the complete spectrum while the one on the right is a subset based on the control curves.  This will allow finite tuning of each audio channel to better match specific instruments, voices or whatever range is required.

Next steps will be to replicate this across multiple channels and pipe all that revised data over to Unity3D.  I still need to tweak some envelop following and data smoothing functions to best capture the music and solidify a cleaner user interface.  More updates in the future with some city animations too!

Thursday, March 9, 2017

Asah & Raeq performance documentation

It's been awhile since I've posted anything.  I finally have performance documentation page on my site for the Asah and Raeq concert collaboration I had with Noniko.  This was a really awesome video game inspired concert with visualizations that spanned across 7 different pieces of music.

Check out the latest update on my site here:

And you can also watch the complete performance on Noniko's Youtube here:

Sunday, August 21, 2016

A city at night - my next visualization project

It's been awhile since I've had the chance to blog or start on any new visualizations.  Got a new job which has been going well and finally settled in to a new routine.  I have a bunch of new projects I've been looking to start and I'm finally in a position to do so.

Vincet Laforet is an incredible aerial photographer.  I'm sure many of you have seen his works from the AIR project.  If you haven't, I've linked a few below.  Definitely check out his site where you can see many more images and order his book AIR or lithographs. 

The use of tilt shift is really great especially in his New York City shots.  I thought it would be really cool to have a music visualization which procedurally generates a city to music.  Various buildings would rise up with different shapes and architectural designs based on changes in the audio.

I'm also looking to explore compound actions and patterns in my visualizations.  This is something I've briefly scraped the surface of this concept in the past, but I've never fully gotten it to work or look great so hopefully I can get it to shine in this visualization.  The idea is to have different visuals react when certain thresholds are met, or when a more complex series of events and thresholds are met.  Music has so many layers and intricacies which can often be hard clearly visualize.  I'm looking to develop a more complex system of thresholds and events that can spawn additional unique visuals that in turn could spawn their own unique systems.  It's very similar to a fractal or an L-system.  Well, much more to come on all this.  Please check out Vincent's work and stay tuned for the visualization inspired by his work!

Wednesday, June 8, 2016

A new VR experience - Bjork's Vulnicura

This isn't a music visualization, but it is really cool.  Check out Bjork's newest VR experience.  I had the opportunity and privilege to work on this project.  It's great to see it getting released.  Please check it out and I hope you enjoy!  Thanks to River Studios for the opportunity to work on this too.

Thursday, November 12, 2015

Noniko Hsu: Video Game, Movie, and Flute

So it's been a long time since I've posted anything.  Unfortunately I've been swamped with two jobs.  One of them just finished and they have a released a short making-of introduction video.  Check it out on YouTube below.  There are lots more videos coming in the next month about this concert so stay tuned.

First shot: City nightlights

First shot of the city at night.  Bloom is too high right now and that will eventually be controlled by the audio but I've got build...