Live Music Visuals: Galaxy v2.0 - Burning Bridges

Still working through Unity2019 and all the preview packages.  Trying to get some cool space stuff that reacts to music.  Second attempt here with the VFX particle system in Unity.  It's coming along nicely.  20K+ particles at any one time.  Definitely pushing some limits on my computer right now.  Much more to do.

Latest demo reel and a new logo update

So I've got a new demo reel to reflect work I've done for the past 4 years.  In addition, it was time for a logo update as well.  Enjoy the new look and check out examples of my recent work.  Some new videos have been posted as well in support of the latest demo reel.

Worldspace particle systems with VFX graph Unity 2018.3

Playing around with Unity 2018.3 visual effects shader graph.  Figured out how to do worldspace particle systems.

Live Music Visuals: Raindrops 2.0

Final version of my Raindrops 2.0  The Song is complete and the visualization is complete.  Leverages alembic fluid caches from Maya Bifrost.  OpenVDB for mesh simplification, and blend shapes for some of the raindrop effects.  Throw in a few particles, post processing, connect everything to 8 channel spectral audio analysis and Voila!  Enjoy.  Next up will be galaxies and side-scrolling graphic designs.

Raindrops visualization: More Alembic fluid cache animations!

Raindrops visualization - Adventures by A Himitsu

Been working with Alembic caches for a little bit now.  Lots of awesome potential here.  Of course I had to instance this effect and see what is possible.  Also utilizing openVDB to simplify the fluid mesh so everything runs better in real-time.  Still a work in progress but I'm expecting to wrap this up soon.



Still need to incorporate a few more different splash sims, tweaks to lighting, fog, and set this up to my Raindrops music.

Alembic fluid caches in Unity

Looking into using Alembic fluid caches in Unity.  Not sure if this is fully ready for prime time but exploring it.  Lots of great functionality here!!

A couple good resources:
https://github.com/unity3d-jp/AlembicForUnity
https://forum.unity.com/threads/alembic-for-unity.521649/page-2

Water splash simulations for music visuals

Trying to find a way to capture a water simulation and reuse it for visuals in Unity.  Not quite sure how I'll finish this approach.  Need to find a way to maintain a uniform mesh across the simulation.  Found a great tutorial on using MASH to distribute across a fluid mesh.  That might be one approach.

https://lesterbanks.com/2018/02/mixing-mash-bifrost-cool-effects/

Converting the fluid sim to a finite number of particles could be another approach.  Anyways, enjoy the work in progress so far. Mesh captures of the sim at different frames.

NYC nightlight visuals: Au5 - Closer

Revisiting a visualization I worked on last year.  Finding great music to accompany the visuals.  I still have some minor tweaks to do for this system.  Looking to get more visual bang for the buck during the louder hits in the audio.  Also need to set the development of these buildings in motion which should help accentuate the audio.

Raindrops

My latest single, Raindrops, has been released.  This piece starts with a gentle series of raindrops represented by marimbas, xylophones, and piano. The piece slowly builds into a complete orchestral thunderstorm.  The music is also available for purchase on iTunes, Amazon, CDBaby, and available to stream on numerous platforms. Enjoy!

Incredible performance and many thanks!!!!

I wanted to express my deepest gratitude to the The Maryland Classic Youth Orchestras for the incredible opportunity to collaborate on the Krzesany piece.  The performance was a huge success!  The musician's performance was superb and Kristopher's conducting and musical direction was exceptional.

Also, thanks to the staff and crew at Strathmore Hall.  Everyone was a pleasure to work with.

Wojciech Kilar's inspiration came to life as majestic mountains illuminated to the music and Polish paper art danced to the intricacies of the folk themes woven throughout the piece.  I have a few images and videos below.  Stay tuned for a segment on the making of the Wycinanki paper arts portions of the visualization.  I'll break down the inspiration, construction, and final composition used in the music visualization.  For now, enjoy some of these images/videos.  I'll have another compilation of the performance soon.

Wycinanki paper demo

MCYO performance December 2nd!

So I have the privilege of working with Maryland Classic Youth Orchestras (MCYO) and I will be presenting a live visualization for their performance of Wojciech Kilar's "Krzesany." In researching the origins and inspiration for this piece, I discovered that Kilar was inspired by the Tatras mountains of Poland. These mountains are gorgeous and very majestic.  They remind me of the Grand Tetons here in the United States.

I also began exploring Polish folk art and stumbled upon "Wycinanki."  This is the Polish word for 'paper-cut design.' You can learn more about this art form here at the Polish Art Center.  The multi-layered, brightly colored paper was very inspirational and I decided to leverage this art style along with the majestic Tatras mountains in my music visualization. Check out some of the works in progress below.

Finally, if you're in the Washington DC area on December 2nd, please come check out the live performance at 4:30PM  The MCYO will be performing the piece and my visualization will be reacting to their performance live and in real-time!!  The performance will be held in the beautiful Strathmore Hall.  I highly encourage everyone to come check it out.  Tickets can be purchased here:

https://www.strathmore.org/events-and-tickets/mcyo-symphonic-devotions

First shot: City nightlights

First shot of the city at night.  Bloom is too high right now and that will eventually be controlled by the audio but I've got buildings growing and reacting to music.  Video of all this in motion is coming soon.

New audio analysis engine 3.0 complete

Version 3.0 of my audio analysis system is finally complete!  This is essentially a complete rebuild of my previous visualization system with additional flexibility added.  Now I can control individual frequency ranges across the audio spectrum.

The previous version had all that stuff hard-coded.  This latest version allows someone to set different frequency ranges for each of the different audio channels.  Some pics of the newest system are below:

A new and improved interface with larger text and more accessible sliders. This new system will also make use of Max/MSP's Mira which allows you to control the interface from a website!  Custom interfaces will be faster to implement versus having to use TouchOSC in previous versions.

 

Interface for controlling different frequency ranges.  Currently this is set to 32 bands. It can be increased if additional frequency resolution is required.

 

Simplified and condensed audio compression interface. It looks like a lot of knobs in there but ideally you would only need to set these once and then forget about it afterwards.

 

Example of some of the data being extracted from each channel. This gets sent to Unity3D via OSC.

Now that I've wrapped this part up, it's time to move back into seeing how all this plays out in 3D.  More updates will be coming soon with a city reacting to music.

A City and Night - visualization progress #1

Development continues.  I realize it's been awhile.  Life just seems to get busy.  So here is a line-up of buildings I've got.  I figured I would go with all the letters of the alphabet so there are 26 different buildings here!  Check them out.

Building Z is obviously the biggest and coolest one while building A is the shortest and smallest.  The idea will be to procedurally increase building type based on amplitude analysis for different frequency ranges.  I'm also exploring the possibility of higher frequencies having more of a spire look to them while lower frequencies will be thicker/heavier shaped buildings.  Height would still be tied to amplitude.

Getting all these buildings built was the easy part.  The next stage will be tracking amplitude as it accumulates across various frequencies.  My previous audio analysis system was somewhat limited. I had 8 bands of frequencies to pull from but I wanted more flexibility. The frequency ranges were limited to 8 distinct bands which were hard-coded.

For example, one band went from 20Hz to 120Hz.  The next band might have been from 150Hz to 350Hz. You can see the original 8 bands in blue here:

The new system I've got will allow almost unlimited flexibility with which frequency bins I collect. There are a great series of tutorials on the web about FFT analysis for Max/MSP.  I highly recommend checking them out below.

https://cycling74.com/tutorials/advanced-max-ffts-part-3

Using the help of the work above, I've constructed a new equalizer with extremely fine controls over which frequencies will be analyzed.  Check out the image below:

The light blue and gray curves at the top control which frequency ranges will be analyzed.  You can see the result in the lower spectrograms.  The one on the left is the complete spectrum while the one on the right is a subset based on the control curves.  This will allow finite tuning of each audio channel to better match specific instruments, voices or whatever range is required.

Next steps will be to replicate this across multiple channels and pipe all that revised data over to Unity3D.  I still need to tweak some envelop following and data smoothing functions to best capture the music and solidify a cleaner user interface.  More updates in the future with some city animations too!

Asah & Raeq performance documentation

It's been awhile since I've posted anything.  I finally have performance documentation page on my site for the Asah and Raeq concert collaboration I had with Noniko.  This was a really awesome video game inspired concert with visualizations that spanned across 7 different pieces of music.

Check out the latest update on my site here:

http://livemusicvisuals.com/asah-raeq.html

And you can also watch the complete performance on Noniko's Youtube here:

https://www.youtube.com/playlist?list=PLyfdvtWMn6HyoaRexBQyFQ8ay1PQSi4PR

A city at night - my next visualization project

It's been awhile since I've had the chance to blog or start on any new visualizations.  Got a new job which has been going well and finally settled in to a new routine.  I have a bunch of new projects I've been looking to start and I'm finally in a position to do so.

Vincet Laforet is an incredible aerial photographer.  I'm sure many of you have seen his works from the AIR project.  If you haven't, I've linked a few below.  Definitely check out his site where you can see many more images and order his book AIR or lithographs.  http://www.laforetvisuals.com/ 

The use of tilt shift is really great especially in his New York City shots.  I thought it would be really cool to have a music visualization which procedurally generates a city to music.  Various buildings would rise up with different shapes and architectural designs based on changes in the audio.

I'm also looking to explore compound actions and patterns in my visualizations.  This is something I've briefly scraped the surface of this concept in the past, but I've never fully gotten it to work or look great so hopefully I can get it to shine in this visualization.  The idea is to have different visuals react when certain thresholds are met, or when a more complex series of events and thresholds are met.  Music has so many layers and intricacies which can often be hard clearly visualize.  I'm looking to develop a more complex system of thresholds and events that can spawn additional unique visuals that in turn could spawn their own unique systems.  It's very similar to a fractal or an L-system.  Well, much more to come on all this.  Please check out Vincent's work and stay tuned for the visualization inspired by his work!

A new VR experience - Bjork's Vulnicura

This isn't a music visualization, but it is really cool.  Check out Bjork's newest VR experience.  I had the opportunity and privilege to work on this project.  It's great to see it getting released.  Please check it out and I hope you enjoy!  Thanks to River Studios for the opportunity to work on this too.

https://www.facebook.com/bjork/videos/vb.6747251459/10154195901066460/?type=2&theater

http://variety.com/2016/digital/news/bjork-digital-virtual-reality-experience-1201787563/

Noniko Hsu: Video Game, Movie, and Flute

So it's been a long time since I've posted anything.  Unfortunately I've been swamped with two jobs.  One of them just finished and they have a released a short making-of introduction video.  Check it out on YouTube below.  There are lots more videos coming in the next month about this concert so stay tuned.