New App! 64 (Video) Fingers.

applications

- CommentAuthoregon77
- CommentTimeAug 24th 2009 edited
(LATEST VERS Attached to the bottom of this post)

Hey Everyone, So it is finally time for me to add an app to the community. It has been an extremely good learning experience for Max/MSP. To begin with I want to thank Dovemouse for his help and also creating the perfect app for me to dive into. I also want to thank Vade on the Cycling74 forums for his work on video optimisation using jitter.

So here is the app.

Inspired by this guys youtube mashups:

http://thru-you.com/

Its called 64 (Video) Fingers. It works very similarly to Dovemouses 64 Fingers with some slight differences.

Here is a demo video I recorded. Please check it out.

http://www.vimeo.com/6255134

A screen capture type tutorial is in the works to show how to setup as well as other ways of using it.

Basically, now each slot has two places to dropfile labeled with a "V" and an "A". "V" is where the video clip goes and "A" is for audio.

There are two modes for each slot labeled with a "1" and "2". "1" is the equivalent of the one shot trigger. Tap the button and the video plays until its end. "2" is similar to what was "3" on the original version which is a sort MPC-style beat repeat with selectable rhythm divisions via a dropdown menu on the slot. This version is also momentary rather than toggle. Because of the video hierarchy that I setup toggles are not complimentary to it.

Each slot also has a dropdown for group select. If more than one slot is part of the same group then they will cut each other off rather than being polyphonic. What the group select also enables you to do is send that group as a separate track back into Ableton via rewire for separate effects treatment.

You will notice some groups are labeled "None1", "None2", "None3". If more than one sample is a part of one of these groups then they will remain polyphonic but be routable to three separate tracks in your DAW. A good selection for percussive sounds where you want the ringing and decay of one to mix with another of the same group.

Other controls for each slot are volume slider as well as an in/out slider. You can trim your clip to your liking by dragging across the in/out slider.

First six buttons on the top row are the pattern recorders. They also double as a layering system for each video feed. Press the button once to activate record. (IMPORTANT NOTE: When you enable the recorder, you will have to record something. You cannot press the button again to deactivate it) It won't start recording until you play something on the buttons. You will have to play through until you see the button begin to flash. Voila! Your pattern is playing.

Press the same pattern recorder button once again and you will disable viewing of that video feed revealing the pattern below. The pattern recorder 1 is at the bottom level of viewing. Recorders 2 is above 1, 3 above 2, 4 above 3 and so on. When many layers are playing at once you can attempt to control which layer the audience sees when.

All soloing done on the buttons will be seen above all pattern recorders.

To clear a pattern first you must disable viewing by pressing it once. Then double tap the button quickly and your pattern is clear.

Midi input is also possible, so you can trigger the video clips from a midi keyboard or your DAW sequencer. When using it with a sequencer it is important to note that while the audio clips can be polyphonic there can only be one video clip up at a time.

(my post is too long so please look at the next post for a continuation)
- 128VideoFingers_1.05.zip
- 64VideoFingers_1.05.zip
- CommentAuthoregon77
- CommentTimeAug 24th 2009 edited
For the best performance encode your videos with Motion-jpeg or photo-jpeg codec. Video and audio should be exported separately. Make sure your video clips have no audio.

For my demo video I went with 720X480, 15 fps, deinterlaced. It can be a bit processor heavy and a lot may come down to your video card because it uses opengl to draw the video to the window.

Whew! Anyways I hope you all enjoy. I am pretty happy with its playability and am continuing to develop it. Some things to look forward to in the next few versions will be: Video Effects, Pattern recorder overdub, mute buttons, midi channel select corresponding to different video layers and eventually multi-screen. As well as plain old debugging.

Feature requests are welcome but my timely response may not be so timely as I have a wife, baby and backyard that have been calling my name for a few months now so your understanding is appreciated.

Lastly, I hope that this app inspires you to use it differently than its intended purpose. I am in the preproduction phase creating a film that will eventually be performed live on the monome. Like the organist of the old days would perform the music live to the film now the organist will be performing the film as well. With each scene being a preset bank, I hope to make the film somewhat modular so that you play it once and its from this characters perspective and you play it a second time from the other persons perspective.

Anyways Happy Monom-ing!!!
- CommentAuthormaersk (dovemouse)
- CommentTimeAug 24th 2009
amazing demo video. can't wait to try this out.

just a thought for optimisation: i noticed that the leds remain on after the vide has switched to another clip in that group. does this mean the video is still playing? might be best to detect another clip in that group is triggered and stop the first one?

haven't looked at the innards yet but how about a video mixer between the groups?
- CommentAuthoregon77
- CommentTimeAug 25th 2009
Wow, dovemouse! Really good observation. Yes the LEDs also represents a metro running somewhere. I tried using the group number with the slot arguments in an IF statement to send a 0 to all other metros in a group when a button is pressed. Although, I had already gotten the app to be playable and then jumped onto gathering material for it. I didn't put enough effort into troubleshooting that IF statement.

Btw, thanks for creating such a well laid out patch. You can look into my patch and see how I went in and messed it all up.

I would really appreciate your advice on how to clean it up in there.
- CommentAuthormaersk (dovemouse)
- CommentTimeAug 25th 2009
haha i opened up the patch and then had to have a lie down.
testing will commence :D
- CommentAuthoregon77
- CommentTimeAug 25th 2009
You Rock, Dovemouse! thanks again for all your help. I could list some bugs that I already know of but I think I will wait for any questions because there is a lot too take in.
- CommentAuthoregon77
- CommentTimeAug 25th 2009 edited
Whoa!!!!!!!!!! Damn!!!!!!. I will have a look now!

Edit: Sorry, in response to whisper from Dovemouse. Still have trouble recognizing whispers.
- CommentAuthorrent
- CommentTimeAug 25th 2009
so awesome. can't wait to try it out. great work and thanks for sharing!
- CommentAuthorro
- CommentTimeAug 25th 2009
wow! great work! >.>!>>>!

chopping up a monome with another monome is pretty epic.
- CommentAuthorSHvanBommel
- CommentTimeAug 25th 2009
this is beautiful.
- CommentAuthorbrothernigel
- CommentTimeAug 25th 2009
whoa. sick patch and great video!
- CommentAuthorstevieraysean
- CommentTimeAug 25th 2009
remixing a monome with a monome? thats so post-monome
- CommentAuthorjulianchaidez
- CommentTimeAug 25th 2009
excellent job
- CommentAuthorsimioliolio
- CommentTimeAug 25th 2009
i've been wanting to make something like this for ages! nice work, great vid, hope to see this continue developing!
- CommentAuthorMatt KU
- CommentTimeAug 25th 2009
kutiman in a box - nice :)
- CommentAuthorscrubberfox
- CommentTimeAug 25th 2009
SIIIICCCCCKKKKKKK!!!!!!
- CommentAuthorjimdrake
- CommentTimeAug 25th 2009
like the idea of one of the instruments being a monome. monome playing a monome!

is it me or does the timing sound a bit off? something's flamming with the snare but by a different amount each time. or is that part of something you're doing?

would be interested in a link to discussions about video at cycling74.

next step, live video record and mashup!
- CommentAuthorswitches&buttons
- CommentTimeAug 25th 2009
HOORAY FOR VIDEO APPLICATIONS.

thank you so much.
- CommentAuthorhypno|sapien
- CommentTimeAug 25th 2009
Amazing!
- CommentAuthoregon77
- CommentTimeAug 25th 2009 edited
I want to thank everyone for all the wonderful comments. It has been exciting to watch the number of plays on the video rack up.

I now want to disclose any caveats I can think of to help you guys get started with the software. It's my seven reasons he released the software to early list.

1. The help button for starters will be only so helpful because it is the same help file from the original 64 fingers. Have to fix.

2. In the settings button for clock input. There is the option of choosing Rewire for your clock. Don't use this. It doesn't work. If your syncing to another application or using Ad_rewire as your DSP option then use "External" as your setting and select the midi port you'll receive the clock signal from.

3. Mode 2 for each of the slots doesn't really work so well with the pattern recorders. The way its built, the off signal comes from a loopnotify message from the end of the quicktime movie. So pattern recorders only record "On" signals. So if you record a slot in mode 2 it will stay on until you clear the pattern and press the button again. (fixed in 1.02)

4. FYI, For those wanting to send your separate tracks back to your DAW for effects processing the Hierarchy is this: Group1=Input 3/4; Group2=Input 5/6; Group3=Input 7/8; Group4=Input 9/10; Group5=Input 11/12; Group6=Input 13/14; Group7=Input 15/16; Group8=Input17/18; GroupNone1=Input 19/20; GroupNone2=Input 21/22; GroupNone3=Input 23/24; Now, here's the catch: In Ableton you can only receive separate tracks for 1-16. So the None1-3 Groups need to share some inputs. They are set through the i/o mappings in the DSP status in Max. Through that you can decide whatever inputs you would like to send these groups.

5. You will notice a button called "Reload bangs" This doesn't really work as advertised unless you press it twice. If you have clips in your slots make sure you store and save your preset file because if you reload bangs then you will no longer have those clips in your slots. I will be fixing this reload bang button soon because it is an easy fix. The only reason I added it was to sort of half-ass address a bug in the pattern recorders. (No longer exists in 1.02)

6. You'll notice in the video around 4minutes in I begin playing the upright bass again and for some of those buttons you may hear the audio but do not see the video. Thats because when a clip becomes a part of a pattern a gate within that slot closes. The gate is supposed to reopen on clear. It does not always work for every clip. This bug is my current main priority. So the reload bang button might become the reopen gate button, if you know what I mean. (fixed in 1.02)

7. Sometimes when you clear a pattern recorder the LED will stay lit. This is a stupid bug but still exists. It doesn't mean anything. The pattern recorder will still work just the same and in fact the second time around the LED will probably go back to normal. (fixed in 1.02)

@ Jim Drake: Regarding the flamming you are hearing I am not 100 percent sure. There is sometimes a lag on the video as you can see by the glockenspiel in the demo. If a button has not been pressed for some time it lags on the first few button presses. It is possible if your looking at my fingers in comparison to the clips it can be deceiving because my screen capture software was running at 30fps while my video camera on my fingers was at 23.98 pulldown to 29.97. Plus I was eyeballing the sync so it could be a frame off.

Also, if your hearing a flam in the audio it is probably due to my lack rhythm rather than the software. That is what most excites me about releasing this software is seeing what others with much more talent at button mashing can do with it. I'm still learning.

Anyways I hope this helps you all get started with it. All the best

Charlie
- CommentAuthorjimdrake
- CommentTimeAug 25th 2009
flamming observation is audio only, not video.

it seems that drums are looping but each time the loop does not sound identical, so that's nothing to do with your playing
- CommentAuthorchalkline
- CommentTimeAug 25th 2009
Great demo video, can't wait to start playing!
- CommentAuthoralexp
- CommentTimeAug 25th 2009
This app looks awesome. Great demo. Great job.
- CommentAuthoregon77
- CommentTimeAug 25th 2009
@jimdrake, hmm, not sure then. I'm having trouble hearing it.

Regarding a video discussion. I could not have gotten this program to run smoothly without finding these patches made by a fellow on the cycling74 forum called vade:

http://abstrakt.vade.info/?p=147
- CommentAuthoragargara
- CommentTimeAug 30th 2009
Although it seems to be extremely buggy still it's still loads of fun. Do you know what font dovemouse used for the interface? Is it Futura Condensed like mlr5? Because I think whatever font it is is a proprietary font I don't have, since the interface looks kinda screwy to me. I won't go into the other bugs for now since they're sort of hard to describe and you probably already know about them. If I test the program out more and figure out exactly what's happening with some of the bugs, I'll let you know.

Great first app/mod, and I look forward to further developments. The monome community could really use more video apps.

Here's a tip for us non-macintosh junkies: Max only likes to play Quicktime .mov files. The easiest way I found to do this was to just export clips from Ableton Live. This is handy anyway since you can cut up the clips mlr style.
- CommentAuthormaersk (dovemouse)
- CommentTimeAug 30th 2009
gill sans if i remember correctly.
you can edit .mov in ableton?!
- CommentAuthoregon77
- CommentTimeAug 31st 2009
Oh, I would love to hear about what you are experiencing bugs-wise on a PC. It is true that .mov files are the best to use with the codec explained above. But, is it generally working for you?
- CommentAuthoregon77
- CommentTimeAug 31st 2009
I also recommend for preparing your clips using Quicktime Pro. I beleive you can get this for PC as well. It $29 but a nice cheap alternative to most expensive editing programs. Some handy keyboard commands listed here:
L=play; K=stop; J=Reverse play; I=Mark in; O=Mark Out; CTRL-L=Loop Play
- CommentAuthoragargara
- CommentTimeAug 31st 2009
dovemouse: Aha, that's why. I think Gill Sans doesn't come with Windoze. :( Oh well, not your fault. It's not a big issue, although it might be worse for someone that doesn't have Max/MSP and can't edit the fonts themselves. Re: Ableton: I was surprised at its capabilities too, considering that it's not really a video editor. I usually use other programs like VirtualDub for video, but Ableton works fine for cutting up videos on the beat and exporting Quicktime files...

egon77: Yes, generally speaking, everything works. The clips, even if they're really short, often don't load fast enough to play perfectly with the sound. But that's not really a bug, probably just a failure of my video card to draw fast enough. I'll probably experiment with different settings during video export.

I'll try to describe some of the real bugs I have for you: Perhaps the biggest one is that the pattern recorder does not always work properly. For example, it sometimes will "miss" button presses - on playback it will only play back some of what I had recorded. For example if I press three buttons, it might play back only the first and third ones. Problems with the pattern recorder seem to increase if I try to midi sync with Ableton using Midi Yoke.

Sometimes the video for some particular slots will stop working for some reason altogether. If I press that button, the audio plays but the video doesn't, even if the video's still working for other buttons in that group. Restarting the app makes it start working again. Sometimes soloing doesn't always show up over the pattern recorder videos, though this might just be because my old video card can't draw everything fast enough.

Another small problem is that lights will often stay on even after the video/audio is done playing, or after you cut off one clip with another clip from the same group. Also, it seems that if you change to Mode 2 and then back to Mode 1, it Mode 1 will go into "repeat mode", repeating the clip forever. The only way to stop it is to turn the DAC off and on again.

Sorry I can't go into more detail, I haven't had a chance to use it lately so not all the problems are fresh in my mind. Hope this helps, and keep up the good work, I love this program already!
- CommentAuthorMatt KU
- CommentTimeAug 31st 2009
ableton is really handy for quick video edits - particularly as it can time stretch the video along with the audio. you can see it in this little promo clip i made at the start of the year

http://www.youtube.com/watch?v=UXZ9TFFcq-8

another use for a monome ;)
- CommentAuthoregon77
- CommentTimeAug 31st 2009 edited
@agargara, Thanks so much for replying! I am currently visiting family in Wisconson so I have to find the internet here and there but I am so excited to hear others experience using the software. I feel good because most of your comments above seem somewhat familiar. The experience your having when the video clips are not showing over the pattern recorders is my least favorite bug and I am focusing on that whenever I have free time, hopefully tonight. It only happens to a clip after it has become a part of a pattern recorder.

My guess on why not all the button presses are making it through is due to the fact that it won't actually begin recording until it recieves a trigger (within the patch) that the first (x)bars duration has past.

Its hard for me to describe but for example: if you have the setting at 4 bars for that pattern recorder the longest you may have to play that pattern before it begins playing back is 8 bars. If you have it set for 8 bars then you might have to play for 16 bars before it plays back. So that means the first few bars you play might not be recorded. The way to get recordings that are consistent is to always play patterns that are half the length. Meaning for an 8 bar recording play a 4 bar pattern. That way you're guaranteed. Sorry for the long explanation.

Regarding the the LED staying on: Sometimes a clip continues to loop underneath it all and you don't see it but the LED stays on. If you tap the button again it should turn back off after the duration of that clip.

This is happening because it is the better of two evils. One late night patching this thing I kept having this issue where every once in awhile when I would retrigger a clip it would just play for about a frame long and then cut off.

The reason was because when the clip reaches the last frame it sends a "0" to turn off viewing and playback. Sometimes I would retrigger that clip just as that "0" was coming through the pipe. So my re-trigger "1" signal would make it in there before the "0" from the first trigger which would result in a nasty blip. So I spent days trying to figure out how to cut the "0" off at the pass. I opted for the lesser of two evils which is the "on" always bypasses the "off" but the byproduct is a sometimes looping clip in the background.

Not the best answer but it made the software more reliable as an instrument. Soon I will implemtent group stops so that if you press another button in the same group it will send an "off" to all others in the same group.

Regarding the modes: I will try to fix that sooner than later. I spent most of my time developing in mode 1 and contemplated just removing mode 2 but it does come in kind of handy for those really quick repeats that you wouldn't be able to do naturally yourself.

As for playback performance. Are you using 15fps? Did you deinterlace? These are two aspects that might help a bit in case you haven't used them.

Also, I would happily make you a version with some windows friendly fonts as well. Tell me the font you prefer :)

Thanks for the feedback!
- CommentAuthoregon77
- CommentTimeSep 1st 2009 edited
Hey Agargara, I just wanted to report that I got to focus on these bugs a little tonight and I'm pretty sure I got two of my least favorites fixed.

1. The problem where suddenly a clip would no longer show video but audio would still come through is fixed. Now all soloing will always be above other patterns. This was the worst.

2. Sometimes when you would clear a pattern recorder the LED for that pattern recorder would stay lit. This no longer happens and anytime you clear the pattern the led goes out as well.

I played with this for at least an hour trying to get either of them to happen and I couldn't so I'm fairly positive they are fixed. They were actually shamefully easy to fix. I can't upload this yet because it's nearly 1am in Wisconson and I have no way of getting the patch from my laptop to this computer. In the morning I will ask the inlaws if they have any CDR's or flash drives otherwise I'll go pick some up.

Also I got mode 2 to work a little nicer with the pattern recorders

One more thing: Once I get this up I would love it if you could give it another go especially with an external clock. Because you said it got a little worse with an external clock I want to find out what kind of difference you are experiencing.

The way I get the external clock setup is like this: start ableton, start 64(video) fingers, press play on Ableton, go into stteings and select your clock input and set clock to external, turn the DAC off and on again. This should then get her going for you.

thanks again
- CommentAuthoregon77
- CommentTimeSep 1st 2009 edited
Okay here you go. Here is a new version of 64(Video)Fingers. Fixes are listed in the post above.

Good luck and have fun ya'll
- 64VideoFingers_1.02.zip
- CommentAuthorbrothernigel
- CommentTimeSep 2nd 2009
yo bro.. was messing around with this last night.. killed a few hours easily on this app alone.. haha.
- CommentAuthoregon77
- CommentTimeSep 2nd 2009
Glad to hear it! Keep on keepin' on. Always interested to hear about people using it. Send a heads up if you experience any issues. Got a new version already coming along with group stops (functionally no different but a little bit cleaner LED feedback and better CPU/GPU optimisation, a loadblack option so that when a video ends it cuts to black rather then freezing on the last frame, an overdub function for the pattern recorders and hopefully the ability to assign layers to video sequences when using it with your DAW midi sequencer.

Have fun!
- CommentAuthoregon77
- CommentTimeSep 5th 2009 edited
Okay so one more update. I got two new things going on this one.

First I have added a group stop function for all groups except the "None1-3" groups because those groups are polyphonic. Like I wrote in the earlier post it changes nothing feature-wise. It just creates cleaner LED feedback and uses less CPU/GPU processing because it guarantees that only one video clip will be playing at a time per group.

Second, I added a "Loadblack" button with an "on/off" toggle. This function means that when a video clip is done playing it will cut to black instead of freeze frame. You need to create your own black video clip that matches the frame rate, resolution and codec of the rest of your video files for best performance. I have provided two black.mov files with the zip which are set at 720X480; Photo-Jpeg codec; one at 29.97fps and one at 15fps consecutively. (On a side note: it doesn't have to be black. It could be looping cloud formations if you like. It's just the video file that it cuts to when nothing is playing)

Nothing super awesome but I wanted to get this one out because it is clean and mean. You can play with this one for hours and not have any hiccups along the way (knock on wood, of course). I started implementing the multiple midi channel function but it is a more complex addition then I had first thought and I feel like I am going to have to debug that addition before releasing it.

I hope you enjoy!
- 64VideoFingers_1.03.zip
- CommentAuthoregon77
- CommentTimeSep 23rd 2009
Okay, not sure if many people are using this app but new features abound.

So the latest is mutes. Once you've recorded a pattern and it begins to play back, you hold down the 7th button on the first row (the one to the left of the DAC button) and then press the pattern recorder and it will not only omit that pattern from the video window but also mute the audio as well. Pretty handy for quick variations in a song. Can't believe it's gone this long without mute control.

Midimode is coming along. In this version you can enable midimode from the settings menu in 64(video)fingers and select your input. Then from Ableton or whatever DAW you can send midi note information on that same port and trigger the video/audio clips. Make multiple midi tracks with the same port except different channels and you can begin to layer the video over one another. The hierarchy works like this: Channel1 = Bottom vid layer; Chan2 = 2nd above that; Chan3 = 3rd above 2nd; etc, etc. Up to 6 channels.

No way to quickly cut back and forth between the video layers in midimode yet but thats what I'm working on.

And after that, experimentation with overlays and alpha channels.

Video tutorial of setup, prep and use is still in the works. Plan on filming once I've got the midimode humming.

All the best.
- 64VideoFingers_1.04.zip
- CommentAuthorlauer303
- CommentTimeSep 23rd 2009
spending some time playing around today and it's even more fun than i expected... wow. :)

was ez to set up and get running on my 128 as well but still a few things i'm trying to work out... like is there a simple way to stop/mute a video clip once it's been triggered (mode 1)?

also any recommendations for a simple way to cut out audio from video clips i've recorded? quicktime pro or final cut or even ableton perhaps?

finally curious in anyone has recommendations on where to find some good video loops/clips? thinking something like freesound.org but for video?

great app! looking forward to updates!
- CommentAuthoregon77
- CommentTimeSep 24th 2009 edited
Awesome! So excited to have questions!

I'm happy you are finding some use out of it. First off, I apologize if a lot of this info is Mac based because it is what I know best.

So first question: Is there a simple way to stop/mute a video clip once it's been triggered (mode 1)? Not exactly, mode 1 is like a one shot mode. Mode2 has an option in the quantization dropdown menu for each particular slot of "None". This means that you press and hold down the button and the clip plays then let go and it stops without any of the quantized repeats. Another possible option within Mode1 is to load in a clip of black about 10 or 15 seconds long or so with no associated audio file and assign it the same group number as the clips you would like it to cut off.

2. Any recommendations for a simple way to cut out audio from video clips i've recorded? quicktime pro or final cut or even ableton perhaps?

I have heard people using Ableton but I have no experience without. Quicktime pro is a good option but my all around favorite is called MpegStreamclip. Find it here: http://www.squared5.com/

(Works for Mac and PC) It's very handy because you can load up long movie files, set ins and outs for the clip you want to export. And you have a lot of options. Plus it's free! (Fyi, some handy keyboard shortcuts: L = play; K=Stop; J=Reverseplay; I=Set in; O=Set Out; SHIFT+I=Goto In; SHIFT+O=Goto Out.)

btw I also recommend downloading Perian (mac only). Get it here: http://perian.org/
It's very handy in regards to numerous codec support. It would allow you to open FLV files directly as well.

3. Any recommendations on where to find some good video loops/clips? thinking something like freesound.org but for video?

My favorite is called the Prelinger archive. It is full of tons of stuff from old school educational, informational, newsreels, propaganda...plus tons of other stuff, you name it. It's a great place to look around.
Find it here: http://www.archive.org/details/prelinger

Also, of course there is vimeo where many of the videos you may download and there is a lot of high quality material there. Of course not sure of the rights of usage there but if you are just experimenting and not making any money I see no problem.

Youtube: There are downloaders available for firefox. If you're using safari just press Option+Command+A and it will bring up a window where you can download the video.

Lastly, I've just come across this site called OpenSourceCinema.org. Which is a database of video projects that people collaborate on but I have had trouble navigating the site. But could have potential.

Sorry so long. Hope this helped.
- CommentAuthorkillchnnl
- CommentTimeOct 6th 2009
Does Anyone know what format my movie files need to be? I'm trying to load them and they are all grey. I remember Egon saying ("For the best performance encode your videos with Motion-jpeg or photo-jpeg codec.") How does one make this happen??? Mpeg streamclip??? Can I not load mov. files?? what format will 64 video fingers take?? Questions! Questions!

feeds

New App! 64 (Video) Fingers.