MidiFx

[hfnukal]

Control MusicBeam effects with midi (or keys)

Run, switch to Midi Effect [0] and try with [asdfghjk] keys.

This is working concept of controlling MusicBeam via MIDI.
So far midi device is hardcoded. You can change it in MusicBeam class.

There is introduced new concept of defining effects. Definition is in fxconfig.json file.

You can define Gobo curve. It is array of vector points in range from -100 to 100.
Curve is defined as array of points under "curves" key fxconfig.json:
"SQUARE_GOBO": [0,100, 100,0, 0,-100, -100,0, 0,100],
See Processing vector() function

"fx" key defines each FX. Combination of gobo, start and stop state, duration and modificators you can create lot of variation of effects.

Fx is sequence that has duration. Duration 1 is one beat of 120 BPM. So far BPM is hardcoded, but ready to be exposed in GUI. start and end defines [x, y, rotation, scale]. To be able to address various resolutions, 0,0 is centre of screen, 1 is top, -1 bottom. Left and right borders depends on projector screen aspect ratio (eg. 16/9). Angle is decimal. For angle 0 is basic position, 1 is full rotation. Scale 1 is basic size, lower then 1 is smaller, greater then one is bigger.

You can use different approach. Most universal is to use GoboFx. GoboFx uses curves and displays it on screen in start position, moving to end position for duration. For GoboFx you need to define gobo. CurveGobo can be smooth or nosmooth, you can define gobo size and shape.

Example of Fx definition:
"PolyCircleGrow": { // name of Fx
"midinote": 49, // Midi note number to trigger Fx
"key": "s", // just for config, not used in code
"keyCode": 83, // keycode to trigger Fx
"FX": { // basic Fx definition
"GoboFx": { // GoboFx uses gobo. You can use custom Fx
"gobo": { // parameter for GoboFx, gobo
"CurveGobo": { // CurveGobo draws lines between points
"curve": "CIRCLE_GOBO", // curve definition for CurveGobo
"gobosize": 0.14, // initial size of Gobo
"nosmooth": null // Use line, smoots use vector
}
},
"hue": 220, // FX parametr for hue
"duration": 4, // duration of fx is 4 beats of 120 BPM
"start": [0, 0, 0, 1], // start position, angle and scale [x, y, angle, scale]) of the gobo in this case
"end": [0, 0, 1, 7] // end position, angle and scale
}
}
},

Description: By note 49 you trigger Fx. It starts with CIRCLE_GOBO, which basic size is 14% of screen defined by gobosize. No smooth draws circle with lines. This draws polygon. It will be visible 4 beats, duration parameter. Start in centre with no rotation and scale 1. Ends in full rotation and scale 7. Collor is defined by hue parameter.

Modificators can dynamically modify path of gobo from start to end. Eg modRotate ads rotation.

You can define you own Fx by extending Fx class.

For each Effect there is midi and key specification. Key midinote defines MIDI Note number, which triggers this Fx. keycode is key code to trigger fx from keyboard (key is required by not used, its for quick reference in configuration).

How I use it:
Run MusicBeamMidiFx and switch to MIDI EFFECT. Then trigger effects by midi messages. I use IAC Driver on Mac to control Fx by Ableton. You can setup IAC Driver in Audio MIDI Setup.app

You can reload config with button on GUI or by pressing "§" key. So its easy to tune your Fx definition.

There is lot of other features, I can describe if there is demand for it.

Some functions:

• Color can be defined with Fx otherwise is taken from hue slider.
• Gobo can be CurveGobo, DotGobo
• you can run several FX in same time using same midinote/keycode in definition
• bw switch to use white
• fill

The MidiBus 8 Library by Severin Smith is required for development.

Good luck, I hope somebody find this useful. Leave me comments, if there is interest I can continue with GUI and so on.

[codingjoe]

wow, thanks for the contribution @hfnukal
However I am going to need some time to review this. 1.5kloc isn't a small review and a don't use processing that frequent anymore. Bare with me. Maybe you can already add a litte more documentation to your code. That might make it easier for me to review. What do you think?
best
-joe

[hfnukal]

Sure, I will put some comments. Basically I want to know it anybody interested in MIDI control for MusicBeam. This is working for me. I can improve GUI and doc if there is interest.
So far it is not finished for public.

[codingjoe]

Hi @hfnukal sorry it took me so long to get into this. It was just a lot to wrap my head around, especially since I don't do much Java these days.

Before I start, let me say: I absolutely love what you did there!

I don't want to get into the code yet, I want to discuss concept first. I do love the idea of having configurable effects. People are often asking if they could write their own effects or are asking for more effects. Having them in a JSON or YAML format would be amazing.
What I don't think makes sense however is to have it both ways. So either everything is configurable via JSON or nothing.

Adding MIDI support is also great and was highly requested.

Here is my suggestion:
Can you create a second PR with only the concept implemented and a single of the existing effects switched to the new JSON configuration? I think once we have the concept nailed down, we can simply add all the effects you want and I can help to convert the existing effects into the new format.

Sounds good?

Best
-Joe

[hfnukal]

Hi Joe,

thanks for answer. I thing this PR has nearly what you request. There is single effect (MusicBeam/Midi_Effect.pde) that implements custom effect implementation. It is configurable by JSON.

In MusicBeam/Effect.pde I adjust size of buttons to fit new switch for this effect. there is also implementation of midi messages.

In MusicBeam/Strobe_Effect.pde is added mapping for midi messages. You can control controls Strobe_Effect with midi messages. It toggle buttons for effect. It is similar to keyCode mapping.

Back to Midi_Effect. There is few buttons for controlling. Most useful is button for reloading configuration also mapped to "§" key. It reload JSON configuration without need to restart app.

Draw method cycle in fxBuffer array and draws Fx. Also remove Fx when its finished.
Adding Fx to fxBuffer is triggered by midi or key event.

Lets describe Fx class. This is root class for effect definition. Basic concept for Fx is, that it has duration. When triggered, it plays some sequence for its duration and than Fx quits. Idea of duration is that 1 is one beat. It is calculated using framerate and BPM. So far BPM is constant. In future it can be detected or passed by midi.
For Fx you can specify start and end properties such is position, angle (rotation) and scale. Fx morph from start to end and adjust curve to make animation.
Coordinates centre (0,0) is in centre of the screen. Edge of screen is 1 or -1. You don't have to thing about screen resolution then. Coordinates are float numbers.
Simple direct implentation of such Fx is CirclePumpFx. It uses rotation property to adjust sise of cicrle. When rotation is greater then 2, it set remove property so Fx is then removed from fxBuffer.
It does not use start and end properties so far. I reimplement ScreenFx and SnowStormFx so it can be triggered with midi.

To be able to configure Fx, there is GoboFx class. It uses Gobo as basic curve. This curve is then modified by position, rotation and scale from Fx. First attempt was to use it in code. Then I came with JSON configuration. It is then easy to adjust without need of changing code and recompiling.

JSON configuration is in MusicBeam/fxconfig.json

Here is example of ScreenFx config:

    "ScreenFx":   {
	      "midinote": 57, <- midi note to trigger Fx
	      "key": "m", <- just visualise keycode, not used
	      "keyCode": 77, <- keycode to trigger Fx
	      "FX": { <- Fx definition section
	        "ScreenFx": { <- We want to use ScreenFx class
	          "duration": 4.0, <- duration is 4 beats
	          "modRotate": [1,1,1,1], <- modify, advanced animation, will be explained later
	          "start": [0, 0.3, 0, 1], <- start properties (x, y, angle, scale) not used by ScreenFx
	          "end": [0, 0.3, -0.5, 1.0]  <- end properties (x, y, angle, scale) not used by ScreenFx
	        }
	      }
	    },

Here comes the fun with Gobos. Gobos are defined separately in the beginning of config file. For easier editing, coordinates are from -100 to 100. Coordinates are vertex point. See example:

	    "LINE_GOBO": [-100,0, -100,0, 100,0, 100,0], 
	    "TRIANGEL_GOBO": [0,100, -86,-50, 86,-50, 0,100],
	    "SQUARE_GOBO": [0,100, 100,0, 0,-100, -100,0, 0,100],

LINE_GOBO is line from point -100,0 to 100,0. First and last coordinates are for vertex. See Processing doc https://processing.org/reference/vertex_.html

I tried use SVG for gobos. It is too complex for this use case. Simple curve gobo works better for MusicBeam style. Thickness of line can be adjusted, also smoothness and so on.

Once we have gobos defined lets use it with GoboFx:

    "LineFlyLeft":   { <- Fx name, must be unique
	      "midinote": 51, <- midi note to trigger Fx
	      "key": "f",
	      "keyCode": 70, <- key to trigger Fx
	      "FX": { <- Fx definition section
	        "GoboFx": { <- We want to use GoboFx class
	          "gobo": { <- gobo definition
	            "CurveGobo": { <- CurveGobo draws curve
	              "curve": "LINE_GOBO", <- use LINE_GOBO defined in config
	              "gobosize": 0.05, <- scale of gobo. 1 is fullscreen
	              "nosmooth": null <- no smoothing, no rounded angles. null mean no argument
	            }
	          },
	          "duration": 4.0, <- duration 4 beats
	          "start": [-0.5, -1.0, 0.0, 1.0], <- Start state: X: -0.5 Y: -1, angle: 0, scale: 1
	          "end": [1.0, 1.0, -1.0,17.0]  <- Start state: X: -1.0 Y: 1, angle: 1, scale: 17
	        }
	      }
	    }, 

LineFlyLeft configuration definition. It start on top of screen on left side. No rotation and default scale. End position is in down right corner, rotate full circle and scale to 17.

Another GoboFx:

    "PolyCircleGrow":   { <- name
	      "midinote": 49, <- midi note to trigger
	      "key": "s",
	      "keyCode": 83, <- key to trigger
	      "FX": {
	        "GoboFx": {
	          "gobo": {
	            "CurveGobo": {
	              "curve": "CIRCLE_GOBO", <- use CIRCLE_GOBO
	              "gobosize": 0.14, <- default scale
	              "nosmooth": null <- no smooth draws circle as polygon
	            }
	          },
	          "hue": 220, <- hue value, overrides default color
	          "duration": 4, <- duration 4 beats
	          "start": [0, 0, 0, 1], <- start at center 0,0, with no rotation and scale 1
	          "end": [0, 0, 1, 7]  <- end in center 0,0 with rotation 1 (full circle) and scale 7
	        }
	      }
	    },

Try to adjust parameters and reload with "§" key. Trigger by key or midi.

Here is parabola with smooth rendering. You can try change smooth to nosmooth to see different.

    "Parabola":   {
	      "midinote": 50,
	      "key": "d",
	      "keyCode": 68,
	      "FX": {
	        "GoboFx": {
	          "gobo": {
	            "CurveGobo": {
	              "curve": "PARABOLA_GOBO",
	              "gobosize": 0.04,
	              "smooth": null <- change smooth to nosmooth
	            }
	          },
	          "duration": 2.0,
	          "start": [1.0, -0.5, 0.0, 1.0],
	          "end": [-0.0, 0.0, 5.0, 5.0] 
	        }
	      }
	    },

Be careful with edits. So far no JSON checking is implemented so JSON must be valid. You need to restart MusicBeam to recover after attempt to load non valid JSON config.

Sometime scaling, rotation and position morphing is not enough. There is possible to add modifiers. Modifiers are inherited from Mod class. See ModRotate for example. Mod class takes 4 parameters and modify any parameter of Fx in mod method. Return modified Fx instance. ModRotate add rotation to path. See Triangel Fx.

I hope this explain concept a bit more. You can play with Fx live with midi or keyboard or connect to sequencer (Ableton Live, Logic, Cubase, etc) to get max power of it.

[codingjoe]

Hi @hfnukal thanks for the long answer. I think I wasn't clean enough in with my question. I know that everything I am asking for is already implemented in your PR. I did already try it out.
My point is: You are adding these new features on-top of the existing effects. I am asking if it's possible to no only add more but replace the existing effects.

[hfnukal]

Oh. I didn't dare to remove existing effects ;)
You mean simply remove all option buttons and start with only Midi_Effect active?
I left existing effects as they are very good for automatic mode. Also triggering by sound works great.
What about GUI?
Let me think about it. Or you have some idea how it should look?
There are too many possibilities in my head. I have to figure out.

• There should be some BPM control, with manual entry, tap tempo or auto detection from sound or midi.
• Fx can loop and there should be toggle/latch or single/loop mode
• Sound detection can fire some effects. It should be configured somehow. Maybe create some kind of sequence that will change step on each trigger.
• Some kind of Fx sets or Fx palette so you can group effect for easier access
• Fx config editor and validator, gobo editor
• Colour settings
• Line thickness setting
• Random mode
• Should Fx class replace Effect class?
• And all should be simple and easy to understand
• Should GUI display FX settings loaded from config and how? Piano keyboard, trigger pad (4x4, 8x8)

If you can throw some ideas or preference, it would be great. This is just what was momentary on my mind.

[codingjoe]

Oh. I didn't dare to remove existing effects ;)
You mean simply remove all option buttons and start with only Midi_Effect active?
I left existing effects as they are very good for automatic mode. Also triggering by sound works great.
What about GUI?
Let me think about it. Or you have some idea how it should look?
There are too many possibilities in my head. I have to figure out.

• There should be some BPM control, with manual entry, tap tempo or auto detection from sound or midi.

Currently we only detect beats not BPM. That should be possible though.

• Fx can loop and there should be toggle/latch or single/loop mode

Yes. Maybe it makes sense to have one active that is looping, but if you trigger one manually (key/midi) it is only triggered once.

• Sound detection can fire some effects. It should be configured somehow. Maybe create some kind of sequence that will change step on each trigger.

Sequences would be great, but complex to build. I would go with random selection for now. This should keep it simple an you can just run MusicBeam without being a pro.

• Some kind of Fx sets or Fx palette so you can group effect for easier access

Would be cool, but also hard to build. I would implement this at a later point too.

• Fx config editor and validator, gobo editor

JSON is fine, that works for most people. Some things should be configurable thought. Maybe the inputs could be defined in the JSON too?

• Colour settings

Maybe defined in the JSON as previously mentioned. You either specify a hard coded value in the JSON or a color input.

• Line thickness setting

That isn't really needed. Line thickness should be always relative to the viewport to yield good looking results.

• Random mode

Very important! I am all for advancing the features, but this is a beginners tool.

• Should Fx class replace Effect class?

Yes. I only want one concept for effects, not many. It is going to be hard to replace some effects with JSON config.

• And all should be simple and easy to understand

That's going to be the tricky part ;)

• Should GUI display FX settings loaded from config and how? Piano keyboard, trigger pad (4x4, 8x8)

I guess so.

If you can throw some ideas or preference, it would be great. This is just what was momentary on my mind.

I hope this helps. Keep the example simple. Once we nailed the UX and JSON schema, we can go nuts on effects.