Give me access to your webcam
Browser Inputs
Browser Inputs - Overview
Analysing Mic Input
Introduction to the p5 sound library
-
Accessing the Webcam
Using the webcam as an input
-
Applying dat.gui in Existing Projects
Create GUI's for your sketches
What gives us access to all of these inputs
In short, we are relying on external libraries of prewritten code to allow us to gain easy access to things such as the webcam, mic of a computer and even to create a GUI (graphical user interface) which can make changes to our variables in real time.
To clarify P5.js is a library for javascript, its purpose is as a creative coding platform for new programmers. It allows us to get drawing to a canvas element without understanding any fundamentals of the web. We can use all of the prewritten code and get the visual output without understanding the specific technologies behind it.
P5.Sound Add-on Library
P5 is a library which we are extending further by adding the p5.sound addon library. This gives us sound related functions and lets us access the mic, play music, perform audio analysis, use instruments, etc. We will be focusing on these sound library methods
- p5.audioIn
- p5.Amplitude
- p5.FFT
Sound Library Reference
Analysing Mic Input
We have to take this to the P5.js editor as browser permissions are tricky business
let amplitude, mic;
function setup() {
createCanvas(windowWidth,windowHeight);
noStroke();
// create a new audio in object
mic = new p5.AudioIn();
// Turns on the mic
mic.start();
// Create a new amplitude object
amplitude = new p5.Amplitude();
// Sets the input of the amplitude object to the mic
amplitude.setInput(mic);
}
function draw() {
background(225);
// Always returns a number between 0 and 1
let level = amplitude.getLevel();
// Remapping the size of the ellipse based on amplitude
let size = map(level, 0, 1, 200, 500);
fill(50);
ellipse(width/2, height/2, size);
}
FFT (Fast Fourier Transform)
FFT (Fast Fourier Transform) is a very powerful tool for complex audio analysis live. It allows us to determine the amplitude at any given hertz range in the sound spectrum. The Human voice generally ranges between 80 Hz to 255 Hz. The range for general interaction such as common noises that humans make is around 100 Hz to 2000 Hz.
FFT can be used to pick out specific instruments in a song with some trial and error.
let hertzPoint = 0;
let sound, fft;
function setup() {
createCanvas(windowWidth, windowHeight);
sound = new p5.AudioIn();
sound.start();
// here we create a new FFT object
// Now we can access all of its methods
fft = new p5.FFT();
// currently fft is listening to your mic
fft.setInput(sound);
}
function draw() {
background(245);
// returns to us an array of all the frequencies
// in the audio input analysed
let spectrum = fft.analyze();
noStroke();
fill(0, 255, 0); // spectrum is green
for (let i = 0; i < spectrum.length; i += 10) {
let x = map(i, 0, spectrum.length, 0, width);
let h = -height + map(spectrum[i], 0, 255, height, 0);
rect(x, height, width / (spectrum.length / 10), h);
}
// Here we visualize the position the mouse is over
fill(255, 0, 0);
rect(mouseX, 0, 2, height);
hertzPoint = round(map(mouseX, 0, width, 0, 20000));
fill(50); // text is white
textSize(40);
textAlign(CENTER, TOP);
text(hertzPoint + " Hz", width / 2, height / 10);
}
function mousePressed() {
// This is to tell the browser that you are consenting to audio being recieved
// browsers generaly have an issue with giving webcam or audio input
// without a user input indicating it is okay
getAudioContext().resume();
}
Accessing the Webcam in P5
We can request access to the webcam by calling the method
createCapture(VIDEO).
This shortcut derives from the P5.js dom library we have have a look at all the methods available too us.
let capture;
function setup() {
createCanvas(windowWidth, windowHeight);
// Creates a new capture object
capture = createCapture(VIDEO);
// Apply size parameters to it
capture.size(width, height);
// Hide the original video element created
capture.hide();
}
function draw() {
background(255);
// Draw the capture objec to screen
image(capture, 0, 0, width, height);
}
Manipulating Webcam Input
let capture;
function setup() {
createCanvas(windowWidth, windowHeight);
noStroke();
capture = createCapture(VIDEO);
// Apply size parameters to it
// Hide the original video element created
capture.hide();
}
function draw() {
background(255);
// Draw the capture object to screen
image(capture, 0, 0, width, height);
// We can use the .get method to sample the
// Color data of a capture object
let myColor = capture.get(capture.width/2, capture.height/2,1,1);
// .get returns an an image when used on a capture object
// They use the image funciton to refer to the color data
image(myColor, width/2, height/2, 50,50);
}
Basic Application of dat.gui
The reference for this library is here, be warned that the examples are in ES5 javascript syntax!
Dat.Gui reference
let settings;
function setup() {
createCanvas(windowWidth, windowHeight);
noStroke();
// Here we give our settings object parameters
settings = {
size: 100,
};
// Create a new gui object using the classes from dat.gui
let gui = new dat.GUI();
// Create a new object with our utilitiy class
// Use the gui object to add parameters to be edited by the GUI
// Here we are adding the size property from the setting varible
// We will cover variable properties next slide
gui.add(settings, 'size', 50, 500, 1);
}
function draw() {
background(230);
fill(50);
// We access object parameters with the dot object remembering back to classes
ellipse(width/2, height/2, settings.size);
}
Javascript Objects for Storing Properties
Another new sytnax to be aware of is the javascript object. This is related to what we have covered in classes last week but the syntax is a little different. Below is a very basic example of a javascript object being applied.
let myVariable = 230;
// JS objects can store any type of value
// Including arrays
let myObject = {
x: 200,
y: 200,
width: 200,
color: [255, 0, 0]
}
function setup() {
createCanvas(windowWidth, windowHeight);
// Referencing to a property that does not yet exists
// Creates a new rotate property for myObject
myObject.height = 300;
}
function draw() {
background(myVariable);
// We call all of our parameters with the dot sytnax
fill(myObject.color);
rect(myObject.x, myObject.y, myObject.width, myObject.height);
}
Complex use of dat.gui
In this example we have three different GUI types. They require the use of javascript objects and classes with methods to run functions on button pressed. The color picker is editing an array with three positions for RGB, the slider is editing a property and the save button is running a method in a class.
Complex use of dat.gui
let settings, myColor;
// Here we have a utility class which will store methods for
// dat.gui to to access and run
class utility {
// This method saves a png of what is currently on canvas
save() {
saveCanvas('savedCanvas');
};
};
function setup() {
createCanvas(windowWidth, windowHeight);
noStroke();
// Here we give our settings object paratamers
settings = {
size: 100,
color1: [random(255), random(255), random(255)]
};
// Create a new gui object using the classes from dat.gui
let gui = new dat.GUI();
// Create a new object with our utilitiy class
let utilities = new utility();
// Use the gui object to add parameters to be edited by the GUI
// Here we are adding the size property from the setting varible
// We will cover variable properties next slide
gui.add(settings, 'size', 50, 500, 1);
gui.addColor(settings, 'color1');
gui.add(utilities, 'save');
}
function draw() {
background(255);
// We can put this array into our fill since there are 3 values for
// each color parameter
fill(settings.color1);
// We access object parameters with the dot object remembering back to classes
ellipse(width/2, height/2, settings.size);
}