Hey there, I'm Wes Bos

So, HTML5 Video eh?

The Basics

Support or UGH IE

Lets take a look at the code

<video controls>
	<source src="Chrome_ImF.mp4" type="video/mp4;">
	<source src="Chrome_ImF.webm" type="video/webm;">		        
	<source src="Chrome_ImF.ogv" type="video/ogg;">
    <p>Your browser sucks, insert a flashfallback here.</p>
</video>

The browser vendors haven't agreed on a codec so we have to serve up three :(

Chrome, Opera and FF4+ = .webm
IE = .MP4 (and .webm if installed)
FF 3.5,36 = .ogg
safari/ios =.MP4

The order above is important for ios compatability.

Miro Video Converter is your friend!

Notable Attributes

First, select your video

var vid = document.querySelector('#myVid'); 

Then use the methods to control it: play, pause, skip ahead, play, volume up.

vid.play(); vid.pause(); vid.currentTime+=10; vid.volume+=0.1

You can control the playback rate, volume and current time which is enough to make your own video player UI

You also have access to all the data/properties of the video element.

vid.duration; vid.currentTime;

Events

Like any element, you can also subscribe to events. Thre are a number that are specific of HTML5 video/audio

These are handy when you are building an application that works with video

vid.addEventListener('seeking', function() {
	console.log("a scrub is a guy who thinks he's fly");
}); 

vid.addEventListener('seeked', function(e) {
	ga(e.type,this.currentTime); // send seek data to analytics
}); 

Creating a Custom Player

VideoSWS (see what sucks) is an amazing resource to compare available players.

Add a dash of CSS3

Because HTML5 Video is just another HTML element, we can use CSS to transform and style our player - with or without our HTML controls

<video> and <canvas> are best friends!

HTML5 Video Manipulation!

Super Basic Example

Video Element

Canvas Element

On a set interval or requested animation frame we take a frame from the video and use the canvas's ctx drawImage() method to write part of all of the frame.

ctx.drawImage(videoEl, x, y, width, height);

Cool, cool cool cool.

But what can you do with that?

Slice and Dice

While the video is playing, we can cut it into tiles and then spit them into a canvas at a random order.

Kind of a silly example, but it gives you the idea that we have total control over how and where the pixels are displayed

Filters and Pixel Manipulation

Some Code

Lets step through a basic example. First setup your HTML.

<video id="example4" width="700" height="350">
      <source id="mp4" src="localVid.mp4" type="video/mp4">
</video>
<canvas id="example4Canvas" ></canvas> 

Now with JS, grab hold of your elements and get the canvas's context.

var example4 = document.querySelector('#example4'),
	canvas4 = document.querySelector("#example4Canvas"),
	ctx4 = canvas4.getContext("2d"),
	example4Pixels;

We also created a variable called 'example4Pixels' to hold our pixels once we nab them from the canvas.

Now that we are set up, we have 4 steps that will repeat every 50ms

or at whatever framerate your choose.

1. Take the frame from the video and write it to canvas
2. Grab the pixels from the canvas and store in our example4Pixels variable
3. Loop over each pixel and adjust your RGBA channels accordingly
4. Spit those pixels back into the canvas

Note: For simplicity, I'm using a single canvas but its often a good idea to have two: One for dumping the pixels into and one for dumping the manipulated pixels.

ctx4.drawImage(example4, 0, 0, example4.width, example4.height);

The above code grabs the current frame from our 'example4' video and writes to canvas.

example4Pixels = ctx4.getImageData(0,0,canvas4.width,canvas4.height);

ProTip: Don't console.log() this variable as its HUGE and will crash your browser.

This is where the fun parts comes in. Every 4 numbers represent 1 pixel.

100		235		54		0.35 	34		245		199		1
RED 	GREEN 	BLUE 	ALPHA 	RED 	GREEN 	BLUE 	ALPHA	

// Loop over RGBA
for (i = 0; i < example4Pixels.data.length; i=i+4) {

example4Pixels.data[i + 0] = example4Pixels.data[i + 0] + 100 ; // RED
example4Pixels.data[i + 1] = example4Pixels.data[i + 1] - 50; // GREEN
example4Pixels.data[i + 2] = example4Pixels.data[i + 2] * 0.5; // Blue

}	

In this example, I upped our RED by 50, decreased GREEN by 50 and multiplied BLUE by 0.5. Alpha is left at 1.

The end result is a red tint on the the video.

Web RTC

( or HTML5 Video & Audio Hardware Access! )

( SO COOL )

WebRTC? Real Time Communication

Say Hello to getUserMedia()

PeerConnections aside, this new spec means that browsers are currently implementing native webcam and microphone access!

Thats right, no flash required!

getUserMedia is currently supported in Opera, Opera Mobile 12 and Google Chrome Canary (or dev channel)

Will hit boot2geko in August-ish

Cool, demo it!

A quick word on performance

HTML5 Video + Canvas in the real world

Popcorn.js [link]

Popcorn.js

Built by Bocoup / Mozilla

Allows you to create interactive HTMl5 web pages that sync up with a video/audio element

Think cuepoints on steroids

Huge Library of plugins to work with everything from twitter to Google Maps Street View

DMV

Build by the super awesome Rick Waldron

Uses WebRTC, Node.js and web sockets to create an online photobooth and save images to the server

100% JavaScript!

Source on Github and demo at http://dmv.nodejitsu.com/

OSX photobooth effects [link]

getUserMedia() Shim

The Wilderness Downtown[link]

Ro.me[link]

So much cool, so little time

Check out a ton more video experiments at Chrome Experiments

Augmented Reality

In the same way we analyzed the pixels in the last examples, we can use an Augmented Reality lib to detect markers.

Here we use JSAR Toolkit which was ported from flash and a wrapper written by f1lt3r/bocoup

Summary

HTML5 Video is awesome

You can use Javascript to control every aspect of it

Canvas makes manipulating video fun and easy

Go make cool things!

Cool Stuff EH?
The future is awesome
Questions?

@wesbos on Twitter