The video.requestVideoFrameCallback() method is soon coming to all browsers. This means you'll soon be able to run code on every frame of a playing <video>, which can be useful to, for instance, update a <canvas> at the same time.
Here's a quick demo video showing how I used this to compute the average color of a video frame (by dropping the frame into a canvas and getting the image color data), and set the page's background color to it.