This article is accompanied by video: Click Play Video
Combined with the article to see the effect is better!
Background of writing
Well, recently, some friends from B station commented on one of my videos and asked me to share how to make this BadApple dynamic effect. I don’t know what a badapple is.
However, since some of you have raised this question, I will write a complete tutorial, next time you ask, directly read this article, ensure that you can do it in any language, any framework.
Break down the requirements first
- Play the video
- Convert the picture of each frame of the video to the dot matrix/pixel RGB value
- Convert RGB to gray value
- Fill characters with gray values
The requirement is very simple, the slightly complicated part only has RGB to gray scale, then we directly open the code, using vanilla. Js framework (this is a reference, do not understand the search itself) to complete the development.
1. Play the video
Create a video tag with JS and set the video source path for it
var videoDom = document.createElement("video");
videoDom.src = "./video/badapple.mp4";
videoDom.style.width = "900px";
videoDom.style.height = "675px";
Copy the codeSince we don’t need to see the original video for the final effect, we don’t need to add the DOM to the body of the web page.
Add a button to control video playback and pause
var btnPlayAndPause = document.createElement("div");
btnPlayAndPause.style.color = "#fff";
btnPlayAndPause.style.textAlign = "center";
btnPlayAndPause.style.position = "absolute";
btnPlayAndPause.style.top = btnPlayAndPause.style.left = "0px";
btnPlayAndPause.style.width = videoDom.style.width;
btnPlayAndPause.style.height = btnPlayAndPause.style.lineHeight = videoDom.style.height;
btnPlayAndPause.style.cursor = "pointer";
btnPlayAndPause.style.fontSize = "30px";
btnPlayAndPause.style.zIndex = 2;
btnPlayAndPause.innerText = "play";
document.body.appendChild(btnPlayAndPause);
Copy the codeWhen the button is clicked, toggle videoDom’s play/pause state
btnPlayAndPause.addEventListener("click".function(){
if(btnPlayAndPause.innerText === "play"){
videoDom.play();
}else{ videoDom.pause(); }})Copy the codeListen for videoDom’s Canplay event and render the first frame
videoDom.addEventListener('canplay'.function(){
renderVideoFrame(videoDom);
});
Copy the codeListen to videoDom’s Play, Pause, and Stop events to start character rendering while playing, and to stop character rendering when paused or stopped.
videoDom.addEventListener('play'.function(){
console.log("Start playing");
btnPlayAndPause.innerText = "";
startRender();
});
// Listen to the playback end
videoDom.addEventListener('pause'.function(){
console.log("Play pause");
btnPlayAndPause.innerText = "play";
stopRender();
});
// Listen to the playback end
videoDom.addEventListener('ended'.function(){
console.log("End of playback");
btnPlayAndPause.innerText = "play";
stopRender();
});
Copy the codeRender the screen at the same rate as the browser, so you don’t lose any frames, but you consume more computing power.
var timerId;
function startRender() {
timerId = requestAnimationFrame(updateRender);
}
function updateRender(){
renderVideoFrame(videoDom);
timerId = requestAnimationFrame(updateRender);
}
function stopRender(){
cancelAnimationFrame(timerId);
}
Copy the code2. Convert the picture of each frame of the video into dot matrix/pixel RGB value
Here, we will use the CANVAS tag of HTML5 to draw the picture of the video intact on the canvas.
function renderVideoFrame(videoDom) {
var videoSize = {width:parseFloat(videoDom.videoWidth),height:parseFloat(videoDom.videoHeight)};
var canvas = document.querySelector("#canvas");
if(! canvas){ canvas =document.createElement("canvas");
canvas.id = "canvas";
canvas.style.width = videoDom.style.width;
canvas.style.height = videoDom.style.height;
canvas.style.position = "absolute";
canvas.style.zIndex = 1;
canvas.style.left = canvas.style.top = "0";
canvas.width = videoSize.width;
canvas.height = videoSize.height;
document.body.appendChild(canvas);
}
const ctx = canvas.getContext("2d");
ctx.drawImage(videoDom, 0.0, videoSize.width, videoSize.height);
}
Copy the codeNote that I have made a judgment here that I only create a canvas if it is not specified in the scene.
And then using the context’s drawImage method, we draw the video onto the scene, and now we don’t have the video tag on the body, but we can see the video.
Then we use the context’s getImageData method to get all the bitmap/pixel data in the canvas.
var imgData = ctx.getImageData(0.0, videoSize.width, videoSize.height).data;
Copy the codeThis is a large array, the length of the array is composed of width*height*4 (width x height x4), 4 represents the four values RGBA.
// If the canvas is 2 pixels wide and 1 pixel high, getImageData gets the array structure as follows
[r,g,b,a,r,g,b,a]
Copy the codeWith that in mind let’s look at how to get the RGBA value for a given position.
for (var h = 0; h < videoSize.height; h++) {
for(var w = 0; w < videoSize.width; w++){
var position = (videoSize.width * h + w) * 4;
var r = imgData[position], g = imgData[position + 1], b = imgData[position + 2]; }}Copy the codeThrough two for loops of canvas width and height, the initial sequence number of all the dots/pixels in the array is obtained by conversion.
- r = imgData[position]
- g = imgData[position + 1]
- b = imgData[position + 2]
As you can see, a canvas of 200×300 = 60,000 dots = 240,000 array length, we definitely can’t draw in pixel 1:1, which is too much calculation, and the drawing effect is not good, you can’t see the text content.
So we need to add an interval gap, such as 1:12, so that the computation is greatly reduced, but the accuracy of the drawing will also be reduced.
var gap = 6;
for (var h = 0; h < videoSize.height; h+=gap) {
for(var w = 0; w < videoSize.width; w+=gap){
var position = (videoSize.width * h + w) * 4;
var r = imgData[position], g = imgData[position + 1], b = imgData[position + 2]; }}Copy the codeTo achieve this we do not need alpha, the next big thing is to convert RGB to grayscale, which is then converted to stroke density text, such as black pixels we use the word ape instead.
3. RGB to gray value
- See this article for various RGB to gray scale algorithms
I use the second method listed in this article
Gray = (R*30 + G*59 + B*11 + 50) / 100 - 0.5
Copy the codeThe value is between 0.5 and 255.5
4. Fill the characters according to the gray value
We first create an array of grayscale characters, arranged in order of stroke density/visual grayscale (from highest to lowest), and finally leave a blank character to represent pure white.
var asciiList = ['apes'.'handsome'.'the old'.'big'.' '];
Copy the codeConvert the grayscale value to the serial number of the character array, using the math.min method to ensure that the serial number is not out of bounds
var i = Math.min(asciiList.length-1.parseInt(gray / (255 / asciiList.length)));
Copy the codeconclusion
This is a special effect THAT I’ve known since I started working with computers, and I’ve implemented it in various languages throughout my career. In fact, after dismantling the requirements, the core is to obtain the picture lattice information, RGB data to gray or binarization (only black and white). Then replace it with a character according to the gray information.
Hopefully, this tutorial will teach you how to develop this effect in any language, environment, and API.
One More Thing
In my usual style, I also made this special effect code into a favorites version, which can be used to play any video on STATION B
javascript:! (function(){console.log("badapple effect enabled");function renderVideoFrame(videoDom){var asciiList=['apes'.'handsome'.'the old'.'big'.' '];var scale=parseInt(videoDom.videoHeight/parseFloat($(videoDom).css("height")));var gap=12/scale;console.log(scale);var videoSize={width:parseFloat(videoDom.videoWidth/scale),height:parseFloat(videoDom.videoHeight/scale)};var canvas=document.querySelector("#badapplecanvas");if(! canvas){canvas=document.createElement("canvas"); canvas.id="badapplecanvas"; canvas.style.width=videoDom.style.width; canvas.style.height=videoDom.style.height; canvas.style.position="absolute"; canvas.style.background="#fff"; canvas.style.zIndex=999; canvas.style.top="0"; canvas.style.left=(parseFloat($(videoDom).css("width"))-videoSize.width)/2+"px"; canvas.width=videoSize.width; canvas.height=videoSize.height; videoDom.parentElement.appendChild(canvas)}const ctx=canvas.getContext("2d"); ctx.drawImage(videoDom,0.0,videoSize.width,videoSize.height);var imgData=ctx.getImageData(0.0,videoSize.width,videoSize.height).data; ctx.clearRect(0.0,videoSize.width,videoSize.height); ctx.font=gap+"px Verdana";for(var h=0; h<videoSize.height; h+=gap){for(var w=0; w<videoSize.width; w+=gap){var position=(videoSize.width*h+w)*4;var r=imgData[position],g=imgData[position+1],b=imgData[position+2];var gray=(r*30+g*59+b*11+50) /100;var i=Math.min(asciiList.length-1.parseInt(gray/(255/asciiList.length))); ctx.fillText(asciiList[i],w,h)}}}var videoDom=document.querySelector("video"); videoDom.style.display="none"; videoDom.addEventListener('canplay'.function(){renderVideoFrame(videoDom)}); videoDom.addEventListener('play'.function(){console.log("Start playing"); startRender()}); videoDom.addEventListener('pause'.function(){console.log("Play pause"); stopRender()}); videoDom.addEventListener('ended'.function(){console.log("End of playback"); stopRender()});var timerId;function startRender(){timerId=requestAnimationFrame(updateRender)}function updateRender(){renderVideoFrame(videoDom); timerId=requestAnimationFrame(updateRender)}function stopRender(){cancelAnimationFrame(timerId)}})()
Copy the codeCreate a new bookmark, copy and paste the code into the url, and use any name you like. Open any station B video page, click the book TAB, you can open the character painting playback mode, quickly try…
The source code to
Wechat search “big Handsome ape”, reply badapple to get the source code
Pay attention to my
Nuggets, bilibili, wechat public number can search “big handsome ape”, I heard that my friends have been promoted and raised yo ~