webgl不同图像不同纹理

In this tutorial you will learn how to use WebGL for image processing. We will cover basic stuff like initialization, texture loading, and simple fragment shaders. I will try to cover all aspects of interaction with WebGL but you should have decent understanding of vanila javascript. If you want more in depth explanations, there is a good book called "Learn WebGL", check it out.

在本教程中，您将学习如何使用WebGL进行图像处理。 我们将介绍一些基本内容，例如初始化，纹理加载和简单的片段着色器。 我将尝试涵盖与WebGL交互的所有方面，但是您应该对vanila javascript有相当的了解。 如果您想要更深入的解释，有一本好书叫做“ Learn WebGL”，请阅读。

Here are examples of what we will do:

以下是我们将执行的操作的示例：

图像扭曲 (Image twist)

影像模糊 (Image blur)

SOURCE CODE

源代码

If you want to skip theory and build setup click here.

如果您想跳过理论并建立设置，请单击此处 。

理论 (Theory)

Everybody who tries to implement good graphics quickly understands he now has severe performance issues. The amount of computation required to produce any decent scene is simply huge. You need to process every polygon (some models have thousands of them) to project them on screen. Then you need to rasterize them. Apply textures, shading, reflections for every pixel on the screen. And you also need to do all of this stuff at least 30 times in a second. CPUs just can't handle this very well. Especially when using some scripting language with lots of overhead for every operation.

每个尝试实现良好图形的人都会很快了解他现在面临严重的性能问题。 产生任何体面场景所需的计算量简直是巨大的。 您需要处理每个多边形(某些模型有数千个多边形)才能在屏幕上投影它们。 然后，您需要对其进行栅格化。 为屏幕上的每个像素应用纹理，阴影，反射。 而且您还需要在一秒钟内至少完成30次这些工作。 CPU只是不能很好地处理此问题。 尤其是在使用某些脚本语言时，每项操作都会产生大量开销。

Luckily people found a solution and invented GPUs. All of the tasks described above are highly parallel in their nature. Polygons and pixels usually do not depend on each other and can be easily (and much more efficiently) processed at the same time. GPUs are especially good at this. While modern processors usually have 4-8 cores, any decent graphics card has thousands of them. They are much less complex then CPU cores and highly optimized for specific 3D-related calculations.

幸运的是，人们找到了解决方案并发明了GPU。 上述所有任务本质上都是高度并行的。 多边形和像素通常互不依赖，并且可以轻松地(并且效率更高)同时进行处理。 GPU在这方面尤其擅长。 尽管现代处理器通常具有4-8个内核，但任何一款不错的图形卡都具有数千个。 它们比CPU内核复杂得多，并且针对特定的3D相关计算进行了高度优化。

WebGL is a web standard for low-level 3D graphics API. It allows you to run your code directly on GPU, giving you all it's power. You write all of the rendering code in OpenGL Shading Language aka GLSL. It's not hard and very similar to C. Programs written in GLSL usually called shaders. They are compiled and loaded into a graphics card in runtime using WebGL API.

WebGL是用于低级3D图形API的网络标准。 它使您可以直接在GPU上运行代码，从而发挥全部功能。 您可以使用OpenGL阴影语言(又称为GLSL)编写所有渲染代码。 这并不难，并且与C非常相似。用GLSL编写的程序通常称为着色器。 使用WebGL API在运行时将它们编译并加载到图形卡中。

制备 (Preparation)

Technically you don't need to install anything. But without a proper web server you won't be able to load images and additional scripts. So it's a good idea to have one. I will use webpack-dev-server, it's easy to setup and use.

从技术上讲，您不需要安装任何东西。 但是如果没有适当的Web服务器，您将无法加载图像和其他脚本。 因此，拥有一个是一个好主意。 我将使用webpack-dev-server，它易于设置和使用。

First thing that you need to do is create an empty folder and run npm init inside. You can skip all of the questions from NPM.

您需要做的第一件事是创建一个空文件夹并在其中运行npm init 。 您可以跳过NPM中的所有问题。

Then add this lines to package.json

然后将此行添加到package.json

{"scripts": {"build": "webpack","serve": "webpack-dev-server"},"devDependencies": {"copy-webpack-plugin": "^5.1.1","html-webpack-plugin": "^4.2.0","raw-loader": "^4.0.1","webpack": "^4.43.0","webpack-cli": "^3.3.11","webpack-dev-server": "^3.10.3"}}

Run npm install and create a new file named webpack.config.js.

运行npm install并创建一个名为webpack.config.js的新文件。

Here is the contents of webpack config:

这是webpack配置的内容：

const path = require('path');
const HtmlWebpackPlugin = require('html-webpack-plugin');
const webpack = require('webpack');
const CopyWebpackPlugin = require('copy-webpack-plugin');module.exports = {entry: './index.js',output: {path: path.resolve(__dirname, 'dist'),filename: 'index.js',},plugins: [new HtmlWebpackPlugin({template: 'index.html'}),new CopyWebpackPlugin([{from: "styles/*.css",to: ""}])],mode: 'development'
};

Now you can start dev server by running npm run serve and open http://localhots:8080. I will not explain this all deeply, as this is not the main topic. Everything should work out of the box.

现在，您可以通过运行npm run serve并打开http：// localhots：8080来启动开发服务器。 我不会对此进行深入解释，因为这不是主要主题。 一切都应该开箱即用。

码 (Code)

Let's deal with HTML right away.

让我们立即处理HTML。

All we need is a canvas, so here it is.

我们需要的只是一块画布，所以就在这里。

index.html (index.html)

<html>
<head><title>Webgl</title>
</head>
<body><canvas id="c"></canvas><div class="slidecontainer"><input type="range" min="0" max="30" value="0" class="slider" id="range"></div>
</body>
</html>

Just basic HTML template with a canvas and slider that we can use.

只是可以使用的带有画布和滑块的基本HTML模板。

Now, it's time to initialize WebGL.

现在，该初始化WebGL了。

index.js (index.js)

// Using webpack's raw loader to get shader code as JS string.
// Much more convenient than writing them directly as string
// or loading in runtime
import vert from '!raw-loader!./vertex.glsl';
import frag from '!raw-loader!./fragment.glsl';function prepareWebGL(gl) {// Creating and compiling vertex shadrlet vertSh = gl.createShader(gl.VERTEX_SHADER);gl.shaderSource(vertSh, vert);gl.compileShader(vertSh);// This line is very import // By default if shader compilation fails,// WebGL will not show you error,// so debugging is almost impossibleconsole.log(gl.getShaderInfoLog(vertSh));// Creating and compiling fragment shaderlet fragSh = gl.createShader(gl.FRAGMENT_SHADER);gl.shaderSource(fragSh, frag);gl.compileShader(fragSh);// This line is very import // By default if shader compilation fails,// WebGL will not show you error,// so debugging is almost impossibleconsole.log(gl.getShaderInfoLog(fragSh));// Linking program and passing it to GPUlet prog = gl.createProgram();gl.attachShader(prog, vertSh);gl.attachShader(prog, fragSh);gl.linkProgram(prog);gl.useProgram(prog);gl.viewport(0,0,c.width,c.height);return prog;
}

Error checks are omitted for clarity. So, how exactly WebGL works? It's sort of a separate world. You pass some programs and data there. Then you GPU executes your program with your data and gives back an image.

为了清楚起见，省略了错误检查。 那么，WebGL到底如何工作？ 这是一个独立的世界。 您在那里传递了一些程序和数据。 然后，GPU使用数据执行程序并返回图像。

Every program consists of two parts: vertex and fragment shaders. Vertex shader is applied to every vertex or just point in space, that you pass in. Here you perform all 3D stuff such as transformations, projections, and clipping. Then GPU rasterizes your shapes, which means filling them with pixels aka fragments. Actually fragments are not exactly pixels. But for the scope of this tutorial they can be used interchangeably. After rasterization every fragment is passed through fragment shader to determine it's color. Finally everything is drawn to the framebuffer and displayed on the screen.

每个程序都由两部分组成：顶点着色器和片段着色器。 顶点着色器将应用于传入的每个顶点或空间中的每个点。在这里，您将执行所有3D内容，例如变换，投影和裁剪。 然后GPU栅格化您的形状，这意味着用像素(也称为碎片)填充形状。 实际上，片段并非完全是像素。 但是在本教程的范围内，它们可以互换使用。 栅格化后，每个片段都会经过片段着色器以确定其颜色。 最后，所有内容都被绘制到帧缓冲区并显示在屏幕上。

It's important to understand that your shaders are executed in parallel. On every vertex and every fragment independently. Also, shaders produce values not by returning them but setting special variables. Such as gl_Position and gl_FragColor, treat them as return statements.

了解着色器是并行执行的，这一点很重要。 在每个顶点和每个片段上独立存在 。 同样，着色器不是通过返回值而是通过设置特殊变量来产生值。 如gl_Position和gl_FragColor ，将它们视为return语句。

For the sake of simplicity we will mostly play with fragment shaders and stay in a 2D world. Here is simple pass-through vertex shader:

为了简单起见，我们将主要使用片段着色器并停留在2D世界中。 这是简单的传递顶点着色器：

顶点 (vertex.glsl)

// This is our input from js world
attribute vec2 coords;
// This is output for the fragment shader
// varying variables are a little special
// you will see why later
varying highp vec2 vTextureCoord;void main (void) {// Texture and verticies have different coordinate spaces// we do this to invert Y axisvTextureCoord = -coords;// Setting vertix position for shape assembler // GLSL has many convenient vector functions// here we extending 2D coords vector to 4D with 2 values// 0.0 is a Z coordinate// 1.1 is a W, special value needed for 3D math// just leave it 1 for nowgl_Position = vec4(coords, 0.0, 1.0);
}

Later we will fill our canvas with a rectangle. This is needed to have some plane for applying textures. To understand how fragment shader works you need to comprehend varying variables. Let's imagine you have two vertices in a line. In vertex shader you set some varying variable to red color from one of them and to green from another. All fragments between these two points will get different values when reading this varying variable. It will smoothly transition from red to green. So if you set fragment color to the value of this variable you will get something like this. Such behavior is called interpolation.

稍后，我们将用矩形填充画布。 这需要具有一些平面来应用纹理。 要了解片段着色器的工作原理，您需要理解各种变量。 假设您在一条直线上有两个顶点。 在顶点着色器中，您可以将其中一个变量设置为红色，将另一个变量设置为绿色。 读取此变量时，这两点之间的所有片段将获得不同的值。 它会从红色平滑过渡到绿色。 因此，如果将片段颜色设置为该变量的值，则将得到类似的内容。 这种行为称为插值。

We will also use a couple of uniform variables. But no need to worry, they are quite simple. Uniform variables are just constant parameters. Useful for passing global setting and texture ids.

我们还将使用几个统一变量。 但不用担心，它们非常简单。 统一变量只是常量参数。 用于传递全局设置和纹理ID。

片段 (fragment.glsl)

// Setting precision for float calculations
precision mediump float;
// This is input from vertex shader
varying highp vec2 vTextureCoord;
// Samplers are needeed to select textures
// actually its integers
uniform sampler2D uSampler;
// This will tell us how much to screw the image
uniform float iter;vec2 coords;
float x;
float y;
float l;
void main(void){// Getting distance from originl = length(vTextureCoord);// Just renaming to reduce typingx = vTextureCoord[0];y = vTextureCoord[1];// Rotating point around origin coords[0] = x * cos(iter * l) - y * sin(iter * l);coords[1] = x * sin(iter * l) + y * cos(iter * l);// Transforming coordinates from GL space to texture space// All math can be done directly to vectorscoords = coords / 2.0 - 0.5;// Fragment shader must set this variablegl_FragColor = texture2D(uSampler, coords);
}

The key here is to understand that angle of point rotation is dependent on it's distance to the center. This will result in a cool effect of texture twisting and sucking in a black hole. If we delete the * l part, the whole thing will just rotate evenly.

这里的关键是要了解点旋转的角度取决于它到中心的距离。 这将导致纹理扭曲和在黑洞中吸吮的凉爽效果。 如果删除* l部分，则整个对象将均匀旋转。

Our program is now ready, compiled, and loaded. Time to deal with the data. Here we are loading vertex coordinates to WebGL memory. Just two triangles to cover the canvas so we have some surface for texturing. Be aware that WebGL coordinates work different from canvas coordinates. Unlike canvas, it's origin is at the center and all coordinates are normalized. So no matter what aspect ratio your canvas has, X and Y are always in -1 to 1 range. Also Y coordinate is pointing upwards.

现在，我们的程序已准备就绪，已编译并已加载。 是时候处理数据了。 在这里，我们将顶点坐标加载到WebGL内存中。 仅两个三角形覆盖了画布，因此我们有一些曲面可以进行纹理处理。 请注意，WebGL坐标的工作方式不同于画布坐标。 与画布不同，它的原点位于中心，所有坐标均已标准化。 因此，无论您的画布具有什么纵横比，X和Y始终在-1到1的范围内。 Y坐标也指向上方。

index.js (index.js)

function setArrays(gl, prog){// Getting WebGL buffer objectlet vertex_buffer = gl.createBuffer();// This is 2 triangles that form a square// Each triangle consists of 3 points// Each point consists of two numbers: X and Y coordinates// GL coordinate space has origin in center// and spans from -1 to 1 on both axes// here is why we need to transform our coords// in fragment shaderconst vertices = [-1.0, -1.0, 1.0, -1.0, -1.0, 1.0,1.0, 1.0, 1.0, -1.0, -1.0, 1.0,]// Loading our data as ARRAY_BUFFERgl.bindBuffer(gl.ARRAY_BUFFER, vertex_buffer);gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices), gl.STATIC_DRAW);// Finding location of variable "coords" in GL memory// and binding ARRAY_BUFFER to itlet coord = gl.getAttribLocation(prog, "coords");// Variable binds to last buffer that was written to GL memorygl.vertexAttribPointer(coord, 2, gl.FLOAT, false, 0, 0);gl.enableVertexAttribArray(coord);// ARRAY_BUFFER is now free and can be reusedreturn [coord, vertex_buffer];
}

Array data is loaded. Time for textures.

阵列数据已加载。 时间纹理。

Better to find one that has sizes of powers of two.

最好找到一个具有2的幂的大小。

WebGL able to automatically generate mipmaps for such textures and scale them properly. At first we create simple 1x1 blue pixel texture and immediately return it. Later, when image loads, we replace this pixel with proper texture data.

WebGL能够自动为此类纹理生成mipmap并正确缩放它们。 首先，我们创建简单的1x1蓝色像素纹理并立即将其返回。 稍后，当图像加载时，我们用适当的纹理数据替换此像素。

index.js (index.js)

function loadTexture(gl, prog, url) {// Creating 1x1 blue tuxtureconst texture = gl.createTexture();const level = 0;const internalFormat = gl.RGBA;const width = 1;const height = 1;const border = 0;const srcFormat = gl.RGBA;const srcType = gl.UNSIGNED_BYTE;const pixel = new Uint8Array([0, 0, 255, 255]);  // opaque blue (RGBA)// bindTexture works similar to bindBuffergl.bindTexture(gl.TEXTURE_2D, texture);gl.texImage2D(gl.TEXTURE_2D, level, internalFormat,width, height, border, srcFormat, srcType,pixel);// Loading imageconst image = new Image();image.onload = function() {gl.bindTexture(gl.TEXTURE_2D, texture);gl.texImage2D(gl.TEXTURE_2D, level, internalFormat,srcFormat, srcType, image);// WebGL1 has different requirements for power of 2 images// vs non power of 2 images so check if the image is a// power of 2 in both dimensions.if (isPowerOf2(image.width) && isPowerOf2(image.height)) {// Yes, it's a power of 2. Generate mips.gl.generateMipmap(gl.TEXTURE_2D);} else {// No, it's not a power of 2. Turn off mips and set// wrapping to clamp to edgegl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);}// Quik re-render to display new texture// See implementation belowrender(0);};// Triggering loadimage.src = url;return texture;
}
function isPowerOf2(value) {return (value & (value - 1)) == 0;
}

Preparations done. Now is time to tie everything together.

准备工作已经完成。 现在是时候将所有内容捆绑在一起。

index.js (index.js)

function main(){// Getting WebGL context from canvasconst c = document.getElementById("c");c.width = 600;c.height = 600;// Getting sliderconst range = document.getElementById("range");const gl = c.getContext("webgl");const prog = prepareWebGL(gl);const coord = setArrays(gl, prog);const texture = loadTexture(gl, prog, "img.jpg");// Handle to control amount of twistconst iter = gl.getUniformLocation(prog, "iter");const uSampler = gl.getUniformLocation(prog, 'uSampler');// As is said samplers are just integers// Tell the shader to use texture 0gl.uniform1i(uSampler, 0)// This is main workhorserender = (it) => {// Binding texture to slot 0gl.activeTexture(gl.TEXTURE0);gl.bindTexture(gl.TEXTURE_2D, texture);// Filling screen with black colorgl.clearColor(0.0, 0.0, 0.0, 1.0);gl.clear(gl.COLOR_BUFFER_BIT);// Setting iter to slider valuegl.uniform1f(iter, it);// Triggering webgl rendergl.drawArrays(gl.TRIANGLES, 0, 6);}render(0);range.addEventListener("input", (e) => {render(e.target.value);})
}main()

That's it. You can try to follow along and write it yourself. If you totally stuck on something, the working source from demos above is here.

而已。 您可以尝试自己编写它。 如果您完全固定在某个东西上，则上面的演示中的工作源在这里 。

奖励：模糊着色器 (Bonus: Blur shader)

GLSL is quite restrictive. For example you can't write loops with non-constant bounds. This may seem strange at first, but actually have decent reasoning behind it. Most of the restrictions are needed to help compilers apply some aggressive optimization techniques.

GLSL具有严格的限制。 例如，您不能编写具有非恒定边界的循环。 乍一看这似乎很奇怪，但实际上背后有合理的理由。 需要大多数限制来帮助编译器应用一些积极的优化技术。

So, here is the blur shader.

因此，这是模糊着色器。

片段 (fragment.glsl)

varying highp vec2 vTextureCoord;
uniform sampler2D uSampler;
precision mediump float;
uniform float iter;
uniform float uTextureSize;void main(void){float pixel = 1.0 / uTextureSize;vec2 coords;vec4 color = vec4(0.0, 0.0, 0.0, 0.0);float div = (iter + 1.0) * (iter + 1.0) * 4.0;for (int i = 0; i <= 100; i++) {if (float(i) > iter){break;}for (int j = 0; j <= 100; j++) {if (float(j) > iter){break;}coords = vTextureCoord.st / 2.0 - 0.5;coords += vec2(float(i), float(j)) * pixel;color += texture2D(uSampler, coords).rgba / div;coords = vTextureCoord.st / 2.0 - 0.5;coords -= vec2(float(i), float(j)) * pixel;color += texture2D(uSampler, coords).rgba / div;}int i2 = -i;for (int j = 0; j <= 100; j++) {if (float(j) > iter){break;}coords = vTextureCoord.st / 2.0 - 0.5;coords += vec2(float(i2), float(j)) * pixel;color += texture2D(uSampler, coords).rgba / div;coords = vTextureCoord.st / 2.0 - 0.5;coords -= vec2(float(i2), float(j)) * pixel;color += texture2D(uSampler, coords).rgba / div;}}gl_FragColor = vec4(color.rgb, 1.0);
}

It's not the best, but it works. And I hope you can learn something from it.

这不是最好的，但是可以。 我希望您能从中学到一些东西。

主意 (Ideas)

You can use this project as a template for future experiments.

您可以将该项目用作将来实验的模板。

Here are some cool ideas:

这里有一些很酷的想法：

• Sobel Edge detection

  Sobel边缘检测

• Hue shift

  色相偏移

• Variuos photo filtersVariuos照片滤镜

Good luck in your programming journey.

祝您编程愉快。

翻译自: https://habr.com/en/post/501080/

webgl不同图像不同纹理