**PV112: Computer Graphics API** Course Notes Introduction =============================================================================== This course expects knowledge at the level of PB071 course. In case you have not yet attended the course or you are attending it alongside this one we provide a short introduction to tools you need to know. ## Structure of the Course - We are currently holding the course live in A318 and B311 lecture rooms. - We will make the recordings of the lectures public in study materials, but please note that the quality may not be optimal and some lecture may not be recorded at all. - We will make available written transcripts of the lectures in study materials. - Every week there will be a seminar group which is not compulsory (except the first one), but you are encouraged to join. Note that the lecturers are *NOT* obliged to answer questions outside these seminar groups, so use this time wisely. - **You are also required to deliver a working project at the end of the semester (the assignment is already available in study materials).** Again, the lecturers will be happy to help you with the assignment, if they see that you are attending the seminars. ## CMake CMake is used as a building tool for the project on both Windows and Linux. You should get acquainted with the basics at [CMake Tutorial](https://cmake.org/cmake/help/latest/guide/tutorial/) and [Modern CMake Tutorial](https://cliutils.gitlab.io/modern-cmake/). You can use CMake from the command line directly or use your favorite IDE. In this course the Visual Studio Code is used. For the PV112 lessons the project is structured in the following way: - We use VCPKG for downloading and compiling required libraries. You can use the same method to include libraries of your own choice (assuming VCPKG supports it) for the final project. To link the library with the target executables, you need to update vcpkg.json file and use CMake command [target_link_libraries](https://cmake.org/cmake/help/latest/command/target_link_libraries.html). - To simplify the lectures, we created our own framework. You can find all its sources in the framework folder. - **Finally, you can find individual lectures under `courses/PV112`. Each produces one executable that you can run. You will have to add any new lectures provided to you during the course here. A new one will be published every week alongside a solution to the previous one.** The lectures are registered by `visitlab_add_subdirectory(FOLDER_NAME)` command in the course `CMakeLists.txt.` Note that the commands are already there, automatically ignoring missing folders. ## Visual Studio Code We will be using Visual Studio Code as a default IDE. Please use the following tutorial to set up your computer. [https://visitlab.pages.fi.muni.cz/tutorials/vs-code/index.html](https://visitlab.pages.fi.muni.cz/tutorials/vs-code/index.html). Make sure that you install all extensions required for PV112. ## Window Initialization - OpenGL SuperBible: Chapter 2. Our First OpenGL Program - OpenGL Programming Guide: Chapter 1. Introduction to OpenGL - Learn OpenGL: [Creating a Window](https://learnopengl.com/Getting-started/Creating-a-window), [Hello Window](https://learnopengl.com/Getting-started/Hello-Window) Before being able to work with OpenGL we must do 2 steps first: 1. Dynamically load OpenGL functions. This is done because OpenGL implementation is usually provided by separately installed driver and not built in OS. Since loading is OS-specific and tedious work, we will use [glad](https://github.com/Dav1dde/glad) library instead. 2. Create a window we can render to. As with 1. this is OS-specific therefore [glfw](https://www.glfw.org/) cross-platform library will be used in these lessons. !!! WARNING You may find that GLFW supports Apple systems. However, we limit ourselves to modern OpenGL 4.5 (Released in 2014) and up. Apple hasn't updated their implementation since 2011 and recently officialy deprecated OpenGL in its operating systems. This limitation cannot be bypassed using virtual machine. We advise you to install either Windows or Linux alongside/instead of macOS or use school computers. ### Task: Clear the Window - [glClearColor](http://docs.gl/gl4/glClearColor) - [glClear](http://docs.gl/gl4/glClear) Your first task is to clear the window using any color. This is done every frame so that user doesn't see overlapping results of computation. First choose your color using `glClearColor`. Then clear the screen with it using `glClear` function inside `render` function. ### Optional Exercise Change the color based on some kind of input or time (for example using sine function). - To retrive values from GUI use `red`, `green`, and `blue` variables. - To retrieve the current time use [glfwGetTime](https://www.glfw.org/docs/3.0/group__time.html). - Learn how to use input at [GLFW Input Guide](https://www.glfw.org/docs/latest/input_guide.html). ### Optional Homework If you want deeper understanding of inner workings of the libraries used, look at the documentation of APIs for your operating system and create a template reliant solely on them. For Windows that means including *windows.h* then calling *wglGetProcAddress* to load OpenGL functions and *CreateWindow* to (no surprise) create a window. For linux there are *glXGetProcAddress* and *Xlib* functions. Shader Compilation =============================================================================== OpenGL SuperBible: Chapter 2. Our First OpenGL Program
OpenGL Programming Guide: Chapter 2. Shader Fundamentals
Learn OpenGL: [Hello Triangle](https://learnopengl.com/Getting-started/Hello-Triangle)
Before being able to draw anything with OpenGL we need to compile shaders which instruct GPU how to draw. In this lesson there are two shaders, `main.vert` and `main.frag`, prepared for you to compile, link and use. Task: Compile Shaders ------------------------------------------------------------------------------- - [glCreateShader](http://docs.gl/gl4/glCreateShader) - [glShaderSource](http://docs.gl/gl4/glShaderSource) - [glCompileShader](http://docs.gl/gl4/glCompileShader) First compile both shaders. You can create your own function to avoid duplicity. Task: Create Program ------------------------------------------------------------------------------- - [glCreateProgram](http://docs.gl/gl4/glCreateProgram) - [glAttachShader](http://docs.gl/gl4/glAttachShader) - [glLinkProgram](http://docs.gl/gl4/glLinkProgram) - [glDetachShader](http://docs.gl/gl4/glDetachShader) - [glDeleteShader](http://docs.gl/gl4/glDeleteShader) After succesfully compiling both shaders, link them together to form a whole program to run on GPU. Task: Use the Program ------------------------------------------------------------------------------- - [glUseProgram](http://docs.gl/gl4/glUseProgram) Lastly, call `glUseProgram` in `render` function before the call to `glDrawArrays` to see the result of shaders. Ignore rest of calls for now, they will be explained in further lessons. Task: Define Viewport ------------------------------------------------------------------------------- - [glViewport](http://docs.gl/gl4/glViewport) Now try to change the size of the window, you will notice that the triangle is no longer rendered in the middle. This is becase when we created the window OpenGL sets the `viewport` such that it spans the whole window, however this is not updated when we resize the window. Viewport is defined by OpenGL as an area of a window where it should output the result of its computation. For our purposes it is desirable to set the origin of the area to starting coordinates of `[0; 0]` and window's `width` and `height` effectively covering entire window. Therefore, set the viewport at the beginning of `render` function. Use the `glViewport` function for this task. Task: Create New Shaders ------------------------------------------------------------------------------- Create a new set of shaders and a new program that will render a geometry in different color. Use the `glViewport` function from previous task to render two triangles. On the left side render the orginal triangle using the first program and on the right side using the second program. Task: Switch Shaders via GUI ------------------------------------------------------------------------------- Switch the order of used programs for drawing based on the GUI state. You can use `ui_reverse_sides` variable that stores the current GUI state. Task: Switch Shaders via Keyboard ------------------------------------------------------------------------------- Switch the order of used programs for drawing on pressing keyboard key of your choice. Ensure that the GUI is updated as well (this should be quite easy ;)). Drawing, Buffers & Vertex Attributes =============================================================================== OpenGL SuperBible: - Chapter 2. Our First OpenGL Program - Chapter 5. Data - Buffers - Chapter 7. Vertex Processing and Drawing Commands, Chapter 7. Vertex Processing and Drawing Commands OpenGL Programming Guide: - Chapter 3. Drawing with OpenGL - Chapter 3. Drawing with OpenGL - Data in OpenGL Buffers, Vertex Specification Learn OpenGL: [Hello Triangle](https://learnopengl.com/Getting-started/Hello-Triangle) In order to draw OpenGL needs 3 things: 1. Bound program describing how to draw supplied data. 2. Vertex Array Object describing how to fetch data from buffers on GPU. 3. Draw command describing which part of data should be taken in what order. The first one was covered in the lesson before. Now we will focus on the Last one. OpenGL has many draw commands (as of OpenGL 4.5 there are 25) but we will focus on the simplest one right now, `glDrawArrays`. `Arrays` set of draw functions takes data in consecutive order and assigns them number `gl_VertexID` which you can use in shader. In our first shader we supply no external data and instead fetch them from the constant array based on the `gl_VertexID` number. You can see this in illustration below. You can offset the number using `first` parameter. You can also try supplying different primitives besides triangles, however, we will stay with triangles in our lessons as this is the native geometry supported by GPUs nowadays and most widely used. ******************************************************************************* * * * * * ^ Y+ * * | * * | * * gl_VertexID = 2 | * * [0.0; 0.5] o * * /|\ * * / | \ * * / | \ X+ * * ----------+---+---+----------> * * / | \ * * / | \ * * o------+------o * * [-0.5; -0.5] | [0.5; -0.5] * * gl_VertexID = 0 | gl_VertexID = 1 * * | * * | * ******************************************************************************* Task: Draw a Square ------------------------------------------------------------------------------- - [glDrawArrays](http://docs.gl/gl4/glDrawArrays) Modify main.vert such that there are 6 vertices ******************************* for 2 triangles forming square. Modify also * 2=5 4 * `glDrawArrays` to draw both of them. * o o-----o * (Alternatively use two calls!) * |\ \ | * * | \ \ | * * | \ \ | * * | \ \ | * * | \ \| * * o-----o o * * 0 1=3 * ******************************* Task: Draw a Square Using Triangle Strip ------------------------------------------------------------------------------- - [Triangle Strip](https://en.wikipedia.org/wiki/Triangle_strip) Modify main.vert such that there are 4 vertices ******************************* for 2 triangles forming square. Use * 2 3 * GL_TRIANGLE_STRIP mode. * o-----o * * |\ | * * | \ | * * | \ | * * | \ | * * | \| * * o-----o * * 0 1 * ******************************* Task: Copy Vertex Data to GPU ------------------------------------------------------------------------------- - [glCreateBuffers](http://docs.gl/gl4/glCreateBuffers) - [glNamedBufferStorage](http://docs.gl/gl4/glBufferStorage) - ([glNamedBufferData](http://docs.gl/gl4/glBufferData)) - [glDeleteBuffers](https://docs.gl/gl4/glDeleteBuffers) Acquaint yourself with the data for this lesson in `data.hpp` file. For every buffer that is located there allocate one corresponding buffer on GPU with the same size and copy the data from CPU to GPU. Remember to deallocate them in the destructor of the `Application`. You can create data for each object in their respective tasks as needed or create them all upfront. That is up to you. !!! note You can use sizeof() operator in C++ to retrieve the size. Recall that calling sizeof on array gives you the size of entire array only if its static(which it is in this case) but gives you the size of pointer in case of dynamic array. Cleaner and more general solution is to use the size of array and sizeof type it contains, for example: square_vertices_length * sizeof(uint32_t). With the data in GPU continue with tasks 3.4-3.7. All of them are exercises for creation of Vertex Array Objects. At the end of each one, draw the object as described in previous lessons. You can skip to Task 3.8 after the first one if you want to see them colored and then go back to Task 3.5. !!! note Feel free to create a new vertex shader file for each of the 3.4-3.7 tasks. However, if you want a challenge, use a single vertex shader for all of the tasks. There is a way. Task: Create Vertex Array Object #1 (Diamond) ------------------------------------------------------------------------------- - [glCreateVertexArrays](http://docs.gl/gl4/glCreateVertexArrays) - [glVertexArrayVertexBuffer](http://docs.gl/gl4/glBindVertexBuffer) - [glEnableVertexArrayAttrib](http://docs.gl/gl4/glEnableVertexAttribArray) - [glVertexArrayAttribFormat](http://docs.gl/gl4/glVertexAttribFormat) - [glVertexArrayAttribBinding](http://docs.gl/gl4/glVertexAttribBinding) - [glBindVertexArray](https://docs.gl/gl4/glBindVertexArray) - [glDeleteVertexArrays](https://docs.gl/gl4/glDeleteVertexArrays) In this lesson you will exercise sending data to GPU and giving GPU their description so that it knows how to fetch them and interpret as input to shaders. Each input/output is described with the following syntax in the shader: ```GLSL layout(location = attribute_index) in/out type name; ``` where `attribute index` is a unique number chosen by you, `in/out` keyword says whether it's input or output (we will learn about outputs later in the lesson) and the `name` is a unique identifier chosen by you as well. The `number` serves as an identification for CPU and driver side. The `name` is used only in main function of respective shader where it is declared. We will have 2 inputs in this lesson, position and color, at locations 0 and 1 respectively. ```GLSL layout(location = 0) in vec3 position; layout(location = 1) in vec3 color; ``` We will pass the position inside the shader to the `gl_Position` built-in variable and ignore the color for now. ![VAO scheme](./images/03_VAO.jpg) The first object has both attributes saved in separate buffers. Each buffer corresponds to one binding and each of those bindings correspond to one location. This is the most trivial and intuitive example. However, it doesn't show why there is a binding point. 1. Create VAO for this object 2. For each buffer associate it with binding number 3. For each attribute 1. Enable it 2. Describe its format 3. Associate it with binding ******************************************************************************* * +---+---+---+---+---+---+---+ * * Buffer 0: | X | Y | Z | X | Y | Z |...| * * +---+---+---+---+---+---+---+ * * * * +---+---+---+---+---+---+---+ * * Buffer 1: | R | G | B | R | G | B |...| * * +---+---+---+---+---+---+---+ * ******************************************************************************* ******************************************************************************* *+----------+ +-----------+ +----------------------+ * *| Buffer 0 +------------>| Binding 0 +-+----------->| Location 0: Position | * *+----------+ +-----------+ +----------------------+ * * * *+----------+ +-----------+ +----------------------+ * *| Buffer 1 +------------>| Binding 1 +-+----------->| Location 1: Color | * *+----------+ +-----------+ +----------------------+ * ******************************************************************************* Task: Create Vertex Array Object #2 (Square) ------------------------------------------------------------------------------- The second object has both attributes saved in one buffer **deinterleaved**. That means you will have to provide offset into the buffer for the second binding. From bindings further the description remains the same. ******************************************************************************* * +---+---+---+---+---+---+---+---+---+---+---+---+---+---+ * * Buffer 0: | X | Y | Z | X | Y | Z |...| R | G | B | R | G | B |...| * * +---+---+---+---+---+---+---+---+---+---+---+---+---+---+ * ******************************************************************************* ******************************************************************************* *+----------+ +-----------+ +----------------------+ * *| Buffer 0 +-+---------->| Binding 0 +------------->| Location 0: Position | * *+----------+ | +-----------+ +----------------------+ * * | * * | +-----------+ +----------------------+ * * +---------->| Binding 1 +------------->| Location 1: Color | * * +-----------+ +----------------------+ * ******************************************************************************* Task: Create Vertex Array Object #3 (Triangle) ------------------------------------------------------------------------------- The third object has both attributes in one buffer **interleaved**. Plus the color is saved as 8-bit unsigned byte. Take care of the fact that we want color to be normalized to range [0.0, 1.0] before being passed to shader. We defined the interleaved attributes as an array of structs for easier consumption. ```C struct Vertex { float position[3]; uint8_t color[3]; } ``` !!! note You can use offsetof(struct, member) macro to retrieve the (relative) offset of an attribute. ******************************************************************************* * +---+---+---+---+---+---+---+---+---+---+ * * | Vertex 0 | Vertex 1 ...| * * Buffer 0: +---+---+---+---+---+---+---+---+---+---+ * * | X | Y | Z | R | G | B | X | Y | Z |...| * * +---+---+---+---+---+---+---+---+---+---+ * ******************************************************************************* ******************************************************************************* *+----------+ +-----------+ +----------------------+ * *| Buffer 0 +------------>| Binding 0 +--+---------->| Location 0: Position | * *+----------+ +-----------+ | +----------------------+ * * | * * | +----------------------+ * * +---------->| Location 1: Color | * * +----------------------+ * ******************************************************************************* Task: Create Vertex Array Object #4 (Indexed Diamond) ------------------------------------------------------------------------------- - [glVertexArrayElementBuffer](http://docs.gl/gl4/glVertexArrayElementBuffer) - [glDrawElements](http://docs.gl/gl4/glDrawElements) This object/task is exactly the same as 3.2, except now the data are indexed. There is one additional buffer for indices you have to associate with VAO. In order to draw this object you have to use `glDrawElements` instead of `glDrawArrays`. Task: Interpolate Color ------------------------------------------------------------------------------- In order to see the color it is neccessary to pass it from vertex to fragment shader. This can be done by adding output attribute to vertex shader that we will write input color to. **Add output to your vertex shader**: ``` layout(location = 0) out vec3 vs_color; ``` and write the input to it. Notice the different name and same location as input. The name has to be different so that it doesn't collide with the input. However, the number can be same as there is no place where it would create ambiguity. The output location serves only for matching between vertex and fragment. **Add the output vertex color to fragment shader as input**. Notice that name can be different in fragment shader but **location and type must be the same to match**. ```GLSL layout(location = 0) in vec3 fs_color; ``` See the illustration of the flow below. ************************************************************************************* * * * Vertex Shader * * +---------------------------------+ +-------------------------------------+ * *-->| (location = 1) in vec3 color +-->| (location = 0) out vec3 vs_color | * * +---------------------------------+ +--------------------+----------------+ * * | * * Fragment Shader +-------------------------------------+ * * v * * +---------------------------------+ +-------------------------------------+ * *-->| (location = 0) in vec3 fs_color +-->| (location = 0) out vec4 final_color +-->* * +---------------------------------+ +-------------------------------------+ * * * ************************************************************************************* At the end the result should look like this: ![Solution to 04](./images/04_final.png) Task: Look at the result in RenderDoc ------------------------------------------------------------------------------- - [RenderDoc Getting Started](https://renderdoc.org/docs/getting_started/quick_start.html) Run your finished application in RenderDoc and capture a single frame. Spent some time tinkering with it and see if you can collect all the information you gave OpenGL through glVertexArray* commands. RenderDoc is widely used in the industry and it is therefore beneficial to have a good handle of it. ![Solution 03 in RenderDoc](./images/renderdoc.png) Optional Homework ------------------------------------------------------------------------------- You can analyze frame draw of your favorite game. An example of such analysis can been seen at [Deus Ex: Human Revolution - Graphics Study](http://www.adriancourreges.com/blog/2015/03/10/deus-ex-human-revolution-graphics-study/). Fixed Function Pipeline =============================================================================== - OpenGL SuperBible: Chapter 9. Fragment Processing and the Framebuffer - OpenGL Programming Guide: Chapter 8. Color, Pixels, and Framebuffers !!! note At the beginning of OpenGL shaders didn't exist and drawing was made using draw commands that were executed immediately. For example: ``` glBegin() glVertex(...) glVertex(...) glVertex(...) glEnd() ``` would draw a triangle. These set of functions are called fixed functions. However, do not confuse it with what you are about to learn in this lesson which deals with *configurable* *non-programmable* parts of GPU's pipeline. So far we successfully pushed data to GPU and rendered them through programmable shaders. In this lesson we take look at the parts of the pipeline which are only configurable through state changing functions. Task: Polygon rasterisation Mode ------------------------------------------------------------------------------- - Functions: [glPolygonMode](http://docs.gl/gl4/glPolygonMode) GPU's rasterizer does not necessarily need to fill the specified geometry. You can let GPU render a geometry as set of lines or only as points. Let the user of your application change these modes by pressing buttons on the keyboard. Pressing F will result in change of rendering mode to FILL and pressing L will change the rendering mode to lines. ![Line rendering](./images/05_lines.png) Task: Depth ------------------------------------------------------------------------------- - [glEnable](http://docs.gl/gl4/glEnable) - Learn OpenGL: [Depth testing](https://learnopengl.com/Advanced-OpenGL/Depth-testing) The color buffer is not the only one OpenGL uses during the rendering. You should know by now from the lectures about the depth buffer. Enable the use of depth buffer and clear it along with the color buffer before rendering. ![Diamond in the upper right corner is properly rendered with the depth buffer](./images/05_depth.png) Task: Multisampling ------------------------------------------------------------------------------- - Learn OpenGL: [Anti Aliasing](https://learnopengl.com/Advanced-OpenGL/Anti-Aliasing) You may have noticed the jagged edges on rendered triangles. From the course PB009 you may remember that this artifact is called aliasing. Enable the simplest anti-aliasing method, multisampling, by creating multisampled main framebuffer in `main.cpp` using `manager.set_multisampling_per_pixel(count)` function. Note that this custom method internally changes the amount of samples by calling `glfwWindowHint(GLFW_SAMPLES, samples_per_pixel);` when the windows is created. Use `glEnable` again, now to allow multisampling, in `Application.cpp`. !!! WARNING If your code seems to be fine but you see no visual difference between different sample counts, your graphics driver might be a reason as it may do automatically some anti aliasing. To fix this behaviour in case of Nvidia GPU, open the NVIDIA Control Panel, go to "Manage 3D Settings" and verify that your "Antialiasing - Mode" is set to "Application-controlled". ![Left: without multisampling, Right: with 4x multisampling](./images/05_multisampling.png) Task: Color Blending ------------------------------------------------------------------------------- - Functions: [glBlendFunc](http://docs.gl/gl4/glBlendFunc), [glBlendEquation](http://docs.gl/gl4/glBlendEquation) - Learn OpenGL: [Blending](https://learnopengl.com/Advanced-OpenGL/Blending) !!! WARNING Disable depth testing before this exercise. Depth + blending do not play well. When OpenGL computes a fragment it has to decide what to do with the previous one at the same position. This process is called blending and can be configured using functions `glBlendFunc` and `glBlendEquation`. The blending equation has a form: $ C_{result} = F_{source} * C_{source} \otimes F_{destination} * C_{destination} $ where source is the incoming fragment recently computed and destination the previous fragment, C is a color of a fragment and $\otimes$ the equation. Typically, blending is used for transparency in a following setup: ``` glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA); glBlendEquation(GL_FUNC_ADD); ``` $ C_{result} = C_{source}^\alpha * C_{source} + (1.0 - C_{source}^\alpha) * C_{destination} $ Enable blending and try out different equations. ![Transparency using blending function](./images/05_blending_transparent.png) Task: Face Culling ------------------------------------------------------------------------------- - Functions: [glCullFace](http://docs.gl/gl4/glCullFace) - Learn OpenGL: [Face Culling](https://learnopengl.com/Advanced-OpenGL/Face-culling) Normally, triangles are drawn from both sides. To avoid a costly rasterisation that may not be visible a process called culling is applied. Each triangle is classified as either front-facing or back-facing based on the order of its points. The culling may be configured in OpenGL to ignore either back or front faces. Typically, triangles given in counter clockwise order are classified as front faces and back faces are culled. Enable culling and set back faces to be culled. ![Culled diamond in the left corner.](./images/05_culling.png) Uniforms =============================================================================== - [glProgramUniform*](http://docs.gl/gl4/glProgramUniform) In order to keep things simple we will use only the first arrangement of data from task 4.2 for remaining lessons. We also provide simple Geometry class inside the framework which provides ability to load objects from `.obj` files using [tinyobjloader](https://github.com/syoyo/tinyobjloader) library. The class also has a few specialized inherited classes for some basic objects such as cube, sphere, and teapot. ``` class Geometry { ... void draw(...); static Geometry from_file(std::filesystem::path file_path); private: GLuint positions_vbo; ... GLuint vao; } class Sphere : public Geometry {} ``` In previous lessons you learned how to provide geometry data for a shader program, a case where each vertex receives unique value. In this lesson, you will provide uniform data and use them for projection, lighting and materials. Geometry data are already prepared for you so you will find in the shader: ``` layout(...) in vec3 position; layout(...) in vec3 normal; layout(...) in vec2 texture_coordinate; ``` All of them are passed through to a fragment shader. Normal vector is multiplied by normal matrix as described in the lecture. Task: Apply Model + View + Projection Transformations ------------------------------------------------------------------------------- - OpenGL SuperBible: - OpenGL Programming Guide: Chapter 5. Viewing Transformations - Learn OpenGL: [Coordinate Systems](https://learnopengl.com/Getting-started/Coordinate-Systems) - [GLM Manual](https://github.com/g-truc/glm/blob/0.9.9.2/doc/manual.pdf) Add MVP matrices as uniforms to main.vert. Create MVP matrices on CPU side and upload them to the shader program. Use different model matrix for each object. ![Scene after using MVP matrices.](./images/uniforms_mvp_final.png) Task: Apply lighting ------------------------------------------------------------------------------- - [Phong Reflection Model](https://en.wikipedia.org/wiki/Phong_reflection_model) - [Blinn-Phong Shading Model](https://en.wikipedia.org/wiki/Blinn–Phong_shading_model) - Learn OpenGL: [Lighting](https://learnopengl.com/Lighting/Basic-Lighting), [Materials](https://learnopengl.com/Lighting/Materials) Add light and material properties as uniforms to main.vert and main.frag. Differentiate between directional and point light based on its .w value in position. Map the change between directional and point light to a key on a keyboard. Calculate Blinn-Phong shading. Use any attenuation for point lights, simple [inverse square law](https://en.wikipedia.org/wiki/Inverse-square_law) is enough. ![Scene with added directional light.](./images/uniforms_light_final.png) Textures =============================================================================== Task: Load and Create Texture ------------------------------------------------------------------------------- - [glCreateTextures](http://docs.gl/gl4/glCreateTextures) - [glDeleteTextures](http://docs.gl/gl4/glDeleteTextures) - [glTextureStorage2D](http://docs.gl/gl4/glTexStorage2D) - [glTextureSubImage2D](http://docs.gl/gl4/glTexSubImage2D) - [glGenerateTextureMipmap](http://docs.gl/gl4/glGenerateMipmap) - [glTextureParameteri](http://docs.gl/gl4/glTexParameter) To load pixel data from a file you can use function from [stbi](https://github.com/nothings/stb/blob/master/stb_image.h) library in a following manner (given filename is a `std::filesystem::path`): ```C int width, height, channels; unsigned char *data = stbi_load(filename.generic_string().data(), &width, &height, &channels, 4); ``` The last option `4` will force the library to always load data in RGBA8 format. `width` and `height` variables will contain size of the loaded image, you will use those in the `glTextureStorage2D` and `glTextureSubImage2D`. With the pixel data loaded from the disk start by initializing the texture using `glCreateTextures` and `glTextureStorage2D`. ```C GLuint wood_texture; glCreateTextures(GL_TEXTURE_2D, 1, &wood_texture); glTextureStorage2D(wood_texture, std::log2(width), // NUMBER OF MIMPMAP LEVELS GL_RGBA8, // SIZED INTERNAL FORMAT width, height) ``` `glTextureStorage2D` allocates a memory needed to store a texture. For that we need to specify its resolution (`width` and `height`) and size of each pixel specified as number of channels and their type. Look at the full table of all possible pixel storage at http://docs.gl/gl4/glTexStorage2D. The `sized internal format` has the following syntax: `GL_[components][size][type]`. For our use-case the `GL_RGBA8` will suffice, which correspond to 4-channel with the pixel data stored as unsigned normalized integers(without the suffix). We want the maximum number of [mipmap](https://en.wikipedia.org/wiki/Mipmap) levels for proper interpolation. An easy way to calculate the maximum number of mipmap levels is to take the log2 of the width of the image. To upload the data from the CPU to the GPU use the `glTextureSubImage2D`. `format` and `type` parameters now correspond to the data on the CPU side, which are (because of stbi library) `GL_RGBA` and `GL_UNSIGNED_BYTE`. `level` will be *0* as we want to fill the base mip. Finally, we can call `glGenerateTextureMipmap()` to let the driver calculate mipmap levels from the base mip. You can adjust several attributes by calling the `glTextureParameter(i/f/iv/fv/...)`. Some of the most useful are: - `GL_TEXTURE_MIN_FILTER` `GL_TEXTURE_MAG_FILTER` which can set the interpolation method. Typically `GL_LINEAR_MIPMAP_LINEAR` and `GL_LINEAR` respectively for the best quality. - Wrapping `GL_TEXTURE_WRAP_S/T/R` which set the wrapping behaviour in x, y, z. Task: Apply Texture ------------------------------------------------------------------------------- - [glBindTextureUnit](http://docs.gl/gl4/glBindTextureUnit) - [GLSL texture()](https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/texture.xhtml) Add a texture binding to the fragment shader and use `texture_coordinates` to sample from it using function `texture(sampler, tex_coord)`. Make the fetched sample a final output color. ``` layout(binding = 0) uniform sampler2D diffuse_texture; void main() { vec4 texture_color = texture(diffuse_texture, texture_coordinates); } ``` !!! WARNING Note that samplers use **bindings** and NOT locations. Bind the created textures to a sampler binding point you created in the fragment shader using `glBindTextureUnit`. Task: Turn Material Properties to Textures ------------------------------------------------------------------------------- Add a diffuse texture to code from previous lessson. Multiply the diffuse texture by diffuse value to create textured lighted object. ![Objects with diffuse material multiplied by texture value.](./images/textures.png) Framebuffers =============================================================================== - OpenGL SuperBible: Chapter 9. Off-Screen Rendering - OpenGL Programming Guide: Chapter 4. Framebuffers - Learn OpenGL: [Framebuffers](https://learnopengl.com/Advanced-OpenGL/Framebuffers) ![Framebuffers scheme](./images/07_Framebuffers.jpg) Task: Create a framebuffer ------------------------------------------------------------------------------- - [glCreateFramebuffers](http://docs.gl/gl4/glCreateFramebuffers) - [glDeleteFramebuffers](http://docs.gl/gl4/glDeleteFramebuffers) - [glNamedFramebufferDrawBuffers](http://docs.gl/gl4/glDrawBuffers) - [glNamedFramebufferTexture](https://docs.gl/gl4/glFramebufferTexture) Create your own framebuffer along with 2 textures: one for the render output and one for the depth writing. Bind both textures to the framebuffer. Task: Pass the output of main shader through the post-process shader ------------------------------------------------------------------------------- - [glBindFramebuffer](https://docs.gl/gl4/glBindFramebuffer) - [glClearNamedFramebufferfv](https://docs.gl/gl4/glClearBuffer) A shader that simply renders a texture over the entire screen is already prepared for you. Give the shader your texture as an input and call it on the default framebuffer. Task: Apply Post-Process Functions ------------------------------------------------------------------------------- - [Conversion to Grayscale](https://en.wikipedia.org/wiki/Grayscale#Luma_coding_big_gameas) - https://en.wikipedia.org/wiki/Kernel_(image_processing) - [texelFetch](https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/texelFetch.xhtml) Now you can apply any post-process effects in the `postprocess` shader to your liking. Try making the screen grayscale as a start. Then try to apply different kernel functions such as blur, sharpen, etc.. ![Scene with edge detection applied.](./images/postprocess.png) Interface Layouts, UBO, SSBO =============================================================================== - OpenGL Programming Guide: Chapter 2. Interface Blocks - [Interface Block (GLSL) - OpenGL Wiki](https://www.khronos.org/opengl/wiki/Interface_Block_%28GLSL%29) Task: Bind UBO buffers ------------------------------------------------------------------------------- - [glBindBufferBase](http://docs.gl/gl4/glBindBufferBase) Uniform inputs can be logically grouped into UBOs in the shaders. For example, all material properties can be grouped into a single struct: ``` layout(binding = 0) uniform Material { vec4 ambient; vec4 diffuse; ... } material; ``` These UBOs can be mirrored on the C++ side: ``` struct Material { glm::vec4 ambient; glm::vec4 diffuse; ... } ``` - Create buffers for Camera, Light and Object. (glCreateBuffer, [glNamedBufferStorage](http://docs.gl/gl4/glBufferStorage) with GL_DYNAMIC_STORAGE_BIT) - Bind them all using glBindBufferBase (as GL_UNIFORM_BUFFER) - Draw any object with them Task: Shader Storage Buffer Objects (SSBO) - Many Lights ------------------------------------------------------------------------------- Replace the single light in shader with SSBO containing dynamic array: ``` struct Light { vec4 position; vec4 ambient_color; vec4 diffuse_color; vec4 specular_color; }; layout(binding = 1, std430) buffer Lights { Light lights[]; }; ``` You can get the length of the array by calling `.length()` method on it. - Bind the lightS buffer before drawing - Iterate over every light in the shader and accumulate the lighting Task: Instanced lights ------------------------------------------------------------------------------- - [glDrawElementsInstanced](http://docs.gl/gl4/glDrawElementsInstanced) Use the `draw_light_program` to visualize where the lights are. - Bind the lightS buffer as before - Call `glDrawElementsInstanced` with the size of the vector and arbitrary object. Use members and methods of the Mesh class to bind and fill the parameters of the draw call. (mesh.bind_vao(), mesh.mode, mesh.draw_elements_count) Task: Instanced objects ------------------------------------------------------------------------------- Now draw many objects using the instanced draw call. Modify the vertex shader to retrieve the object from SSBO using its instance ID. ``` struct Object { mat4 model_matrix; vec4 ambient_color; vec4 diffuse_color; vec4 specular_color; }; layout(binding = 2, std430) buffer Objects { Object objects[]; }; ... Object object = objects[gl_InstanceID]; ``` To get the object in the fragment shader as well, you will need to pass the gl_InstanceID as an attribute because the variable is not available in the fragment shader. Since the attribute needs to stay the same for every vertex without any interpolation, the `flat` modifier must be used. ``` // vertex layout(location = 2) out flat int fs_instance_id; ... fs_instance_id = gl_InstanceID; // fragment layout(location = 2) in flat int fs_instance_id; ``` Final Project: FAQ, Tips, Tricks, Hints =============================================================================== Resources for 3D Models ------------------------------------------------------------------------------- Here, we will provide you with a list of websites you can download free 3D models. We will continually expand this list. If you have found anything by yourself, feel free to message it in response to this thread. [Kenney • Home](https://kenney.nl/) [Render Gallery • Poly Haven](https://polyhaven.com/gallery) [CGTrader - 3D Models for VR / AR and CG projects](https://www.cgtrader.com/) [3D Models for Professionals :: TurboSquid](https://www.turbosquid.com/) [Free Public Domain 3D Models | 3DModelsCC0](https://www.3dmodelscc0.com/) Including Libraries ------------------------------------------------------------------------------- Adding libraries to a C++ project is one of the most difficult things. We show 3 main options on how to include most libraries in CMake. - Vcpkg - The easiest way is to use Vcpkg package manager. Including the library consists of 3 steps: - Find your library at vcpkg packages - Add it to the list in vcpkg.json - Modify your CMakeLists in the following way: - Let CMake find it through find_package command - Use target_link_libraries to add it to your target(executable) - [assimp example](https://gist.github.com/MatusT/158e286fed6fa2a70b97a5c364402945) - Compiled library - In the case, the library is **not open source**, you may have to resort to including it as a shared library in binary form(.dll for windows, .so for linux). Usually, you would have to cover case for each platform you want to support, however, we will be satisfied if you only support one platform (it will be much easier for you). - The process consists of explicitly adding library files and header files to CMake target. In addition, the dynamic files will have to be copied over next to the target executable. - We provide an example for irrKlang library that shows every step. This example will only work on Windows: [irrKlang example](https://gist.github.com/MatusT/eec0c6f342ddd8151acd1c0c21b121e5) - From source code - This is the most difficult path. Depending on the library (and its CMake if there is any), you may need to use a different method. However, if the desired library has properly written and recent version of CMakeLists.txt you may use command **FetchContent**. - The biggest problem is that the library may have dependencies that are not resolved (unlike in the case of vcpkg) and it is up to you to include every single dependency and their dependencies etc... - See [CMake Tutorial](https://cliutils.gitlab.io/modern-cmake/chapters/projects/fetch.html) and [CMake documentation](https://cmake.org/cmake/help/latest/module/FetchContent.html#id5) for quick examples how to use FetchContent. Debugging OpenGL ------------------------------------------------------------------------------- Debugging graphics applications is its own topic. There are two primary areas to be concerned about: - Shader compilation and linking - Calling correct OpenGL commands on the CPU side (binding, drawing etc...) ### Shader Compilation and Linking The best and easiest way to know if there are any compilation errors in your shaders is through external compilator, best used as part of your IDE. The official compilator for GLSL shaders, called *glslangvalidator*, can be obtained by installing [Vulkan SDK](https://vulkan.lunarg.com/sdk/home#windows). Both, VS Code and Visual Studio, have extensions that can use this compilator and underline any errors in your shaders directly in the IDE. However, even if shaders are correctly compiled, it does not mean that they link correctly. To check for exact errors during the linking process you can use the procedure described [here](https://www.khronos.org/opengl/wiki/Shader_Compilation#Error_handling). **DRIVER OPTIMIZATION WARNING**: OpenGL drivers tend to remove any global inputs that they do not see used in the shader. Typically, those errors can be ignored. Do not be surprised when you bind a buffer to a binding point from which the shader never reads. ### Calling correct OpenGL commands on the CPU side #### Callback The simplest method to debug OpenGL calls was shown to you at lectures and during exercises. Feel free to make a breakpoint inside the callback function to stop your IDE whenever any error happens. #### GlGetError() It might happen, due to asynchronous nature of some OpenGL commands, that callback is called later from where the error occurs. For these occasions it is possible to simply call glGetError after any command you suspect of being wrong (or all of them). glGetError forces synchronization up to the point it is called. See more at [khronos.org](https://www.khronos.org/opengl/wiki/OpenGL_Error#Testing_for_errors) #### RenderDoc It is best to view your program holistically in a debug program such as RenderDoc. The program can be overwhelming with complexity; therefore, we will provide here some simple instructions on how to answer some questions about the correctness of your program. - Did I really call all the draw commands? - Look at the list of commands in the **Event Browser** panel and see for yourselves. - Are my buffers/uniforms/samplers bound to correct locations/bindings - Select the draw call of interest 'Event Browser' panel and select tab 'Pipeline State'. In there, you can click through passes of the entire rendering pipeline. When you click on Vertex/Fragment shader you can see all the relevant bindings. - Do my buffers contain the data I expect there to be during the studied draw call? - Select any buffer in any part of the program. Directly in **Resource Inspector** panel or indirectly in **Pipeline State** tab. Click on the **view contents** and contents of the buffer will be displayed in binary form. OpenGL does not know what format of your data you expect, but you can provide this information in GLSL format. For example, typing vec4 positions' in 'Buffer Format' and clicking 'Apply' will reformat the binary data into columnar view of vectors of four elements. - Are the data properly transformed when being passed from vertex to fragment shader? - In the 'Mesh Viewer' tab it is possible to see RenderDoc's interpretation of input data as a 3D model. - You can also see the same data after transformation in the tab 'VS Out'. - Do fragment shaders write to my framebuffers what I expect them to write? - When a draw call is selected, the 'Texture Viewer' tab shows all textures that are inputs and outputs. #### Getting better errors through naming things (mostly in external programs such as RenderDoc) You might notice that in applications such as RenderDoc, as well as in the debug messages you see objects referred to by only their numbers. It is naturally quite difficult to discern what 'invalid enum in program 54796' means. For this purpose, there is an additional set of OpenGL commands. The first group of commands can create so-called 'debug groups'. Debug groups will organize all your OpenGL commands into separate sets that you can look at. For example, you can create a post process debug group. The second group of commands can give arbitrary names to every OpenGL object. So, your 'Buffer 67' can become 'Bunny Vertex Positions' buffer. [https://www.khronos.org/opengl/wiki/Debug_Output#Scoping_messages](https://www.khronos.org/opengl/wiki/Debug_Output#Scoping_messages) [https://www.khronos.org/opengl/wiki/Debug_Output#Object_names](https://www.khronos.org/opengl/wiki/Debug_Output#Object_names)