Browsed by
Tag: raytracing

A Graphics Programmer’s Thoughts on RTX

A Graphics Programmer’s Thoughts on RTX

As a programmer and computer graphics enthusiast it is always an exciting time when a new line of graphics cards is announced. This is certainly true about Nvidia’s launch of their new line of RTX branded cards. Much has been written about the launch from the perspective of gamers, and frankly if you just look at today’s games the picture doesn’t look so good. But in this post I’m going to look a bit more at the future and the promise that the new RTX architecture holds for games yet to come.

First we need to talk about the price. These cards are expensive. I know some people try to shrug it off as Nvidia re-aligning their naming convention or some claptrap like that. Sure a 2070 has similar performance to a 1080 for about the same price but in every previous new generation the new cards provided better performance at the same price point. So what happened? Well, it looks like a perfect storm of events. One obvious issue is that AMD is still not remotely competitive with Nvidia at the high end, meaning these cards are mostly competing against the 10xx generation GTX cards. A lack of competition is never good for pricing (at least not from the eyes of the consumer, I’m sure Nvidia is laughing all the way to the bank). The other issue is that these new chips are huge. All those extra features for realtime raytracing and the tensor cores for machine learning all come at a hefty price in terms of silicon real estate. They offer exciting possibilities for the future, and I will talk about that more later, but right now all of those transistors are doing very little to speed up current games. Given the massive die sizes on these chips it leads me to believe that Nvidia was counting on a process shrink to 7nm when they planned the RTX chips initially. Then, when it became obvious that the 7nm node wouldn’t be ready in time for RTX they decided to go ahead with the raytracing and tensor core features anyway rather than wait and push them out another generation. Thus we are left with huge (754mm² for the 2080Ti), and therefore expensive, chips. Of course all of that is speculation on my part, but it would explain a lot about the pricing of the RTX cards.

Now let’s talk about what makes RTX exciting: all the new features. I’m actually going to start with the tensor cores. I’m somewhat surprised that they left these in the consumer cards actually, rather than keeping them for the Titan line and up. I don’t actually think these will do much for gaming graphics directly at all. I know Nvidia has made a big deal out of their deep learning antialiasing solution but frankly it looks a bit gimmicky to me and the fact that you need to train it specifically for every game is not great. However, the tensor cores are fantastic news for anyone playing around with deep learning. In fact the 2080Ti provides almost identical deep learning performance to the Titan V at half the price. This should go a long way to making machine learning accessible to many more people, and that will further accelerate progress in what is already a fast-moving field. Fantastic news for students and hobbyists that want to experiment with deep learning, but also perhaps a bit of wasted silicon for what is still the biggest market for these cards: gamers. Only time will tell if game developers find interesting uses for the tensor cores in their games, but my hunch is that they won’t, at least not before the current cards are long obsolete.

Now on to the real meat of RTX: raytracing. This is where this new generation of chips promises a true revolution in computer graphics. It really is exciting, and I say this as someone who is actually rather skeptical as to the value of raytracing. It is often billed as a magic bullet solution to producing realistic computer graphics, and it really isn’t (something I talk about in my previous post comparing raytracing and pathtracing here). I’m actually really glad that the realtime raytracing solutions (DXR, RTX) focus on a hybrid approach rather than trying to push pure raytracing. Why? Because it would require radical changes in the tooling and development process for making games, as well as sidelining a lot of silicon (which would be needed to be retained to support all the non-raytracing games anyway) and years of R&D that have pushed the limits on rasterization.

So what can we expect from raytracing? Going by the likes of Battlefield 5 it looks like it will be mostly fancy reflections to begin with, which makes sense since that’s something that raytracing excels at (and something that requires a ton of nasty hacks to simulate in a rasterization-based engine). This also mirrors the way that shaders were first used when they became a standard feature: we initially saw a wave of games that added a single effect to an otherwise fixed-function pipeline (such as the shiny water effect in Morrowind). So that’s the majority of what we can expect from raytracing in games for the next few years: nicer reflections and fancier shadows, since those are relatively simple to execute. Far more interesting I think is the possibility of far improved realtime global illumination (something that the new Metro game promises for RTX owners), as well as other creative uses for the raytracing capability, because casting rays into a scene has all sorts of interesting applications. Consider for example parallel occlusion mapping. This technique creates a very good 3D offset effect using a 2D texture and ray-casting. This is quite expensive to do using current shader hardware, but might be possible to accelerate vastly with raytracing capabilities of RTX. Another possibility is to use the raytracing hardware to render voxel scenes without the need for rasterization at all. Again this is possible using current shader technology, but it is taxing (see for example the GigaVoxels rendering library). An RTX-optimised version of this might be significantly faster. Outside of games there is also a possibility to accelerate GPU-based offline raytracers/pathtracers such as OTOY Octane Render, or the GPU mode of Cycles, making these much faster in the future.

So where does that leave us? Well, I think for now we can mostly expect gimmicks like nicer reflections and/or shadows (at significant framerate cost), and I think right now the RTX cards are mostly of interest to graphics developers looking to experiment with the next generation of graphics techniques. Adoption will also depend of AMD and Intel also supporting DXR in their upcoming graphics chips (AMD has already hinted that they are working on some sort of raytracing support for their future chips). As the hardware becomes more common outside of just the very high-end enthusiast crowd I think we’ll start to see some really exciting things done with it, but I would also expect this take at least a couple of years (at which point we should also see the second or maybe even third-generation of raytracing chips from Nvidia). The future for graphics continues to be interesting, I can’t wait to see it.

Path Tracing vs Ray Tracing

Path Tracing vs Ray Tracing

Path tracing is all the rage in the offline rendering space these days. From Cycles to SuperFly (based on the open source Cycles) to Octane, most new rendering engines seem to be using this technology. Sometimes referred to as “unbiased, physically correct rendering” what is path tracing, how is it different to ray tracing and is it the future of high quality offline rendering? I will be looking to answer all of those questions in this blog post for anyone confused by the changing landscape of rendering engines (note that I will be talking about offline rendering in this post, as opposed to realtime rendering).

So first up the question: what is path tracing? Unfortunately the name fails to be terribly descriptive and when I first heard about it I thought it was simply a different name for ray tracing. In fact perhaps the easiest way to explain path tracing is to compare it to the more familiar ray tracing. So in ray tracing a ray is sent out from the virtual camera into the scene and traced until it intersects with a solid body. At this point a ray is cast to each of the light sources in the scene to calculate illumination and surface shading is calculated for the intersection point. If the surface is transparent the ray is sent out further into the scene, possibly at an angle to simulate refraction. If the surface is reflective a ray is sent out at an angle away from the object. Now I often see ray tracing touted as a magic fix for rendering (usually in discussions on realtime rendering for games) in online discussions, as if ray tracing somehow provides physically accurate results. Well it doesn’t. It comes closer than triangle rasterization (the technology employed in almost all games, and what graphics cards are optimized for) but it’s no simulation of reality. It gives us reflections and refractions virtually for free and it gives very nice hard shadows (unfortunately in the real world shadows are rarely if ever perfectly sharp). So just like rasterization engines have to cheat to achieve reflections and refractions (pay close attention to reflective surfaces in games, they either reflect only a static scene, or are very blurry or reflect only objects that are on screen), a ray tracer has to cheat to get soft shadows, caustics, and global illumination to name a few effects required to achieve photo realism.

Now a path tracer is like a ray tracer on steroids. Instead of sending out one ray it sends out tens, hundreds or even thousands of rays for each pixel to be rendered. When it hits a surface it doesn’t trace a path to every light source, instead it bounces the ray off the surface and keeps bouncing it until it hits a light source or exhausts some bounce limit. It then calculates the amount of light transferred all the way to the pixel, including any colour information gathered from surfaces along the way. It then averages out the values calculated from all the paths that were traced into the scene to get the final pixel colour value. If that sounds like a rather brute force approach to you then you are right. It requires a ton of computing power and if you don’t send out enough rays per pixel or don’t trace the paths far enough into the scene then you end up with a very spotty image as many pixels fail to find any light sources from their rays. It also requires light sources to have actual sizes, a bit of a departure from traditional point light sources that have a position but are treated like an infinitely small point in space (which works fine for ray tracing and rasterization because they only care about where the light is in space, but a path tracer needs to be able to intersect the light source). Now path tracing gives us all of the things that ray tracing doesn’t give us out of the box: soft shadows, caustics and global illumination. You should still not confuse it with being a true simulation of the real world however, since it still doesn’t fully simulate complex surfaces like skin, instead relying on shader tricks like subsurface scattering to fake these. There is also a practical limit to the number of paths you can trace from each pixel and how far you can trace them before giving up. If you were to simulate photons in the real world you would have to cast billions of paths and trace them almost infinitely (well at least until they leave the area you are rendering) and you would have to do this in an environment modelled down to an atomic scale. That’s not practical so I think we will always be stuck with an approximation, after all we just need to create images that look real to humans, which is a much lower threshold than “simulate reality completely”.

So is path tracing the future of high quality rendering? I do think it is. As computers (and in particular GPUs) continue to scale up in speed path tracing continues to become more and more practical and it requires less cheating than ray tracing (and far less than rasterization). This means less work for the artists to achieve stunning photoreal results, and that’s always a good thing (I think anyone who has worked with a high end ray tracer like MentalRay can appreciate just how tricky it can be to tune all the myriad of options to achieve the result you want). That being said I think that at the present I would not recommend using a path tracer in all circumstances. In the end it is all about choosing the right tool for the job. For single images where you want as much quality as possible and don’t mind it taking potentially hours to render path tracing is great. However if you need to render a large number of images (for a comic or an animation) path tracing may not be the right choice (especially if you are a solo artist without a render farm). In that case tweaking a ray tracer can give you almost as nice results but with a fraction of the render time.

The crux of the problem is that with a path tracer you are locked into an all or nothing approach. If you turn down quality too much you get a grainy image, which you can use to preview but which is wholly unsuitable for production use. So to get a usable image you have to tweak the quality settings until you are just at the point where most of the grain is gone or use progressive refinement and let it run until it looks good (which is a great feature by the way). In contrast with a ray tracer you can generally turn off most of the expensive features (global illumination mostly) and get a very high quality result rendered very quickly. And losing GI is often not a big deal, most competent artists can quickly fake most of what you get with GI by tweaking ambient lighting and popping in a few extra weak lights in places that don’t actually have any light sources (for example to fake light bouncing off a red wall you might just place a weak red area light onto the wall that is good enough to fool just about anyone looking at the resulting image). Of course faking always requires more time and skill on the side of the artist so eventually this won’t be needed, but until path tracing times are measured in minutes per frame, as opposed to the hours or days they are now, ray tracing (or rasterization, especially micropolygon rasterizers like the one powering RenderMan) remains the better option for many classes of rendering tasks.