You've got a couple of things going on here, one of which is stretching the limits of PM as it's currenlty implemented...
I duplicated your 100M photon test, and found that this does get to be a choke point. That's added another thing to my "Things to do" backlog, but out of scope for this discussion.
Back to the scene:
The shadow effect is there, its simply getting buried under the noise. Its subtle to begin with, because your environment texture is lighting brightly from pretty much any angle. This is normal for the sort of day portrayed in that texture.
20 samples is way too low, and you do want some environment smoothing. 2000 samples may even be too low for a 100M photon map of such a scene. In general, you will want the samples to change roughly in proportion to the total photons, with some fine tuning to minimize noise and light-bleed.
This particular scene seems to benefit from "Final Gather." This means that it uses Monte-Carlo sampling for the first bounce. The manual says this is the slowest method (Given identical settings), but that is partly compensated by the fact that you can get fairly good results with a much smaller map, which is faster to build, and smaller number of sample rays, so you don't have to spend as much time tracing them.
I did some renders with final gather for you, with 20M photons and 8 MC sample rays. they had Med AA, 4/8 turned on.
First has 200 sample photons, which is a bit noisy. Second has 2000, which is starting to cause some color bleed. (See the bottom edge of the cube! This means that 2000 are a bit too many for that map, but may still not be enough for a larger map.)
I realized that the cylinder, with its reflective surface, is exactly the thing to cause caustics, so I did a third run. I started with the same settings as the second, but added a 2M Caustics map, sampled at 1000. See the difference?
You've got a couple of things going on here, one of which is stretching the limits of PM as it's currenlty implemented...
I duplicated your 100M photon test, and found that this does get to be a choke point. That's added another thing to my "Things to do" backlog, but out of scope for this discussion.
Back to the scene:
The shadow effect is there, its simply getting buried under the noise. Its subtle to begin with, because your environment texture is lighting brightly from pretty much any angle. This is normal for the sort of day portrayed in that texture.
20 samples is way too low, and you do want some environment smoothing. 2000 samples may even be too low for a 100M photon map of such a scene. In general, you will want the samples to change roughly in proportion to the total photons, with some fine tuning to minimize noise and light-bleed.
This particular scene seems to benefit from "Final Gather." This means that it uses Monte-Carlo sampling for the first bounce. The manual says this is the slowest method (Given identical settings), but that is partly compensated by the fact that you can get fairly good results with a much smaller map, which is faster to build, and smaller number of sample rays, so you don't have to spend as much time tracing them.
I did some renders with final gather for you, with 20M photons and 8 MC sample rays. they had Med AA, 4/8 turned on.
First has 200 sample photons, which is a bit noisy. Second has 2000, which is starting to cause some color bleed. (See the bottom edge of the cube! This means that 2000 are a bit too many for that map, but may still not be enough for a larger map.)
I realized that the cylinder, with its reflective surface, is exactly the thing to cause caustics, so I did a third run. I started with the same settings as the second, but added a 2M Caustics map, sampled at 1000. See the difference?
Thanks again for looking at this Luke.
Yes, you can see the reflection from the cylinder on the ground, which is cool.
I turned the caustics and the AA off to squeeze more memory and time for the photons whilst experimenting - renders at 100M take 2 to 13 hours and most of 6GB. Plus I wanted to remove as many variables as possible.
I have two issues with MC / Final Gather - I cannot get rid of the remaining noise without losing the shadows too. The checkered surface is (supposedly) 100% smooth, but looks like sandpaper.
Also, whilst the scene has a lot of 360 degree lighting, the sun in this HDRI is quite small and about 4 times the surrounding brightness. On a sunlit day, you get a pronounced shadow from anything tall, so why not in my scene?
In a raytrace, even though diffuse reflections effectively sample a range of angles, shouldn't the largest component be the directly reflected one, which would show the shadows?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've done a few more tests. I was able to get stronger shadows by reducing the environment smoothing, at the expense of creating more noise. For my test (still at 20M photons, for reasonable testing speed) 1000 smoothing seemed to be the best balance, as not smoothing enough runs into issues. I also reduced the sample photons to 1000 & 1500, to reduce color bleed. This tends to increase noise as well. I've included a sample with the MC ray phase boosted to 16 rays to see if that makes a difference.
At this scale, you really are trading off several downsides, and try to find the best balance.
Currently, photons are spawned from entirely random locations on the environment sphere, with random in-coming vectors. They are then bounced through the scene until they are entirely absorbed. This tends to create noise, especially in environments that are very contrasty, as yours is.
There are two potential solutions to this:
Increase the number of photons. This is a Stochastic process, and as the number of samples grows, so will the smoothness of the image. Problem is, you've already caped out the top end of what is practically possible on your system. It may be possible to reduce the memory footprint of the photon map, by about 50%, but that only buys you so much.
Try to improve the data spacing in the map. Instead of totaly random, go for something well-distributed, possibly based on a Poisson distribution. Perhaps also give priority to brighter areas of the environment map..
Either one will take some thought and planning to implement in code.
I've done some simple profiling, and I think that the crazy render times that you are experiencing are due to lots of GC activity as you get close to your memory cap. I have at least one location where I may be able to reduce that. Time to experiment....
First, you should almost always use final gather for photon mapping. I'm not sure if it was even a good idea to include any other option. I suppose directly using the photon map is useful for debugging your scene, if you want to see where the photons are getting stored. But it's not a good way to generate your final image. It has artifacts that can't be completely eliminated in any way.
There are four ways to reduce noise: by using more rays/pixel (by a combination of "max rays/pixel" and "rays to sample environment), by using more photons to estimate the light, by increasing the environment smoothing, and by adding a noise reduction filter to your camera. Each of those has strengths and weaknesses, so you want to figure out the best combination of them for your scene.
Using more rays is the "best" option in the sense that it reduces noise without making anything else about the image worse. But of course, it also slows down rendering. So it's a question of how much time you're willing to let it spend.
Using more photons to estimate the light reduces noise, but also blurs the photon map. You can then make it sharper again by increasing the total number of photons, but that costs time and memory. So again, it's a tradeoff. Increase the photons to estimate light until the noise is acceptable, then increase the total photons until the sharpness is acceptable.
Environment smoothing similarly reduces noise from the environment, at the cost of blurring the lighting. So this is another thing you can experiment with.
The noise reduction filter reduces noise, but also can blur fine details in your image. Use it cautiously, but it can be a useful piece of your overall strategy. I've found that it can reduce the overall number of rays needed by about a factor of 2 without noticeable blurring.
Your scene is a very simple one. It shouldn't need anything close to 100 million photons. Something in the range of 100,000 seems more reasonable.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Talk about the forest and the trees... did a little more experimenting. Peter is right, I got tunnel vision focusing on the photons, when most of the noise in recent renders is from undersampling in the monte carlo stage.
See attached for purely PM renders. The shadows are not sharp, but it is also not nearly as noisy.
And here's one with Final Gather 64 rays, and the AA turned up to 32/64. There is still noise, but if you compare it to the renders from a couple of days ago, you see that its smoothing out. The shadows are also sharper. This does end up taking longer to render. (~40 min vs <3 for the pure PM at the same map settings)
MThis is really helping thanks Luke, it's helping me really get the final scene right and I am learning a lot about how AOI works. I've had one 1920x1080 render running for four days now - which will be sub-standard, but you've given me enough help that the next one should be right and a lot quicker.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
MThis is really helping thanks Luke, it's helping me really get the final scene right and I am learning a lot about how AOI works. I've had one 1920x1080 render running for four days now - which will be sub-standard, but you've given me enough help that the next one should be right and a lot quicker.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You've got a couple of things going on here, one of which is stretching the limits of PM as it's currenlty implemented...
I duplicated your 100M photon test, and found that this does get to be a choke point. That's added another thing to my "Things to do" backlog, but out of scope for this discussion.
Back to the scene:
The shadow effect is there, its simply getting buried under the noise. Its subtle to begin with, because your environment texture is lighting brightly from pretty much any angle. This is normal for the sort of day portrayed in that texture.
20 samples is way too low, and you do want some environment smoothing. 2000 samples may even be too low for a 100M photon map of such a scene. In general, you will want the samples to change roughly in proportion to the total photons, with some fine tuning to minimize noise and light-bleed.
This particular scene seems to benefit from "Final Gather." This means that it uses Monte-Carlo sampling for the first bounce. The manual says this is the slowest method (Given identical settings), but that is partly compensated by the fact that you can get fairly good results with a much smaller map, which is faster to build, and smaller number of sample rays, so you don't have to spend as much time tracing them.
I did some renders with final gather for you, with 20M photons and 8 MC sample rays. they had Med AA, 4/8 turned on.
First has 200 sample photons, which is a bit noisy. Second has 2000, which is starting to cause some color bleed. (See the bottom edge of the cube! This means that 2000 are a bit too many for that map, but may still not be enough for a larger map.)
I realized that the cylinder, with its reflective surface, is exactly the thing to cause caustics, so I did a third run. I started with the same settings as the second, but added a 2M Caustics map, sampled at 1000. See the difference?
You've got a couple of things going on here, one of which is stretching the limits of PM as it's currenlty implemented...
I duplicated your 100M photon test, and found that this does get to be a choke point. That's added another thing to my "Things to do" backlog, but out of scope for this discussion.
Back to the scene:
The shadow effect is there, its simply getting buried under the noise. Its subtle to begin with, because your environment texture is lighting brightly from pretty much any angle. This is normal for the sort of day portrayed in that texture.
20 samples is way too low, and you do want some environment smoothing. 2000 samples may even be too low for a 100M photon map of such a scene. In general, you will want the samples to change roughly in proportion to the total photons, with some fine tuning to minimize noise and light-bleed.
This particular scene seems to benefit from "Final Gather." This means that it uses Monte-Carlo sampling for the first bounce. The manual says this is the slowest method (Given identical settings), but that is partly compensated by the fact that you can get fairly good results with a much smaller map, which is faster to build, and smaller number of sample rays, so you don't have to spend as much time tracing them.
I did some renders with final gather for you, with 20M photons and 8 MC sample rays. they had Med AA, 4/8 turned on.
First has 200 sample photons, which is a bit noisy. Second has 2000, which is starting to cause some color bleed. (See the bottom edge of the cube! This means that 2000 are a bit too many for that map, but may still not be enough for a larger map.)
I realized that the cylinder, with its reflective surface, is exactly the thing to cause caustics, so I did a third run. I started with the same settings as the second, but added a 2M Caustics map, sampled at 1000. See the difference?
Thanks again for looking at this Luke.
Yes, you can see the reflection from the cylinder on the ground, which is cool.
I turned the caustics and the AA off to squeeze more memory and time for the photons whilst experimenting - renders at 100M take 2 to 13 hours and most of 6GB. Plus I wanted to remove as many variables as possible.
I have two issues with MC / Final Gather - I cannot get rid of the remaining noise without losing the shadows too. The checkered surface is (supposedly) 100% smooth, but looks like sandpaper.
Also, whilst the scene has a lot of 360 degree lighting, the sun in this HDRI is quite small and about 4 times the surrounding brightness. On a sunlit day, you get a pronounced shadow from anything tall, so why not in my scene?
In a raytrace, even though diffuse reflections effectively sample a range of angles, shouldn't the largest component be the directly reflected one, which would show the shadows?
That's a good reason to do without Caustics....
I've done a few more tests. I was able to get stronger shadows by reducing the environment smoothing, at the expense of creating more noise. For my test (still at 20M photons, for reasonable testing speed) 1000 smoothing seemed to be the best balance, as not smoothing enough runs into issues. I also reduced the sample photons to 1000 & 1500, to reduce color bleed. This tends to increase noise as well. I've included a sample with the MC ray phase boosted to 16 rays to see if that makes a difference.
At this scale, you really are trading off several downsides, and try to find the best balance.
Currently, photons are spawned from entirely random locations on the environment sphere, with random in-coming vectors. They are then bounced through the scene until they are entirely absorbed. This tends to create noise, especially in environments that are very contrasty, as yours is.
There are two potential solutions to this:
Either one will take some thought and planning to implement in code.
I've done some simple profiling, and I think that the crazy render times that you are experiencing are due to lots of GC activity as you get close to your memory cap. I have at least one location where I may be able to reduce that. Time to experiment....
A few thoughts on this.
First, you should almost always use final gather for photon mapping. I'm not sure if it was even a good idea to include any other option. I suppose directly using the photon map is useful for debugging your scene, if you want to see where the photons are getting stored. But it's not a good way to generate your final image. It has artifacts that can't be completely eliminated in any way.
There are four ways to reduce noise: by using more rays/pixel (by a combination of "max rays/pixel" and "rays to sample environment), by using more photons to estimate the light, by increasing the environment smoothing, and by adding a noise reduction filter to your camera. Each of those has strengths and weaknesses, so you want to figure out the best combination of them for your scene.
Using more rays is the "best" option in the sense that it reduces noise without making anything else about the image worse. But of course, it also slows down rendering. So it's a question of how much time you're willing to let it spend.
Using more photons to estimate the light reduces noise, but also blurs the photon map. You can then make it sharper again by increasing the total number of photons, but that costs time and memory. So again, it's a tradeoff. Increase the photons to estimate light until the noise is acceptable, then increase the total photons until the sharpness is acceptable.
Environment smoothing similarly reduces noise from the environment, at the cost of blurring the lighting. So this is another thing you can experiment with.
The noise reduction filter reduces noise, but also can blur fine details in your image. Use it cautiously, but it can be a useful piece of your overall strategy. I've found that it can reduce the overall number of rays needed by about a factor of 2 without noticeable blurring.
Your scene is a very simple one. It shouldn't need anything close to 100 million photons. Something in the range of 100,000 seems more reasonable.
Talk about the forest and the trees... did a little more experimenting. Peter is right, I got tunnel vision focusing on the photons, when most of the noise in recent renders is from undersampling in the monte carlo stage.
See attached for purely PM renders. The shadows are not sharp, but it is also not nearly as noisy.
And here's one with Final Gather 64 rays, and the AA turned up to 32/64. There is still noise, but if you compare it to the renders from a couple of days ago, you see that its smoothing out. The shadows are also sharper. This does end up taking longer to render. (~40 min vs <3 for the pure PM at the same map settings)
Last edit: Luke S 2016-10-27
MThis is really helping thanks Luke, it's helping me really get the final scene right and I am learning a lot about how AOI works. I've had one 1920x1080 render running for four days now - which will be sub-standard, but you've given me enough help that the next one should be right and a lot quicker.
MThis is really helping thanks Luke, it's helping me really get the final scene right and I am learning a lot about how AOI works. I've had one 1920x1080 render running for four days now - which will be sub-standard, but you've given me enough help that the next one should be right and a lot quicker.