Hey, I'm back from my two week long (including travel) USA business trip! I've been back for a few days and I am suffering from massive jet lag. Last night was the first night I managed to sleep. But I'll talk about my trip some other time. After such a prolonged absence I don't want my first post to be off-topic So it is not going to be about America or my Guild Wars impressions, but about development.
This will be basically the same post (plus some recent developments) I would have written two weeks ago if my trip didn't have prevented it, but since so much time has passed I am not going to structure it a journal as I wanted, but instead go for a feature list. This has the added bonus of creating the illusion than I worked more :).
Bump mapping & specular mapping
Ah yes, the ever so elusive first half of an unified lighting system, my true goal vis-a-vis shader lighting. I finished bump mapping, but results where not that great. As you saw in the last screenshots, the bump effect was visible and seemed correct, but the end result was not that pretty. I attributed this to my poor quality normal maps. And parallax mapping seemed to work again, but I couldn't get rid of swirly artifacts. So I decided not to use parallax mapping.
This would have been my status two weeks ago.
But yesterday I got the idea to create some new normal maps using some better tools. I downloaded CrazyBump. Using the demo version I created some normal and height maps, and these did behave a lot better. The problem was that the stool at least did not look as good as back in the day when I attempted deferred rendering. That was a long time ago, long before I abandoned the Dwarf Fortress concept. Deferred rendering is such a relativity new and complicated technique that last time, while I did manage to put together a working implementation of it based on scarce resources and documentation available on the net and I did understand the principles of it, I did not truly understand a lot of the full implementation. When I tried to modify it, fix bugs and add specular mapping to it, I failed. Maybe one day I will be successfully rolling with deferred rendering, once all the kinks have been ironed out, like antialiasing, transparency, multiple materials, etc., but for now I am sticking with forward rendering. But my stool with CrazyBump generated maps was not looking as good as it did with deferred rendering.
So I came up with the idea not to generate the maps based on the textures I have, but on the height map that was specially created for the deferred renderer. And the stool now looks great! Maybe not quite as good, but very close. It is hard to tell because last time I had a lot of small lights illuminating the stool, but this time it is only one. Anyway, the shader is working and all I need now is professional height maps & CrazyBump or just professional normal maps!
But this was not enough. The main reason I stopped using deferred rendering was that I couldn't get specular mapping to work. But yesterday I added specular maps to the bump mapping shader in about 5 minutes and it looks great!
Even parallax mapping has less swirly artifacts, but they are still present. I might figure out a correct bias value, but I won't bother. I won't be using parallax mapping right now. Probably by the time DirectX 11 tessellation is common, I'll probably be rolling with parallax mapping.
Content management system
Game engines, except for rendering, are largely logistical problem resolvers, They make sure that out of a very large pool of data, larger than what can fit in memory, the right resources are available at the right time, are drawn only when needed and interact with other resources when needed, while maintaining interactive framerates. One part of this is locating and loading the resources.
XNA uses content projects in order to locate and load all the resources. It even "compiles" the resources that are listed there, converting formats and packaging them. But for my needs this is not ideal. I want to be able to pass around the executable so that anyone can add new resources. This game will have a highly procedural nature, so I want it to even be able to create a completely new object type without having to restart the application or reload the level. Having content projects that must be compiled by artists is anyway a huge no no.
So I am creating a system that keeps track of resource locations and their use, allowing very flexible control on what is used and what isn't. With this system I can add a new object type to the engine with only one line of code or dynamically by reading some information from the disk or due to user input.
I am doing this in two stages. In the first stage textures and meshes are still part of the content project, but my content management system does not care about this. For the second stage I'll make the system loads textures from external files. XNA 4.0 removed the possibility to load a bitmap from a random file from the disk, so I've been searching around the net for alternatives. I found one and once I make sure it works, I'll tell you which one it is. For the third stage I'll make the system load also meshes from external resources.
There is an interesting situation right here. I like systems that react promptly. I hate waiting for stuff to happen. And start up time for the game is getting quite long. So I did some benchmarking, and I found that loading meshes and textures takes 2.6 second. Add to this procedural terrain generation, building and grass placement and you get to quite a few seconds. Eliminating texture loading, I got to 1.6 seconds. Loading meshes is a massive bottleneck and I do not know why. I need to figure out why this bottleneck exists.
How did I eliminate texture loading time? Using the next feature!
Multithreaded texture streamer
Right now when the game loads up all textures are blanks. If I press a button, a new thread is launched that queries the content management system on the current status of objects, find out that the textures are missing, and proceeds to stream in all the missing texture. It is very fast, so in order to make the streaming more aparent I added a 250 ms delay after each texture load. Needless to say, the final version won't have this delay. Nor will you need to press a button to trigger texture load These are pure development features, because I can test a lot without waiting for textures to load. Currently, the system does not require synchronization The thread runs and gladly stomps all over the data it needs without having to use mutexes. This was intentional, but if the streamer grows in complexity, I will probably need to add mutexes and figure out a solution so that the streaming thread and the rendering thread only keep each other locked for the minimum time possible.
The game picks up the loaded textures and renders the whole scene with minimal state changes to maximize performance. I really need to add frustum culling.
New terrain texturing shader
Remember the problem I was having with terrain texture tilling? Low tiling value meant good looking distant terrain but very blurry and stretched out near terrain. Setting it to high results in good close terrain, but in distant lands there is a very apparent and disturbing tilling effect visible.
I fixed this by sampling textures twice, one time with low tiling factor and one time with high and using both values to texture the terrain. This is a compromise but pretty much solves the problem. I am currently torn between two settings for tilling, both giving good results.
I came up with this idea reading some UDK terrain texturing tutorials. They mention this problem and give a lot of solutions for it. In UDK you do everything visually, adding elements to and editor and configuring the relations between these elements, but that didn't stop me from implementing a similar idea in a traditional HLSL shader.
I'm still not done with terrain texturing. Currently the texture is given by the height of a point. This is good for an initial setup of the map, but I want to give full control over texturing, so I'll have to expand upon the system in the future.
Diffuse wrap lighting is a strange system, where the light given by a directional light wraps around an object. It is mainly used for (very poorly) approximating subsurface scattering. I probably won't end up using it a lot, but I did implement it and I might have a few objects that will behave this way.
With all these visually centered features, the game engine now looks a lot better. So how come I didn't show any pictures? This is because after such an absence I want to show something more impressive, and while it looks better, you'll end up still seeing the same objects. Barrels are boring Give me a couple of days to add back all the objects I have, including trees, and I'll create a new video.