Maybe you already have a vision of a map in mind (be it a puzzle, series of puzzles, or an adventure), and maybe that map is highly detailed. The problem is, if you don't know a few basic things about the optimization of a map, it could take forever to compile. vvis, the program that works with these things called visleaves (we'll get to the definition in the "Visilibility" section below), will blindly obsess over every little detail of the visibility of your map, and at its worst it could take some hours to fully optimize all the visleaves! It has to do this for reasons described fairly clearly at this VDC Wiki page.

Here's an actual example I found recently of a map that was poorly optimized:



So, how do we get rid of that ugly blue mess and also reduce the compile time? Read on to learn the common tricks of the trade.

I'm covering this first because the compiler will not give you what you need to optimize your map if it has a leak.

Did you ever see this line in red when compiling a map (or even just a flash of red and then the compiler quit on you)? It's based on one simple principle: nothing except worldbrushes can ever touch the "void" that surrounds your map — even one teeny hole can render your map unplayable. So likely the compiler also said "Entity XYZ (<location of entity>) leaked!" One exception is brush entities: as long as their origin is on the inside, it's OK to have the actual brush be outside of your map.

To fix these leaks, first and foremost, DO NOT SEAL IN YOUR CHAMBER(S) WITH A BIG BOX. This is actually bad mapping practice, in part because it adds extra unnecessary visleaves, which can cause higher compile times and poorer performance. Instead, follow this process:

1) Hammer will have generated a pointfile (*.lin) that you can load (red text):



2) Look for a red line somewhere in your map. It should be easy to see at any viewing scale in the 2D views:



3) Once you've found it, look for where it goes "out of bounds"



4) Fix that instead of using the "big box" trick :P

It's likely you weren't even paying attention to detail and one little hole escaped our eye. You could've also been using the wrong kind of nodraw texture to seal your maps, or using a func_detail or other brush entity to seal your map. If the line goes through any of these and into the void, check the brush and its textures to see if you missed anything. The other piece of advice I can give is to work on the grid — 32 is a good grid size to work with for world brushes (e.g. chamber walls, floors, ceilings, deadly goo pits, and even many testing elements that aren't worldbrushes), and 8 or smaller for details only (e.g. overlays/decals, props, func_detail, tweaking alignment of doors).

Optimizing visibility is the biggest issue in Portal 2, more so than any other kind of optimization. That is what this guide will focus on.

Visleaves

So, you may be wondering at this point, "what's a visleaf?" A visleaf is a cuboid-shaped box (highlighted in a blue wireframe) that helps define a "cluster" of visible areas (or viscluster for short). Unless your map has only one visleaf, each visleaf touches another visleaf, and the two are connected by a portal. This is not at all in the sense of the word that we think of when we hear it; it's just a helper to determine which visleaves are potentially visible to each other — something which vvis computes to help ease the pain of our computers. Visleaves are necessary in Portal 2 (and every other Source game in fact) because they trim down the number of objects that need to be drawn (everything outside of the current viscluster needn't be drawn). If the whole map had to be drawn at all times, we'd have a lot of melted computers (remember, our computers are not like Aperture's — they cannot tolerate temperatures of up to 4000 degrees Kelvin).

Before we begin the optimization process, let's notice that you don't even have to run the map in game and use the mat_leafvis command. You can view visleaves right in Hammer via these things called portal files (*.prt). Pardon the image reuse; just focus on the blue text:



NB In order for this to work, you have to at least compiled the BSP/Geometry portion of your map. Your map should also be free of leaks, so in case you did not read the previous section, now would be a good time to go back and read it if your map has a leak. Remember, Aperture doesn't release anything into the wild until it's good and ready, and that includes stuff in your map.

A few things to know about how visleaves are made:

• Every 1024 units, on the thick, colored lines in Hammer's grids (2D views), a visleaf is created. This is simply to divide up large areas and has nothing to do with the worldbrushes (walls, floors, and ceilings) that you have created.
  
• Any and all worldbrush edges will slice up visleaves, though it depends on the texture. Most textures have this slicing property, but a few tool textures, such as clips, do not.
  
• Brush entities such as func_detail or func_door will NOT slice up visleaves — only worldbrushes do that. So if you want to make as complex-shaped a rotating panel as you desire, go for it.

Most of the entities mentioned in this guide can be used here fairly liberally without bumping up the entity cost for your map, as they are gone once the BSP is written — they are only needed for processing.

These force the compiler to cut a visleaf along a certain plane. Creating one is easy: make a worldbrush of nothing but the skip texture (tools/toolsskip), place your brush accordingly, and then use the hint texture (tools/toolshint) on the plane you wish to slice with. The hint texture's only purpose is to slice visleaves. The skip texture has no effect on anything, and that's why it's useful with the hint brush. It's also useful for easy selection when using instances (or those semi-deprecated prefabs). It has a very special effect in game because (for example) the grating in the image below is not bound by any other plane (which is how BSP works — not by vertices, but by planes). VBSP removes all skip brushes before compilation:



This means that you don't necessarily have to stretch the skip/hint brush to fit your hole (or whatever it is you want to fill), but it's good practice to do it anyway.

Often the purpose of these brushes is simply to reduce visleaves:



But there's more to it. Consider the above chamber. Without hinting, Bendy can somehow "see" what's inside that (deadly) pit. Moreover, Bendy doesn't always "see" what is on the other side of said pit. That simply doesn't make sense. Now with hinting, Bendy can only see what Bendy was meant to see: the entire chamber, and the pit is separate, so if you have something like deadly goo in there, the visleaf won't draw and use up extra resources on your computer. (Aside: in practice, things would be configured as per the second diagram without hinting, but this is just an illustration of how the hinting process works.)

Let's look at a real example of this:



And yet:



One last thing to know, especially if you're working on an overgrown-themed map (which often has lots of ceiling detail):



For more complex areas, if you've tried using func_detail and func_viscluster (where applicable) but still have issues, that's where these skip/hint brushes can help. Using them can initially be daunting at first, but think about it this way: what would be the cheapest way to fill your entire chamber with trigger brushes? Does the portal file reflect that? If not, then use hints to make sure that it does. Good candidates are often holes in the wall, small or large. This includes light strips and observation rooms, but perhaps also little cave-like areas in your test chamber. Doorways might be good places for skip/hint brushes too.

To help reduce clutter for this phase of optimization, you can uncheck the "Entities" and "World Detail" auto-created visgroups, as shown:

This simple brush entity is useful if you have visleaves in a big space but you need them to all act as one. Valve used this entity in the Wheatley-themed maps outside of the testing area (where there is low detail) to greatly reduce visleaf count and compile times.

As an example, here are some results from my compile log for one of my maps:

Without func_viscluster:

• visdatasize: 216351, compressed from 294576
  
• 6 minutes, 8 seconds (368 seconds total) elapsed

With func_viscluster:

• visdatasize: 161316 (approx. 25% reduction), compressed from 246272
  
• 36 seconds elapsed

In general the numbers are lower with vvis when working with func_viscluster.

To create a func_viscluster, just make a brush (ideally a huge one), and use the trigger (tools/toolstrigger) texture on it. Then use the "Tie to Entity" feature to turn it into a func_viscluster. Done. No need to set any fancy keyvalues, nor any need to use I/O on this entity.

If you have lots of little brushes in your map that will never move, turn them into a func_detail. Being a brush entity (and conveniently, the default one if you haven't changed Hammer's settings), func_detail doesn't chop up visleaves like regular worldbrushes do. Creation is simple: just select your brushes, use the "Tie to Entity" feature to turn it into a func_detail — no need to set any fancy keyvalues, nor any need to use I/O on this entity either. Good candidates for brush selection are:

• Round/curvy brushes: they have a lot of faces and are best turned into a func_detail. For example, stuff with round holes in them are best turned into a func_detail.
  
• Small or thin brushes: for example, frames on glass and grating, and even the glass and grating brushes themselves.
  
• Other special cases: for example, the beam structure outside of a Wheatley-themed test is often turned into a func_detail to save visleaves.

NB if any one of these is touching the void (e.g. you have a curvy test chamber wall), you'll need to seal it in with nodraw.



The case for func_detail can be made when you try compiling without knowing what func_detail even is and it takes forever. The general idea is that worldbrushes should be fairly simple and grid-aligned (kinda typical for Portal 2, thankfully). A few odd angles (such as 45° angles) won't hurt, but if it gets significantly more complex than that, a func_detail is needed. The downside of using func_detail is that lighting won't look as sharp or pretty on func_detail brushes as it will on worldbrushes.

Interesting fact: when the map is compiled, no matter how many func_details you have in your map, the compiler will merge them all into one, so go ahead and be as liberal as needed with this entity.

Why have all the Hammer pros out there obsessed over using nodraw in places they never expect the player to normally be? It's because no lightmap is compiled for it and it thus saves file space. For the same reason, it might also help save a bit of compile time during vrad (lighting), a phase of map compilation that often takes the longest.

But let's all learn a lesson from that old Portal 2 mod Aperture Tag: don't go too crazy with nodraw in every little place you think the player will never see! In that mod, players found ways to use the blue gel and get to places where nodraw can be clearly spotted. If you're new to Hammer, I suggest you only texture faces of brushes that touch each other as well as the outside.

Now, you may have noticed that in the texture browser, there is a nodraw texture (tools/toolsnodraw) and a portalable variant (tools/toolsnodraw_portal). You will never need to use the portalable variant and in fact doing so can be dangerous for your puzzle. There's also one other nodraw variant: glass footsteps (tools/toolsnodraw_glass_footsteps). This was used in the very first official Portal 2 map where the player breaks through the glass (only for the foley effect). Although it is a nodraw texture, it does NOT seal leaks!

The last thing I'll cover here is area portals. No, these are not anything like our favorite blue and orange elliptical things, nor are they even like world portals — they are just there to ensure that your computer can rest a bit easier. There are two main entities: func_areaportal and func_areaportalwindow. Both are meant to reduce the impact of a long draw distance — everything behind a closed area portal will not be drawn, so that your computer won't crash and burn while trying to run a huge map that features combustible lemons.

Area Portals and Visleaves

What effect do area portals have on visleaves? None really. They are just meant to reduce draw distnce by essentially "cutting off" parts of your map while it's running in-game. You see, visleaves are only affected by the geometry of your map, func_viscluster, and skip/hint brushes. They are meant to be optimized before your map is even running, whereas area portals are meant to "optimize" your map while it's running.

"Where to place them though..." you might be wondering. Just about any place you can place a skip/hint brush is also a decent place for an areaportal, but smaller holes and doorways are really the best places to use them. The entrance and exit of a test chamber are good examples of where using an area portal would be good, but not so much in the middle of a test chamber.



Something to know about func_areaportalwindow: it can be tied to the use of a prop_testchamber_door simply by selecting the door's "Area Portal Window Name" keyvalue to the name of your func_areaportalwindow. When you do this, the func_areaportalwindow will automatically open and close with your door, all without the need for additional I/O.

Warning: while Portal 2 maps won't compile if there's a regular leak, they will compile if there are areaportal leaks, and these could make your map look bad. So, if you ever see a flash of red during the compile process, STOP COMPILING, read the compile log if needed (located in the same location as your VMF file), and fix up those areaportal leaks! The pointfile can help here too — in this case it will find an open path from an areaportal to a different side of the same areaportal.

As a final word of advice, area portals aren't always necessary, and in fact a Portal 2 map can be well-optimized without them. It's really only worth it to use between two heavily-detailed areas.

• Too much nodraw, as noted above.
  
• Too little nodraw, as noted above.
  
• Incorrect use of area portals.
  
• But above all, forgetting to optimize your map is the biggest mistake relevant here. You'll be stopping lag issues cold (if not yours, someone else's) if you optimize your map. You'll also benefit from not having your time wasted because the compiler is taking 999999... seconds to finish.

More information from the VDC Wiki:

• The main page for optimization
  
• Optimizing visibility: more prevalent in Portal 2 than any other kind of optimization.
  
• What's a visleaf again?
  
• Area Portals
  
• Using Hint Brushes
  
• The func_detail entity
  
• The func_occluder entity: not necessary to use, but worth knowing about.
  
• The func_viscluster entity
  
• Tool Textures

I'd like to send out a very special thank you to Nintendo.erk, who kindly provided me with a map for some of these screenshots, and also to help me understand better the process of optimizing a Hammer map. I'd also like to send out thanks to Innocentive for his valuable insight on the optimization process and for helping tweak this guide.

Please note that there are still some works in progress. This section will be updated as I finish more guides for you!

You are currently viewing part 12/19 in the series.

| 00 | 01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 |

Previous Guide (miscellaneous useful stuff)
Next Guide (about elevator rooms and beginning and ending a singleplayer map)

Please feel free to leave a well-reasoned question or comment below. Either I or a qualified Test Chamber Associate (you'll know because they make all the good maps on the workshop) will respond. If your question or comment is not well-reasoned, I recommend reading over my guides again until you understand them 100%. No further information is required here or will be provided, and you will become an excellent Test Chamber Designer — using Hammer!