First, you need the mental ray plugin enabled. Windows>Settings>Plug in
manager:
Mayatomr.mll. That's the library you want to load. I set up mine
to auto-load, you might want to as well. Check both:
Make a new material for your things. I just choose Lambertian at
first, it won't matter. Then in the material settings on right side,
click the boxy thing next to "colour". We are going to specify colour
from some other function... :
It takes you to create render node window. In mental ray tab,
choose "mib_amb_occlusion":
Back at the main window, it's properties will be there. (Red)
Choose 256 samples, this is how many outward rays sample the scene at
each point. More you use, smoother the effect. 256 is good. (Orange)
These handy arrow things take you to and from attribute windows. What
it's doing is moving you between node connections in the hypergraph, if
you've seen that thing. (Yellow) Click that movie snappy thing to go
to the rendering window:
You won't see much yet. Choose mental ray from pulldown as the
rendering engine to use. It might just work after doing that. If it
doesn't, go into the render settings (red), movie cut thing with 2 radio
buttons:
In order to get what you see, in my case I didn't have to have any
global illumination or other "ambient occlusion" setting things even
enabled. It just worked because of the shader we chose in the material.
You will want the renderer to take multiple samples at each
pixel/fragment. You can manually specify this, or choose "production".
You will notice a smoother result: