15th
Notes on Mastering OpenGL ES for iPhone
Continuing my iPhone Tech Talks Toronto notes with “Part One” of Allan Schaffer’s talk, Mastering OpenGL ES for iPhone. Schaffer is the Graphics Technology Evangelist at Apple.
Side note: if you’re not familiar with Jeff LaMarche’s excellent OpenGL ES From the Ground Up series on his blog, bookmark it now.
Schaffer opened the talk with a high level overview.
“Beyond 3D… is 2D, you weren’t expecting that, were you?”
(My take on this comment was the “two dimensional” iPhone user interface is all implemented in 3D leveraging the GPU. Apple has famously done this with OS X for years and built on this experience to great effect on iPhone. Not for Aero in-your-face stuff, but fast composition, transparency, drop shadows etc. that would be painful to pull off in software.)
“OpenGL is easy to learn, but hard to master.”
Schaffer’s backed this claim with a few slides that grouped many detailed OpenGL topics into broad areas. His talk would skip the introductory part and launch right into the middle. He asked for a show of hands to gauge the audience’s experience with OpenGL, and determine who would learn the most, “or can just tolerate it for two hours.” which got a laugh.
Schaffer’s reviewed the major differences between OpenGL ES 2.0 and 1.1 : “it’s not just a feature bump.” He emphasized OpenGL ES 2.0 devices are “a different class of hardware” and older iPods and iPhone would never support OpenGL ES 2.0.
OpenGL ES 1.1 is fixed-function, with a pre-defined pipeline. You can enable or disable operations, but you can’t change them. A complex scene may require multiple passes.
OpenGL ES 2.0 is a programmable pipeline, with no limit on the functions. You can apply complex algorithms in a single pass. It is not backwards compatible.
OpenGL ES 2.0 rendering path uses Shaders. (I believe Apple implemented OpenGL ES 1.1 functions using shaders to allow 1.1 code to run on 2.0 hardware, but that wasn’t discussed here.)
“You program the operations. Listing the possible…would be like listing all the algorithms you could implement in C.”
Demo time. Schaffer showed a pong game where the ball left swirling, gaseous trails. The room sat in shocked silence. “That wasn’t the reaction I was looking for,” Schaffer says. “Let me repeat: we’re seeing a real-time fluid dynamics simulation implemented on the GPU. On a phone!” Now, the applause.
Different pong demo. The ball now casts a green glow on a cobblestone background. The cobblestones cast complex shadows as the glow passes back and forth above them. “This is nine lines of shader code,” says Schaffer. Ridiculously impressive.
Someone asked if this code would be available anywhere—not yet.
Schaffer showed some of the common OpenGL books, noting ES is a subset, so a book on OpenGL 2.0 may only have a chapter about ES. He called out the OpenGL ES 2.0 Programming Guide by Aaftab Munshi, Dan Ginsburg, and Dave Shreiner. It was published before iPhone shipped with 2.0 support, but the authors have posted Xcode projects for all their sample code online at http://www.opengles-book.com as well as an online iPhone chapter.
Speaking of Xcode projects, you should check out the OpenGL ES Application Template that ships with Xcode. It includes the details to determine at runtime whether the device you’re running on can render 2.0 or should fall back to 1.1.
Unfortunately you can’t statically check for OpenGL ES 2.0 support because some constants were defined in earlier APIs before it was actually supported. Instead, use initWithAPI for kEAGLRenderingAPIOpenGLES2 and check against a nil context:
context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2];If your application requires OpenGL ES 2.0, you must add that requirement to your info.plist. There are now several UIRequiredDeviceCapabilities, including wifi, gps, video, etc. Do not attempt to “sniff” these features, let Apple tell you what’s available. (The relevant keys for this talk are opengles-1 and opengles-2 if your application requires the presence of the OpenGL ES 1.1 or 2.0 interfaces, respectively.)
EAGL (“Eagle”) stands for Embedded Apple Graphics Library. Maybe.
“CoreAnimation is your display manager.”
UIView contents are cached by CALayer.
OpenGL ES renders into a CAEAGLLayer.
16-bit RGB565 is optimal, versus the iPhone default RGBA8 (RGB565 is the Android default, using 5 bits for red, 6 bits for green and 5 bits of blue for each pixel, totaling 16 bits. The human eye is most sensitive to green, so it gets twice the shades as red or blue.) The issue is memory bandwidth—you can send more 16 bit pixels than 32 bit pixels. Of course there are tradeoffs, since 16 bit color can lead to visual artifacts (such as banding.)
Avoid layer.transform—swap width and height, rotate 90 degrees. 90 degree rotations are free for most cases. Unfortunately, your view controller may be applying rotations as well (e.g. in response to the user flipping the device to landscape mode.) You can note these and cancel them out.
You can layer UIKit atop an OpenGL CAEAGLLayer, if the UIKit content is static. But the reverse is very expensive. There are only two classes of programs that have a good reason to overlay OpenGL atop UIKit, one being Augmented Reality applications that are all the rage this year.
It makes no sense to have multiple EAGL Layers (“that’s INSANE!”) …do your compositing inside a single EAGL layer.
CoreAnimation uses pre-multiplied alpha (transparency), which is expensive. Opaque layers are much faster.
Schaffer gave a live demo of the Core Animation Instrument in action. Basically it will colorize the display to show layers, updates, etc. Unlike logging or debugging, this instrument enables a modes on the device that don’t require it to be tethered—so once you flip it on, you can unplug the phone and walk around with it in Psychadelic Mode. An impressive part is this enables you use it to examine every app on the device, not just what you have the source code for.
CADisplayLink was introduced in SDK 3.1, which lets you synchronize your updates with the actual display refresh (no tearing, no over-driving, etc.) You should stop drawing if nothing’s happening. displayLink.paused:YES to stop notifications, or [timer invalidate].
Note iPhone 3GS requires redraw every frame if presentRenderBuffer is used.
“If you’re floating-point heavy, try this.” Schaffer demonstrated how to set up an Xcode project to optimize for Thumb-2. You can disable Thumb for ARMv6, which slows floating point operations, but leave it enabled for ARMv7. Search Build configurations for ‘Thumb’, select ”Compile for Thumb” row and click the gear in the lower-left corner to ‘Add Build Setting Condition’. Change ‘Any Architecture’ to ‘ARMv6′ and leave that box unchecked. The result is a “fat” binary with code for both processors, but the size increase is negligible: “if you’re using OpenGL your assets are huge anyway.”
Hat tip to Revolution Robotics for detailing the Thumb setting! I’d forgotten the gear step.
That’s it for Part One. I may have missed some details :) I’m really looking forward for Part Two of Allan Schaffer’s talk on iTunes. (You can find the 2008 iPhone Tech Talks http://bit.ly/7Z9rHz (ADC on iTunes) and Apple said they would post 2009 “in a few weeks”. )
My next iPhone Tech Talks Toronto notes will be from Safari and Web Technologies Evangelist Vicki Murley, from her presentation on Finding Your Way with Location and Maps.
