osx - Capture all NSWindows as active images like Mission Control in Mac OS X -

i'm looking aggregate live representations of windows. mission control (exposé), want extremely quickly access image buffer of given nswindow or screen. ideally, want composite these live images in own opengl context can manipulate them (scale , move windows screen captures around).

things slow:

• cgdisplaycreateimage
• cgwindowlistcreateimage
• cgdisplayidtoopengldisplaymask & cglcreatecontext & cgbitmapcontextcreate

any other ideas? i'm trying achieve 60 fps capture/composite/output best can of these methods ~5 fps (on retina display capturing entire screen).

unfortunately, haven't found away capture framebuffers of individual windows, figured out next best thing. method capturing live view of entire screen(s) opengl:

avfoundation setup

_session = [[avcapturesession alloc] init]; _session.sessionpreset = avcapturesessionpresetphoto; avcapturescreeninput *input = [[avcapturescreeninput alloc] initwithdisplayid:kcgdirectmaindisplay]; input.minframeduration = cmtimemake(1, 60); [_session addinput:input]; avcapturevideodataoutput *output = [[avcapturevideodataoutput alloc] init]; [output setalwaysdiscardslatevideoframes:yes]; [output setsamplebufferdelegate:self queue:dispatch_get_main_queue()]; [_session addoutput:output]; [_session startrunning]; 

on each avcapturevideodataoutput frame

- (void)captureoutput:(avcaptureoutput *)captureoutput didoutputsamplebuffer:(cmsamplebufferref)samplebuffer fromconnection:(avcaptureconnection *)connection {   cvpixelbufferref pixelbuffer = cmsamplebuffergetimagebuffer(samplebuffer);   const size_t bufferwidth = cvpixelbuffergetwidth(pixelbuffer);   const size_t bufferheight = cvpixelbuffergetheight(pixelbuffer);    cvopengltextureref texture;   cvopengltexturecachecreatetexturefromimage(kcfallocatordefault, _texturecache, pixelbuffer, null, &texture);   cvopengltexturecacheflush(_texturecache, 0);    // manipulate , draw texture want...   const glenum target = cvopengltexturegettarget(texture);   const gluint name = cvopengltexturegetname(texture);    // ...    glenable(target);   glbindtexture(target, name);    cvopengltexturerelease(texture); } 

cleanup

[_session stoprunning]; cvopengltexturecacherelease(_texturecache); 

the big difference here between other implementations avcapturevideodataoutput image opengl texture might use cvpixelbufferlockbaseaddress, cvpixelbuffergetbaseaddress, glteximage2d, , cvpixelbufferunlockbaseaddress. issue approach it's typically terribly redundant , slow. cvpixelbufferlockbaseaddress make sure memory it's hand not gpu memory, , copy general purpose cpu memory. bad! after all, we'd copying gpu glteximage2d.

so, can take advantage of fact cvpixelbuffer in gpu memory cvopengltexturecachecreatetexturefromimage.

i hope helps else... cvopengltexturecache suite terribly documented , ios counterpart cvopenglestexturecache better documented.

60fps @ 20% cpu capturing 2560x1600 desktop!