This was inspired by an xkcd on Depth Perception.
I know this is possible because of a simple but awe-inspiring piece I saw at Burning Man years ago. As I remember it, you sat in a swivel chair and looked into a binocular viewer. Through a clever system of mirrors and lens, your stereoscopic vision has been extended out to about the width of your arms. Looking through the viewer made distant objects — normally flat — appear in 3D. Middle distant objects were hyper-3D. Close objects were unaccountably confusing, like holding an object in front of one eye. 
The project was called the Telestereoscope, or informally Eyestilts. Visit the eyestilts site for a charming diary of design, prototyping, building, and installation.
The only problem with Randal’s scheme is that you couldn’t so much walk among the clouds as merely stand among them. The cameras are fixed and so there is no way to change your perspective. I thought about a larger version of the Burning Man Hyper-3D Swivel Chair, but considering we are talking about cameras separated by as much as a few hundred feet, I dismissed that as impractical.
But what if the cameras were fixed, but the view swiveled? As you turned, you could be selecting the views from a matched pair of cameras pointed at the horizon, a different view of the horizon as you swiveled. Buildings, clouds, even distant mountains would appear in 3D.
I imagined a circle with maybe 8 to 16 poles dividing the circle, each pole holding two cameras pointed in opposite directions tangential to the circle. At first I imagined that your turning would activate pairs of cameras. Then I realized that the cameras didn’t need to do any shifting, only your view. The cameras could be steady state, with software in your viewer selecting the proper pair of images. In fact, multiple people could share the array, looking hither and yon in whatever direction interested them.
More so, if the camera array was internet streamed, you would be able to look 360 degrees in full 3D perspective in real time! Imagine this mounted on a building high above a city. You’d be a giant hovering above a toy city, it’s poor citizens helpless to the destructive power of your merest foot-shuffle.
Questions that come up for me are:
- What stereo separation is sufficient? What is too much?
- 16 to 32 weatherproof cameras?
- Streaming video for 16 to 32 feeds?
- What software could be used to process the images?
- What software would work for pairing the cameras?
- How do you keep pigeons from fouling the cameras?
- Would this be too cool to handle?
