During World War II, the military discovered the true value of training pilots in flight simulators. In fact, studies showed a pilots chances of coming back from a mission jumped 95% if he had made it through the first 5 missions. Thus, anything that increased their chances of survival in those first few missions was critical to saving more pilots lives.
As early as 1929, the Link corporation was building flight simulators. A full-sized mock-up of a fighter cockpit was constructed and mounted on a motion platform. The cockpit physically pitched, rolled, and yawed based on the pilot’s actions. Results from the simple Link trainer showed that a pilot’s training could be successfully simulated on an earth-bound platform.
Decades later, as aviation became more complex, spending thousands of dollars on a flight simulator that saved millions of dollars in operating a real fighter or replacing one lost made sense. These economics were the driving force behind an entire industry committed to the realistic simulation of aeroplanes, helicopters, tanks, and even ships.
Early simulators had one major limitation. Real-time visual feedback was impossible. The pilot couldn’t view the outside world or watch it change by moving the control stick. Without visual feedback, training was limited to instrument flying. This all changed in the early 1950s, as commercial video cameras became available. Cameras were mounted on movable platforms suspended over scale models of airports. The motion of the camera platform was controlled by the simulator pilot. If the pilot pulled back on the control stick, the camera tilted back, projecting a new image on the pilot’s window. The camera “flew” over the model, controlled by the pilot.
This improvement in the perception of reality increased the simulation’s effectiveness. Soon, multiple cameras were combined to project additional perspectives, allowing left and right views along with a forward view for the first time. Over the next twenty years, video cameras continued to shrink in size as models grew in complexity. The advent of sophisticated computers capable of generating similar images slowly replaced the cameras and models.
While working on the first electronic head-mounted display, Ivan Sutherland recognized the potential for computers to generate the images for the flight simulators instead of video cameras and scale models. Sutherland teamed up with David Evans in 1968 to create electronic “scene generators.” Later that year, they formed Evans and Sutherland to sell these systems.
Instead of constructing scale models from paint, foam, and glue, the computer constructed images using stored data. Any 3-D object could be digitized and entered into the computer. For example, an airport could be represented as a large number of 3-D points with lines connecting each point. After a series of calculations, a view from any vantage point could be constructed.
Custom hardware-accelerated the calculations so that each scene was drawn in a fraction of a second, in less time than you can blink. This process of calculating and drawing images is called rendering. A rapid sequence of scenes rendered by the computer would appear movie-like to the pilot. Controlling the computer like a movable camera platform created the same sense of flying around the model. The number of scenes rendered in a fixed amount of time is called the update rate and is measured in frames per second (fps) or hertz (Hz). Early scene generators (circa 1973) achieved around 20 fps with simple 3-D models (200-400 polygons). Studies showed that pilot training performance suffered below this update rate.
One of the first flight simulators to use computers was built by General Electric Company‘s Electronics Laboratory for the Navy in 1972. Their Advanced Development Model was used as an instrument to measure the effectiveness of computer-generated images for pilot training. Consisting of three independent rendering channels, images were projected onto three screens that surrounded the physical mockup of the cockpit. This provided the pilot with a realistic 180-degree field of view.
This early simulator revealed many of the demands of simulating reality. The Navy discovered that the complexity of a scene (determined by the number of polygons) and texturing (each polygon is “painted” with an image) was crucial in providing adequate velocity and altitude cues for line ups and landings on both airfields and aircraft carriers. The Advanced Development Model also showed that simulation of haze significantly contributed to the realism of the experience. Generally, the more realistic the image, the better the training.
As early as 1979, the military was experimenting with head-mounted displays. If an effective one could be built, it could significantly reduce the expense and physical size of the simulation system. By projecting the image directly into the pilot’s eyes, bulky screens and projection systems could be eliminated.
One of the first of these, McDonnell Douglas‘s VITAL helmet, used an electromagnetic head tracker to sense where the pilot was looking. Dual monochromatic cathode ray tubes were mounted next to the pilot’s ears, projecting the image onto beam splitters in front of the pilot’s eyes.
This allowed the pilot to view and manipulate mechanical controls in the cockpit while seeing the computer-generated image of the outside world. Problems with bulky headgear and the unnaturalness of viewing through beam-splitters however limited the acceptance of these early head-mounted displays.
Without the initial investment by NASA and the Defense Department, many of these technologies would have taken much longer to evolve. It’s likely that virtual reality could not exist in its present form without this initial investment and big commercial companies like Facebook would never have put their money into it. It’s one of many valuable spin-offs of military research for a number of technologies, with the military currently fueling the growth of AR technology.
But it wasn’t just the military finding uses for this new form of technology, soon all pilots would be trained in much the same way.
