Apple wins patent for a scene camera system for a mixed reality headset that includes a 2-dimensional camera array

Apple wins patent for a scene camera system for a mixed reality headset that includes a 2-dimensional camera array

Today, the US Patent and Trademark Office officially granted Apple a patent for a new HMD camera system comprising a two-dimensional array of cameras that capture images of respective parts of a scene.

Today, the US Patent and Trademark Office officially granted Apple a patent for a new HMD camera system comprising a two-dimensional array of cameras that capture images of respective parts of a scene.

The cameras are positioned along a spherical surface so that the cameras have adjacent fields of view. The entrance pupils of the cameras are positioned at or near the user’s eye while the cameras also form optimized images at the sensor. Methods for reducing the number of cameras in an array, as well as methods for reducing the number of pixels read from the array and processed by the pipeline, are also described.

While today’s patent report might be appreciated by all Apple fans, the patent will be especially appreciated by engineers, optical engineers, and technicians who follow all things VR headsets.

Virtual reality system simulated environments and/or mixed reality system mixed environments can be used to provide an interactive user experience for multiple applications, such as applications that add virtual content to a real-time view of the viewer environment, applications that generate 3D virtual worlds (Metaverse), interact with virtual training environments, play games, remotely control drones or other mechanical systems, view digital media content, interact with the Internet, explore landscapes or virtual environments, etc.

Apple’s patent covers stage cameras for transparent head-mounted displays (HMDs) that can be used in mixed reality (MR) or virtual reality (VR) systems.

In conventional HMDs, one or more scene cameras can be mounted on the front of the HMD. However, the point of view (POV) of the scene cameras is noticeably offset and different from the POV of the user’s eyes. Apple’s invention corrects the point of view (POV) of the cameras to match the user’s POV by moving the cameras’ entrance pupils towards the user’s eyes.

In some embodiments, an HMD includes two-dimensional arrays of small form factor cameras (e.g., one left-eye array and a second right-eye array) that capture images of respective portions of a real world scene in front of the user. The cameras are positioned along a curve or spherical surface so that the cameras have adjacent non-overlapping fields of view (FOV).

To obtain a more accurate representation of the user’s perspective, the camera optics are configured so that the entrance pupils of the cameras in the array are positioned behind the camera image planes formed at the sensors. image, and at or near the user. the eye while the cameras also form images optimized at the sensor level. In this way, each camera array captures views of the scene from substantially the same perspective as the user’s respective eye.

In some embodiments, one sensor may be used to capture images for multiple (eg, four) cameras in the network. In these embodiments, the optics of cameras used with a sensor can be shifted or adjusted to align the image planes with the sensor surface.

In some embodiments, a curved sensor and simpler lens system can be used to provide a wider FOV for each camera and thus reduce the number of cameras in the array.

Apple’s patent FIG. 1A below illustrates a head-mounted display (HMD) that includes an array of cameras with entrance pupils at or near the user’s eye; FIG. 2 illustrates part of a network of cameras.

Apple’s patent FIG. 3 below illustrates an example of a camera with an entrance pupil at or near the user’s eye that can be used in a network as shown in Figs. 1A and 2.

(Click on the image below to enlarge)

3 pics.  3 hmd camera patent

Apple figures. Figures 9A and 9B below illustrate the reduction in the number of cameras by reducing the resolution and shifting the entrance pupil of the cameras in the peripheral regions.

4 apple hmd camera patents figs 9a b 16 17

Apple’s patent FIG. 16 above graphically illustrates the addition of a negative meniscus lens in front of a camera array to increase the cameras field of view (FOV) to the periphery; FIG. 17 graphically illustrates the addition of a negative meniscus lens in front of a camera array to gradually shift the point of view (POV) of the cameras towards the periphery.

Apple’s patent FIGS. Figures 20A-20C below graphically illustrate an example scene camera that includes a negative meniscus lens in front of the cameras in the camera array.

5 figs patent apple 20a bc camera system hmd

For details, see Apple granted patent 11,448,886.

Some Apple Inventors

Dan Hennigan: Optical Mechanical Engineering Manager

Saito Kenichi: Optical Engineer (previously worked at Cannon & Fuji Photo Film)

Bret Miller: Engineering Manager, Camera Incubation

Noah Bedard: Prototyping engineer

Catherine Berkner: Senior Engineering Manager – Camera Incubation

10.52FX - Granted Patent Bar


#Apple #wins #patent #scene #camera #system #mixed #reality #headset #includes #2dimensional #camera #array

Leave a Comment

Your email address will not be published.