Scientific Analysis of Lucyd Patents by Lucyd Optics Lead, Dr. Kayat

Lucyd Lens:  next generation AR glasses with pioneering optics patent portfolio

By Michael Kayat, Ph. D.
Optics Lead, Lucyd

Augmented reality is moving to mainstream

Augmented reality (AR) devices and applications are now poised for substantial growth, according to a recent 2017 Gartner report¹.  Following the usual hype cycle, AR technology platforms are moving toward mainstream adoption in the next few years to potentially create a multi-billion dollar market. One of the key drivers will be the availability of reasonably priced, hands-free AR devices in the form of lightweight, stylistic “smart glasses” (or AR headsets) which have the potential to replace smart phones, smart watches and other wearables like fitness trackers.

IDC² has also forecasted that AR headsets may reach 25 million units by 2021, with over 80% for commercial use for field and enterprise applications, over four times the consumer market segment.   Near eye AR wearers can see real world scenes and objects overlaid with non-occluding virtual digital content.  Commercial uses include collaborative virtual product R&D and testing, guided technical maintenance & training, medical simulation & training and guided procedures, together with applications in many other industrial segments like transportation.  Of course consumer uses will be dominated by gaming, together with other forms of individual and group entertainment.  Advertising, tourism, shopping, sports and many new AR applications will open up.

As with any new emerging market, there are many first generation AR headset products now being launched with a diverse range of technical features and capabilities^3. These products have different sensors, optics, communications, human interfaces and applications, together with different price points.  The size and weight of these AR glasses are important to mainstream users.  The market leaders will need to produce AR eyewear formats that are small, lightweight with minimal optical distortions.  Advanced features like eye tracking and the ability to use traditional lenses in a range of styles and high resolution images will be highly desirable.

Optical issues with current AR glasses need to be resolved

Optical designs directly affect the form, size, weight, cost and overall capabilities of AR headsets. Optical quality and design are key differentiators between commercially available AR headsets.   An AR headset should have a large enough field of view, close to the typical human’s 180°, so that virtual content can be placed anywhere in the wearer’s line of sight.

There are several problems for current AR headset products.  These include bulky size and geeky design requiring custom lenses, narrow field of views (typically less than 20° to 40°-50°), limited spatial resolutions that cause blurriness, motion latency and restricted vision.  Other problems are light leakage and stray light effects due to mismatches between virtual image generators and optical combiners, caused by rigidly symmetric geometric designs. Optical aberrations and offset occlusion effects between real views and virtual objects giving depth perception and accuracy issues, round out the big problems.

The main optical design tradeoffs are between field of view, resolution, and eye box size.  The secret to success is for a manufacturer to be able to optimize these parameters in a range of different style forms, minimizing field distortion effects (blur/warping from astigmatism and coma).  Minimizing the number of optical elements that are required will reduce weight, power requirements and lower costs.

Novel technologies embodied in patent portfolio for next generation AR glasses

Based on technologies embodied in a portfolio of patents exclusively licensed from the University of Central Florida UCF), Lucyd is planning to develop a next generation AR headset that largely overcomes existing optical issues.  These patents describe novel technologies for head mounted displays (HMDs) and were developed over a period of 10 years at UCF’s world-class optics facility, the Center for Research and Education in Optics & Lasers (CREOL)^4.  Professor Jannick Rolland is the principal researcher and inventor of the optics design and underlying methodology that has been embodied in the patent portfolio.

The Lucyd patent portfolio embody optical designs for HMDs for AR applications based on a reflective curved mirror combiner in a pupil forming architecture with dual micro-projectors.  A phase conjugate material is coated on to the primary mirror to enable high efficiency retro-reflecting see through stereoscopic performance, as shown in Figure 1.

                                  Fig. 1.  Conceptual design of projection-based HMD⁵

Prototype HMDs have been developed at UCF consisting of a pair of micro-projection lenses (one per eye), beam splitters, and miniature displays were integrated with phase-conjugate nanomaterial into complete see-through embodiments^5-11.  The phase conjugate material used comprised micro-corner cube arrays for total internal reflections.  UCF demonstrated that these novel configurations enable 3D visualization capabilities with large field of views (up to and wider than 120°), lightweight optics (6 -8 g per eye), large eye box sizes 8 to 10 mm, and low stereoscopic optical distortions (less than 1.5%) so eliminating any subsequent corrections. A key property of retro-reflective material is that the light rays are incident on the surface at any angle and are reflected onto the same incident optical path.  Then the image perception is independent of the shape and location of the screen, so these can be curved or tilted with no apparent distortion. This effect enables flexible designs in the shape (form) of the headsets.

The Lucyd patent portfolio has several unique novel technology embodiments. Optics were miniaturized to develop compact, low weight projection HMDs. Eye tracking using illumination optics was integrated into the HMD optics.  Occlusion support was implemented in HMDs where virtual objects occlude real objects, real objects occlude virtual objects.  Methods for freeform surfaces for off-axis asymmetric designs in HMDs were described and developed.  High resolution micro-displays (SXGA 1280×1024 and higher) were integrated with low light loss retro-reflective surfaces and high spatial resolution (~ 2’/pixel) were achieved.  Compact projection optics were developed to give wide field of views for projection HMDs (~ 120° and potentially wider) and optimized field of view for small eyeglass format HMDs (~ 25°).

To facilitate flexible HMD configurations and also take advantage of diamond turning technology, the UCF team developed a free form optics design methodology for:  (1) optimizing field of view, pupil size, resolution and depth of field; (2) creating non-rotationally symmetric, off axis designs for small form factors: (3) minimizing field distortions such as blur/warping from astigmatism & coma; and (4) minimizing the number of optical elements in an HMD design.  Using low light loss retroreflective materials gave an added benefit of being able to use lower brightness micro-displays.

The patent portfolio comprises four key optical technology areas.  The first area is the design of head-mounted projection displays^12-17, the second area describes eye tracking ^18-19, the third area describes eye-glass formats^20-21 and the fourth area describes the free-form compact optics^22-24.

Lucyd Lens has the potential to represent the next-generation of AR headsets that will overcome some of the issues inherent in many of the current products on the market today.

References

1. “Gartner Identifies Three Megatrends That Will Drive Digital Business Into the Next Decade”. Gartner, August 15, 2017 https://www.gartner.com/newsroom/id/3784363

2. “Worldwide Quarterly Augmented and Virtual Reality Headset Tracker”. IDC, June 19, 2017 https://www.idc.com/tracker/showproductinfo.jsp?prod_id=1501

3. Ron Padzensky, “The Definitive Guide to Augmented Reality Glasses”, AugmentedReality.org, 2017 http://www.arglassesbuyersguide.com/

4. Rolland and O. Cakmakci, “Head-worn displays: The future through new eyes”, Optics and Photonics News (OPN), April issue 2009

5. Martins and J. Rolland, “Diffraction of Phase Conjugate Material in a New HMD Architecture”, Proceedings of SPIE Vol. 5079 (2003)

6. Martins et al. “A mobile head-worn projection display,” Opt. Express 15, 14530-8 (2007).

7. Cakmakci et al., “Optimal local shape description for rotationally non-symmetric optical surface design and analysis”, Optics Express 16(3), 1583-1589 (2008)

8. Cakmakci, et al., “Design and assembly of a head-worn display”, PC Magazine (2007)

9. Cakmakci and J. Rolland, “Head-Worn Displays: A Review”, J. Display Tech. Vol. 2, No. 3, Sept. (2006)

10. Rolland and H. Hua, “Head-mounted display systems,” in the Encyclopedia of Optical Engineering, R.G. Driggers, ed., Taylor & Francis (2003).

11. Cakmakci et al, “Design of a free-form single-element see-through head-worn display”, Proceedings of the SPIE 7618-02 (2010)

12. Hua and J. Rolland, “Compact lens assembly for the teleportal augmented reality system,” US Patent 6,731,434 (2004).

13. Ha and J Rolland, “Compact lens assembly for the teleportal augmented reality system,” US Patent 6,804,066 (2004).

14. Martins, J. Rolland and Y. Ha, “Head-mounted display by integration of phase-conjugate material,” US Patent 6,963,454 (2005).

15. Smith, M. Shah and N. Lobo, “Algorithm for monitoring head anfd eye motion for driver alertness with one camera”, US Patent  6,927,694 (2005).

16. Zou and J. Rolland, “Iterative least-squares wavefront estimation for general pupil shapes” US Patent 7,008,457 (2006).

17. Rolland, Y. Ha and L. Davis, “Head mounted projection display with a wide field of view” US Patent 7,119,965 (2006).

18. Smith, M. Shah and N. Lobo, “Algorithm for monitoring head anfd eye motion for driver alertness with one camera”, US Patent  6,927,694.

19. Curatu and J. Rolland, “Projection-based head-mounted display with eye-tracking capabilities” US patent 7,522,344 (2009).

20. Cakmakci and J. Rolland, “Imaging systems for eyeglass-based display devices,” US Patent 7,499,217 (2009).

21. Cakmakci and J. Rolland, “Imaging systems for eyeglass-based display devices. US Patent 7,969,657 (2011)

22. Chaoulov, R. Martins and J. Rolland, “Compact microlenslet arrays imager” US Patent 7,009,773 (2006).

23. Ha, J. Rolland and O. Cakmakci, “ Compact optical see-through head worn display with occlusion support,” US Patent 7,639,208 (2009).

24. Shaoulov, J. Rolland and Y. Ma, “Systems and methods for providing compact illumination in head mounted displays, US Patent 7,843,642 (2010).