Microsoft is known for adding excitement to typically mundane product launches through captivating tech demonstrations, and the Windows 10 unveiling was no different. They used the platform to generate significant buzz around the HoloLens, a futuristic headset that offers a glimpse into the future of Augmented Reality (AR). However, Microsoft also has a history of notable hardware failures, reaching a peak during the Ballmer era. Does anyone remember the Kin phone? I certainly don’t.

The launch of HoloLens is unlikely to face a similar fate for several reasons. Firstly, the HoloLens is still in its early stages of development and has a considerable way to go before becoming commercially viable – this could take a few quarters or even a couple of years. Secondly, the concept itself is solid, capitalizing on promising emerging trends like wearable technology and Virtual Reality (VR) headsets. Although the HoloLens aims to differentiate itself by integrating various functionalities into a single device, in this Microsoft HoloLens review, we’ll examine existing and upcoming technologies in the market.

Given that this is an engineering blog intended for VR professionals and engineers, I won’t delve into defining “HoloLens” or explaining the difference between AR and VR. Augmented reality has diverse potential applications across industries, but limited use in entertainment. Virtual reality, conversely, is geared more towards entertainment with some professional applications.

Both technologies face significant limitations and numerous technical hurdles before achieving mass market adoption. This gradual process will span years rather than months. The technology required to create such products affordably simply isn’t ready yet, although it’s steadily progressing.

Let’s examine what’s currently available and what’s lacking.

Hardware Limitations - Google Glass vs. Oculus Rift vs. Microsoft HoloLens

Google Glass was announced in early 2012 and shipped a year later with a $1,500 price tag. This high cost restricted its reach to a small niche of early adopters, dubbed “explorers” by Google’s PR machine. The device offered limited AR functionality with a small prism projector (640x360 resolution) powered by an outdated processor.

Despite briefly captivating the public, Google Glass can hardly be considered a triumph. App developers initially eager to participate gradually lost interest, mirroring the enthusiasm of “explorers” who seemed to tire of the fad within months. Recent rumors suggest a new Google Glass iteration featuring Intel silicon, so an obituary might be premature. Regardless, Google Glass wasn’t a resounding success by any measure.

Currently, Oculus Rift is arguably the most discussed VR system. Unlike Google Glass, it’s yet to launch. Oculus VR has been developing the device for years, releasing two development kits. The consumer version, with revised specifications, is projected to launch sometime in 2015. Notably, Facebook acquired Oculus VR for over $2 billion in cash and stock in March 2014.

Samsung’s Gear VR takes a different approach, utilizing the Galaxy Note 4 phablet instead of a built-in screen, but incorporating some Oculus technology. The modular concept is intriguing, potentially allowing similar implementation across mobile devices from various vendors, effectively enabling users to upgrade hardware with each new phone. Qualcomm’s Vuforia platform boasts promising features for mobile devices and potential AR/VR applications.

So, what’s missing? While processing power might seem like the simple answer, it’s more nuanced.

Both concepts are ahead of their time, limited by current technology. Microsoft’s HoloLens is bound to face similar teething problems, but its distinct concept might allow it to overcome at least some of these challenges.

Google Glass, designed as a lightweight wearable, involved several compromises. Its single display on a thick prism in front of the user’s right eye offered limited resolution for its field of view (FOV). For instance, smartwatch displays with similar vertical resolutions occupy a significantly smaller portion of the user’s FOV. Furthermore, Google Glass relied on an outdated System on Chip (SoC) and suffered from limited battery life.

Designing mobile devices is a complex balancing act involving numerous trade-offs. Higher resolution displays demand greater GPU power, necessitating larger SoCs with more powerful GPUs operating at higher loads, consequently requiring larger batteries, and so on. AR headsets are simply too small to accommodate large batteries like those found in high-resolution tablets.

Oculus Rift, at first glance, doesn’t appear to face similar hardware limitations, as it’s not restricted by battery life or portability concerns. It doesn’t rely on an integrated SoC, and its 1080p display seems promising; however, it falls short of photorealism. The device has a large FOV, but its pixel density remains insufficient.

To achieve photorealism, VR devices would require higher resolution displays (4K/UHD or even 8K in the future). While this technology is nearing feasibility, it’s expensive and far from portable.

Running the latest AAA games on a 4K display with maximum detail settings necessitates two high-end discrete graphics cards, such as Nvidia and AMD’s flagship Maxwell and Hawaii generation GPUs. Eliminating frame tearing (using technologies like Nvidia G-Sync or AMD FreeSync) requires even more power, and true 3D rendering for both eyes demands yet more GPU power.

Essentially, powering a 4K VR device with current technology would require at least two GPUs with a combined 12-14 billion transistors (28nm), consuming 350W to 500W, excluding the CPU and other system components. This is a conservative estimate based on currently available GPUs and CPUs, and we haven’t even touched upon powering two 4K screens (one per eye).

Nvidia’s latest mobile SoC, the Tegra K1 64-bit used in the Google Nexus 9, features 192 CUDA cores based on the Kepler architecture, not the more efficient Maxwell architecture. In contrast, their current flagship discrete graphics cards boast 2048 Maxwell CUDA cores operating at higher clock speeds than the Kepler cores in mobile Tegra SoCs.

Clearly, portable VR devices with photorealistic graphics are years away, and even wired devices like Oculus Rift have a long way to go. The overall platform cost is another concern. Gaming PCs capable of delivering playable frame rates at 1080p are relatively affordable due to the capabilities of mainstream GPUs. However, 2160p rendering requires four times the GPU power, along with more memory and a faster CPU.

There’s an alternative solution to this problem, which I’ll discuss later.

So What Did Microsoft Get Right?

Remember the Facebook-Oculus Rift deal mentioned earlier? A few days after its announcement, it was revealed that Microsoft had acquired intellectual property (IP) assets related to augmented reality and wearable computers from Osterhout Design Group (ODG). Some patents covered “see-through near-eye display glasses” with a partially transmitting optical element.

In essence, Microsoft acquired the IP necessary to develop HoloLens. The deal reportedly encompassed dozens of ODG patents, including several pending patent applications. In contrast, Oculus VR reportedly holds only a single patent vaguely describing “a virtual reality headset.”

Microsoft seems to be aiming for the best of both worlds: a wide FOV typically associated with VR devices and a transparent display surface suitable for AR applications. This approach could enable HoloLens to utilize significantly less processing power than VR devices while offering greater functionality due to its wide FOV. Instead of striving for photorealistic rendering, HoloLens could leverage a slightly lower resolution and image quality due to the limited opacity of its displayed content. Since it doesn’t need to create a complete illusion of reality, there’s less hardware overhead. Existing technology could enable HoloLens to minimize or eliminate aliasing and generate visually appealing composites by leveraging the real-world backdrop.

While this limits HoloLens’ entertainment appeal compared to true VR headsets, it unlocks numerous possibilities in sectors like engineering, healthcare, architecture, and defense. HoloLens could assist healthcare professionals, engineers, industrial machinery operators, soldiers, and law enforcement personnel.

However, HoloLens still holds potential for consumer applications. Microsoft’s Phil Spencer stated that HoloLens needs to succeed as a standalone product, adding that they’re exploring its use with PCs and Xbox One consoles. The device could function as a heads-up display (HUD) for gamers or even for fitness enthusiasts in gyms.

HoloLens Hardware Conundrum

Microsoft hasn’t disclosed the exact hardware specifications, leaving much to speculation. There’s no information about display resolution, GPU GFLOPs, connectivity, or battery life. While this fuels speculation in the tech press, nothing is official yet.

As mentioned earlier, HoloLens shouldn’t require as much GPU power as Oculus Rift or similar VR devices. However, this doesn’t mean Microsoft can rely on a cheap SoC like those commonly found in mobile devices. Microsoft utilizes various chips from different vendors – Qualcomm Snapdragon SoCs with integrated 4G/LTE for mobile phones, Intel chips for Surface Pro tablets (alongside Nvidia SoCs in the discontinued Surface RT line), and custom AMD APUs in the Xbox One.

Power considerations make a Snapdragon SoC, similar to those in Lumia phones, the most likely choice. This doesn’t imply HoloLens would be as underpowered as Google Glass. As a larger device, HoloLens has room for a larger battery, and the latest Snapdragon SoCs are considerably more powerful than the Google Glass chipset (which lags behind even those in smartwatches). Early benchmarks suggest that the Adreno 430 GPU in upcoming flagship SoCs like the Snapdragon 810 is potent enough to handle 4K resolutions and render relatively complex 3D content in 1080p.

But it’s not just about raw rendering performance. GPUs offer significant computing potential beyond gaming. Google utilized the Tegra K1 for Project Tango, which also encompasses technologies relevant to AR/VR devices – automation, driverless cars, etc. While other players exist in the GPU industry, Nvidia, with its CUDA cores and market leadership in professional graphics and GPGPU computing, has a distinct advantage.

However, we shouldn’t limit ourselves to “what’s currently available.” HoloLens won’t launch for a while, and subsequent generations will inevitably feature even more powerful hardware. Intel’s 14nm Atoms are on the horizon, with ARM-based 14nm and 16nm SoCs expected a couple of quarters later. These new non-planar nodes will enable greater performance per watt, significantly enhancing performance without sacrificing battery life.

Streaming As An Alternative

As mentioned earlier, cloud computing and streaming present an alternative solution for displaying complex, resource-intensive 3D content. Latest-generation SoCs feature 802.11ac wireless and fast LTE modems, enabling high-resolution streaming. However, lag, particularly with LTE, is a drawback.

Lag could be minimized if additional content is rendered locally on a PC workstation or even an Xbox One. However, remote cloud rendering could be problematic. For instance, Nvidia is addressing this challenge by establishing GRID servers in strategic locations to provide low-lag game streaming to major markets. Even a few milliseconds of added lag could significantly impact the user experience in AR applications.

While a mobile SoC should suffice for most everyday tasks (Skype, basic AR applications, etc.), more demanding scenarios would require additional hardware. For instance, if an architect wants to visualize a finished building using augmented reality on a construction site, HoloLens would need more power to render complex scenes with numerous polygons, advanced lighting effects, etc.

The advantage is that HoloLens could offer extensive out-of-the-box functionality with a relatively powerful integrated GPU capable of handling everyday tasks like high-resolution video streaming, browsing, and even casual gaming. Professionals could then utilize 802.11ac or LTE to stream more complex content rendered remotely.

Microsoft could essentially use the same hardware platform for both consumer and professional markets, with professionals leveraging local or cloud streaming for demanding tasks.

Is There A Use Case And A Market For HoloLens?

Microsoft showcased HoloLens in various scenarios. While the demos were intriguing, they didn’t clearly define a realistic and commercially viable use case for the device.

What’s appealing about HoloLens is its position between true wearables like Google Glass and tethered VR solutions like Oculus Rift. It doesn’t need to be as lightweight and portable as smart glasses, nor does it require a constant connection to a computer or power source – essentially, it represents the best of both worlds. Microsoft’s decision to lead rather than follow is also commendable. HoloLens is innovative, futuristic, and original, offering a refreshing change of pace from Redmond.

However, this approach raises important questions about its use case and target market. It can’t replace displays like VR solutions, and its bulk and appearance make it impractical for everyday use. While you might encounter commuters or athletes using smart glasses, you’re unlikely to see skiers or joggers sporting HoloLens headsets.

What can mainstream users do with HoloLens? What software platforms and operating systems will it support? What about professional applications? What about cross-platform functionality, hardware specifications, retail price, and Bill of Material (BOM)?

Many questions remain unanswered, and Microsoft will likely take time to disclose all the details.

Microsoft will need to target both mainstream and professional markets with the same hardware. Depending on the price and BOM, they could leverage their Xbox user base and a segment of the PC gaming market to introduce HoloLens to mainstream consumers. Marketing such a product at a high price point would be challenging, but the user base exists and is willing to spend on new gadgets. A mainstream market approach could also attract more developers, expanding the ecosystem and creating new use cases.

But if HoloLens is priced for the mainstream market, how will Microsoft penetrate the professional market and generate revenue?

Having worked in offline 3D graphics, I see immense potential in HoloLens for 3D/CAD users. But does this mean every designer will purchase a mainstream-priced HoloLens for work? Possibly, but unlikely.

There are other ways to market products in this space. Drawing on my years of experience covering the GPU industry, I’ve learned a thing or two about its workings. While high-end gaming graphics cards grab headlines, professional graphics and compute solutions are the true cash cows for Nvidia and AMD. They are the unsung heroes in this duopoly. The BOM for consumer and professional cards based on the same GPU is roughly equivalent, but professional cards are significantly more expensive, generating higher margins and revenue despite lower sales volumes – check any Nvidia quarterly earnings report for confirmation.

Microsoft could adopt a similar strategy. They could use the same hardware for both consumer and professional HoloLens versions, limiting functionality in consumer models and expanding it in professional models through different licensing tiers.

Of course, this is all speculation at this point – but that’s how this market operates. Microsoft doesn’t need to reinvent the wheel.