Microsoft HoloLens 2 has been revealed, a smaller, more powerful version of the augmented reality headset, complete with a much larger field-of-view. The company also unveiled a new cloud-connected camera, the Azure Kinect, which could power checkout-less stores, production lines, and more.
All the same, impressive as HoloLens is, the first-generation hardware wasn't without its flaws. The size and weight is an obvious drawback, as are the limits of battery life. Most notable, though, is the limited field-of-view of those transparent eyepieces. Turn your head a little, and it's all too easy to lose sight of the virtual objects.
For the immersion, Microsoft developed its own MEMS displays, with more than twice the field of view of the first-generation headset. In fact, Kipman says, it's the equivalent of going from 720p to 2K resolution, for each eye. That's while still keeping the 47 pixels per degree resolution for each eye, too.
In addition to an Intel Cherry Trail SoC containing the CPU and GPU, HoloLens features a custom-made Microsoft Holographic Processing Unit (HPU), a coprocessor manufactured specifically for the HoloLens by Microsoft. The SoC and the HPU each have 1GB LPDDR3 and share 8MB SRAM, with the SoC also controlling 64GB eMMC and running the Windows 10 operating system. The HPU uses 28 custom DSPs from Tensilica to process and integrate data from the sensors, as well as handling tasks such as spatial mapping, gesture recognition, and voice and speech recognition. According to Alex Kipman, the HPU processes "terabytes of information". One attendee estimated that the display field of view of the demonstration units was 30°×17.5°. In an interview at the 2015 Electronic Entertainment Expo in June, Microsoft Vice-President of Next-Gen Experiences, Kudo Tsunoda, indicated that the field of view is unlikely to be significantly different on release of the current version.
We got a taste of direct manipulation with Magic Leap (especially the Tónandi experience), which has a larger field of view than the original HoloLens. But most of the applications are still using the controller instead of direct manipulation.
In addition to enhanced comfort, the Hololens 2 has gained new features. The headset uses windows hello to scan your retina for authentication purposes. For making the experience more immersive, the team were able to more than double the field of view while retaining the holographic density of 47 pixels per degree. That translates to a resolution of 2K for each of the eyes. With improved spatial and depth mapping, Microsoft is able to detect and recognize your hands. You will now be able to interact with holograms by directly manipulating them.
DESCRIPTION: It has become imperative that the Army develop alternative capabilities to communicate with reduced electro-magnetic footprint, while assuring low probability to detect and low probability of incept (LPD/LPI) capability and supporting necessary bandwidth for modern battlefield operations. The Free-Space Optical (FSO) communication concept provides an alternative pathway for inherent LPD/LPI communications, while providing significant bandwidth and low electro-magnetic (radio frequency)emissions. One of the inhibiting factors preventing widespread use of traditional FSO communication systems based on macro-scale optics can be linked to their size, weight, complexity and overall cost per link. An ultra-low SWAP-c, FSO communication system could provide accessibility of this technology geared toward the Army need for ensured communication while on the move and at the lowest echelon. Challenges associated with accomplishing this goal are many-fold and will require modern-day automated photonics technology manufacturing to achieve the long-term goal of a low cost while overcoming specific issues associated with pointing-and-tracking (PAT), transmitter beam divergence, receiving aperture size limitations, and low signal detection at GHz-level speeds. Given these challenges is it envisioned that one of the few solutions would be derived from modern integrated photonics technology ARL is seeking a small business to demonstrate an ultra-compact FSO communication system. This demonstration should be capable of high bandwidth (Gb-level), low bit error rate (BER)(10-6), automatic PAT in an extremely compact form factor (< 100 cm3, 100s of grams, < 10 Watts of power consumption). There have been several rudimentary demonstrations of one necessary aspect for this program (e.g. wide-field of view beam steering) in a chip-scale form factor [1-2]. These systems might have the potential to address the SWAP-c requirements due to inherent size and long-term high volume fabrication pathway. Although these systems are interesting, none have demonstrated FSO communication functionality and there remains many impediments to embodiment of a full communication system that needs innovative and applied research and development to overcome. These ultra-compact FSO systems must overcome technical hurdles which the macro-scale system have done in past, including: aperture limits, low-signal detection at high data rate, full implementation of PAT with required field of view (FOV) and slew rates and finally low power consumption.
PHASE II: Demonstrate an initial prototype Free-Space Optical (FSO) communication system with data bandwidth of 1 Gbps, automatic pointing-and-tracking (PAT) function with 30 degree field of view (FOV) and maximum FOV slew time of 500 microseconds, bit error rate (BER) of 10-6 over 1-hour interval at 90% network capacity at an outdoor-range exceeding 200 meters in a modem form factor of 100 cm3 or less, weighing less than 400 grams and consuming 10 Watts or less power. Technology should be at the level of TRL 4/5 at the end of this phase with a dedicated plan toward fabrication scaling for reduced unit cost.
PHASE III: Advance prototype Free-Space Optical (FSO) communication system to TRL 7/8 with data bandwidth of 2 Gbps, automatic pointing-and-tracking (PAT) function with 45 degree field of view (FOV) and maximum FOV slew time of 500 microseconds, BER of 10-6 over 1-hour interval at 90% network capacity at an outdoor-range 1000 meters in a modem form factor of 50 cm3 or less, weighing less than 200 grams and consuming 5 Watts or less. It is envisioned that this technology will enable near range dispersal of secure FSOC network for US tactical ground forces, which could also provide a dual-use commercial application pathway for local area networks in highly congested urban environments. Similarly a system of this type and capabilities would greatly reduce the cost of setting up urban Local Area Networks in developing areas. Applications of FSO communication system have direct pathway to transition through existing Army development investments currently underway for the alternative communication space. Finally, it is expected that the core of this technology will mature aspects of beam steering and integrated receivers, which could have direct dual-use in low SWAP-c laser ranging (LADAR) application for military and civilian use on autonomous platforms
PHASE II: Produce planar optical elements on appropriate substrate. Demonstrate diffraction-limited or near diffraction-limited optical performance across a large waveband with high-efficiency at a low f-number. Demonstrate ability to maintain performance across a 30 to 50 degree horizontal field of view. Subject planar optical elements to necessary environmental tests, such as temperature, to show applicability to military systems. Demonstrate path to manufacturing planar optical elements at production quantities. Deliver prototype lenses to C5ISR Center NVESD for further testing and application.
PHASE II: The performer shall further develop algorithms that detect, recognize, and identify personnel that pose a threat. These algorithms shall be applied to additional scenario data collected by the performer and Government. The new scenarios shall have greater complexity, occlusions, and clutter. These scenarios should include realistic urban scenes that include urban objects and street level activities typical of this environment, e.g., unarmed civilians and commercial vehicles. The rural environments will assume a larger field of view with fewer pixels on target; implicit in this environment is vegetation ranging in scale from grass and shrubs up through forests. In both scenarios, the scenes shall include static and dynamic clutter that represent bystander human and non-human activity. The performer shall quantify detection results in terms of detection probability and false positive probability and confusion matrices.
DESCRIPTION: Artificial Intelligence has yet to surpass the human brain in terms of training time: even the best algorithms require huge datasets that train carefully-tuned models over a long period of time. Current state-of-the-art artificial neural networks for image identification, called Convolutional Neural Networks (CNNs), are achieving at, or higher, than human level performance in recognizing 2D images from the open source ImageNet database ( -net.org/) of labeled images (plant, animals, etc.) which benchmarks performance. CNN performance in ImageNet data for standard color photos, greyscale, and photos based on textures (i.e.; elephant skin) are on par or slightly better than human performance. CNN performance degrades substantially with images of object silhouettes (black object with white background) and edges (image features represented with only lines), when objects under observation are small in scale relative to surrounding area, and when object viewpoint, rotation, size, and illumination vary. CNN training on ImageNet data requires on the order of 1000 examples per object class yet humans need to see a new object only once or twice and it becomes instantly recognized at a later time. We are seeking brain-inspired artificial neural network algorithms that can meet the performance objectives of recognizing objects in images from less than 10 training examples with 90% confidence of object identification under a full range of image observation conditions to include varying scale, size, illumination (full sunlight to low light), occlusion (from zero to 90% in both height/width increments of 15%), and rotation (in increments of 30°). A virtual 3D environment to train and demonstrate viability of the proposed algorithm is desired such as the Unity open source game engine ( -personal?_ga=2.145300465.433590269.1559218374-1672088338.1559218374). It is desired that artificial neural network algorithm be developed with open source development code such as TensorFlow ( ) or Python ( ). It is desired to have a high resolution color video camera ( ) with a minimum of 3840 x 2160 pixels be used to observe raw pixels from the 3D virtual environment to train and demonstrate feasibility and performance of the algorithm. Novel approaches to train for object recognition that realistically emulate the human vision system (e.g., stereopsis, foveation, etc.) are desired if a breakthrough in capability is feasible. 2b1af7f3a8