Everything but the kitchen sink

Using a single SoC to drive a full-blown infotainment stack, 3D surround view, driver monitoring, smartphone connectivity, and dual HD displays.

TI and QNX have been working together in the infotainment space for a long time. The nice thing about this ongoing relationship is that lots of cool technology gets built along the way. Speaking of which, TI have put together a compelling demo that they will show at the TU-Automotive Detroit conference, on June 8 and 9. I’m pretty pumped about it and invite you to take the time to check it out.

The demo is built on the DRA75x (Jacinto 6 EP) SoC, which includes dual ARM Cortex-A15 processors, an Imagination SGX544MP2 GPU, dual TI C66x DSPs, and an IVA-HD video processing core. For starters, it runs the QNX CAR Platform for Infotainment with navigation, multimedia, speech recognition — all the goodies you’ve come to expect. Not surprising, as the platform has been running on Jacinto 6 longer than on any other SoC.

One SoC, two displays, many applications

It’s what they’ve managed to pile on beyond the QNX CAR Platform that makes this demo so exciting. You may not know it but we also work closely with TI on the informational ADAS (infoADAS) front. There’s a full port of the InfoADAS stack available today on the QNX platform and it’s included in the demo. Taking 4 camera inputs and the processing power available on the C66x and SGX, TI can demonstrate full 3D surround view concurrently with everything else. If that wasn’t enough, they’ve added a fifth camera and partnered with FotoNation to add driver monitoring and identification, which runs on the second C66x.

Normally, the smartphone projection runs on ARM, but for this demo, it runs on the IVA-HD to further demonstrate the capabilities of the chip. And to top it all off, the demo drives two HD displays. One display shows the QNX CAR Platform and the other shows the 3D surround view, along with the driver monitoring and identification.

So, to summarize, on one dual-core ARM A15 part, TI is showing a full-blown high-end infotainment system, driver monitoring, the ability to see everything around the car in real time, and the ability to connect to pretty much any smartphone in the world. Take a second to think back to just 5 years ago. It’s amazing how fast this industry moves.

If you aren’t going to be at TU this year, reach out to TI. I’m willing to bet they’d be happy to show it to you…

Now with ADAS: The revamped QNX reference vehicle

Tina Jeffrey

Since 2012, our Jeep has showcased what QNX technology can do out of the box. We decided it was time to up the ante...

I walked into the QNX garage a few weeks ago and did a double take. The QNX reference vehicle, a modified Jeep Wrangler, had undergone a major overhaul both inside and out — and just in time for 2015 CES.

Before I get into the how and why of the Jeep’s metamorphosis, here’s a glimpse of its newly refreshed exterior. Orange is the new gray!



The Jeep debuted in June 2012 at Telematics Detroit. Its purpose: to show how customers can use off-the-shelf QNX products, like the QNX CAR Platform for Infotainment and QNX OS, to build a wide range of custom infotainment systems and instrument clusters, using a single code base.

From day one, the Jeep has been a real workhorse, making appearances at numerous events to showcase the latest HMI, navigation, speech recognition, multimedia, and handsfree acoustics technologies, not to mention embedded apps for parking, internet radio streaming, weather, and smartphone connectivity. The Jeep has performed dependably time and time again, and now, in an era where automotive safety is top of mind, we’ve decided to up the ante and add leading-edge ADAS technology built on the QNX OS.

After all, what sets the QNX OS apart is its proven track record in safety-certified systems across market segments — industrial, medical, and automotive. In fact, the QNX OS for Automotive Safety is certified to the highest level of automotive functional safety: ISO 26262, ASIL D. Using a pre-certified OS component is key to the overall integrity of an automotive system and makes system certification much easier.

The ultimate (virtual) driving experience
How better to showcase ADAS in the Jeep, than by a virtual drive? At CES, a 12-foot video screen in front of the Jeep plays a pre-recorded driving scene, while the onboard ADAS system analyzes the scene to detect lane markers, speed signs, and preceding vehicles, and to warn of unintentional lane departures, excessive speed, and imminent crashes with vehicles on the road ahead. Onboard computer vision algorithms from Itseez process the image frames in real time to perform these functions simultaneously.

Here’s a scene from the virtual drive, in which the ADAS system is tracking lane markings and has detected a speed-limit sign:



If the vehicle begins to drift outside a lane, the steering wheel provides haptic feedback and the cluster displays a warning:



The ADAS system includes Elektrobit EB Assist eHorizon, which uses map data with curve-speed information to provide warnings and recommendations, such as reducing your speed to navigate an upcoming curve:



The Jeep also has a LiDAR system from Phantom Intelligence (formerly Aerostar) to detect obstacles on the road ahead. The cluster displays warnings from this system, as well as warnings from the vision-based collision-detection feature. For example:



POSTSCRIPT:
Here’s a short video of the virtual drive, taken at CES by Brandon Lewis of Embedded Computing Design, in which you can see curve-speed warnings and lane-departure warnings:



Fast-boot camera
Rounding out the ADAS features is a rear-view camera demo that can cold boot in 0.8 seconds on a Texas Instruments Jacinto 6 processor. As you may recall, NHTSA has mandated that, by May 2018, most new vehicles must have rear-view technology that can display a 10-by-20 foot area directly behind the vehicle; moreover, the display must appear no more than 2 seconds after the driver throws the vehicle into reverse. Backup camera and other fastboot requirements such as time-to-last-mode audio, time-to-HMI visible, and time-to-fully-responsive HMI are critically important to automakers. Be sure to check out the demo — but don’t blink or you’ll miss it!

Full-featured infotainment
The head unit includes a full-featured infotainment system based on the QNX CAR Platform for Infotainment and provides information such as weather, current song, and turn-by-turn directions to the instrument cluster, where they’re easier for the driver to see.



Infotainment features include:

Qt-based HMI — Can integrate other HMI technologies, including Elektrobit EB Guide and Crank Storyboard.

Natural language processing (NLP) — Uses Nuance’s Vocon Hybrid solution in concert with the QNX NLP technology for natural interaction with infotainment functions. For instance, if you ask “Will I need a jacket later today?”, the Weather Network app will launch and provide the forecast.

EB street director — Provides embedded navigation with a 3D map engine; the map is synched up with the virtual drive during the demo.

QNX CAR Platform multimedia engine — An automotive-hardened solution that can handle:

• audio management for seamless transitions between all audio sources
• media detection and browsing of connected devices
• background synching of music for instant media playback — without the need for the synch to be completed

Support for all smartphone connectivity options — DLNA, MTP, MirrorLink, Bluetooth, USB, Wi-Fi, etc.

On-board application framework — Supports Qt, HTML5, APK (for Android apps), and native OpenGL ES apps. Apps include iHeart, Parkopedia, Pandora, Slacker, and Weather Network, as well as a Settings app for phone pairing, over-the-air software updates, and Wi-Fi hotspot setup.

So if you’re in the North Hall at CES this week, be sure to take a virtual ride in the QNX reference vehicle in Booth 2231. Beneath the fresh paint job, it’s the same workhorse it has always been, but now with new ADAS tech automakers are thirsting for.

One day I’ll be Luke Skywalker

Cyril Clocher

What happens when you blend ADAS with infotainment? Guest post by Cyril Clocher, business manager for automotive processors at Texas Instruments

As we all begin preparing for our trek to Vegas for CES 2015, I would like my young friends (born in the 70s, of course) to reflect on their impressions of the first episode of Lucas’s trilogy back in 1977. On my side, I perfectly remember thinking one day I would be Luke Skywalker.

The eyes of young boys and girls were literally amazed by this epic space opera and particularly by technologies used by our heroes to fight the Galactic Empire. You have to remember it was an era where we still used rotary phones and GPS was in its infancy. So you can imagine how impactful it was for us to see our favorite characters using wireless electronic gadgets with revolutionary HMIs such as natural voice recognition, gesture controls or touch screens; droids speaking and enhancing human intelligence; and autonomous vehicles traveling the galaxy safely while playing chess with a Wookiee. Now you’re with me…

But instead of becoming Luke Skywalker a lot of us realized that we would have a bigger impact by inventing or engineering these technologies and by transforming early concepts into real products we all use today. As a result, smartphones and wireless connectivity are now in our everyday lives; the Internet of Things (IoT) is getting more popular in applications such as activity trackers that monitor personal metrics; and our kids are more used to touch screens than mice or keyboards, and cannot think of on-line gaming without gesture control. In fact, I just used voice recognition to upgrade the Wi-Fi plan from my Telco provider.

But the journey is not over yet. Our generation has still to deliver an autonomous vehicle that is green, safe, and fun to control – I think the word “drive” will be obsolete for such a vehicle.

The automotive industry has taken several steps to achieve this exciting goal, including integration of advanced and connected in-car infotainment systems in more models as well as a number of technologies categorized under Advanced Driver Assistance Systems (ADAS) that can create a safer and unique driving experience. From more than a decade, Texas Instruments has invested in infotainment and ADAS: “Jacinto” and TDAx automotive processors as well as the many analog companion chips supporting these trends.

"Jacinto 6 EP" and "Jacinto 6 Ex"
infotainment processors

A unique approach of TI is our capability to leverage best of both worlds for non-safety critical features, and to provide a seamless integration of informational ADAS functions into existing infotainment systems so the vehicle better informs and warns the driver. We announced that capability at SAE Convergence in Detroit in October 2014 with the “Jacinto 6 Ex” processor (DRA756), which contains powerful CPU, graphics multimedia, and radio cores with differentiated vision co-processors, called embedded vision engines (EVE), and additional DSPs that perform the complex ADAS processing.

For the TI’s automotive team, the CES 2015 show is even more exciting than in previous years, as we’ve taken our concept of informational ADAS to the next step. With joint efforts and hard work from both TI and QNX teams, we’ve together implemented a real informational ADAS system running the QNX CAR™ Platform for Infotainment on a “Jacinto 6 Ex” processor.

I could try describing this system in detail, but just like the Star Wars movies, it’s best to experience our “Jacinto 6 Ex” and QNX CAR Platform-based system in person. Contact your TI or QNX representative today and schedule a meeting to visit our private suite at CES at the TI Village (N115-N119) or to immerse yourself in a combined IVI, cluster, megapixel surround view, and DLP® based HUD display with augmented reality running on a single “Jacinto 6 Ex” SoC demonstration. And don't forget to visit the QNX booth (2231), where you can see the QNX reference vehicle running a variety of ADAS and infotainment applications on “Jacinto 6” processors.

Integrated cockpit featuring DLP powered HUD and QNX CAR Platform running on a single “Jacinto 6 Ex” SoC.

One day I’ll experience Skywalker’s life as I will no doubt have the opportunity to control an intelligent and autonomous vehicle with my biometrics, voice, and gestures while riding with my family to the movie theater playing chess with my grandkids, not yet a Wookiee.

Scale

Guest post by Matt Watson, TI product line manager, on the new Jacinto 6 Eco processor

Matt Watson

Whenever I hear someone mention the word “scale,” several images come to mind. The first one is the thing I avoid stepping on for fear of its shocking readout. The second is what happens to my tender, east Texas skin whenever I venture to high altitudes — which is anything higher than a Houston overpass.

Since my colleagues at QNX were nice enough to let me post on their blog, I should avoid pursuing those activities further for fear of never being invited back. [Matt, you are *always* welcome here — Ed.] Instead, I’ll focus on how TI and QNX, together, are bringing incredible performance scalability to our customers in the automotive infotainment space through the new DRA72x “Jacinto 6 Eco” processor. This processor builds on the successful foundation of the “Jacinto 6” family and offers significant value to a very wide range of in-vehicle systems. This value takes three forms:

• software reusability and hardware pin-compatibility with “Jacinto 6,” resulting in faster time to market
   
• the ability to leverage the same integration capabilities as “Jacinto 6” for a lower BOM
   
• scalability to bring elements and features of high-end infotainment down to the entry-to-mid segment in a cost-effective manner
   

QNX and TI have been collaborating for over 12 years, specifically on the “Jacinto 6” platform for more than 2 years, to help bring industry-leading performance, integration, and scalability to the automotive market. Due to its similarity with “Jacinto 6”, “Jacinto 6 Eco” allows customers to leverage a mature base of TI silicon and QNX software solutions to hit the ground driving.


Jacinto 6 Eco running the QNX CAR Platform for Infotainment and Crank Storyboard Suite

We’ll see you on the road! To view more blogs from the TI team, please be sure to check out Behind The Wheel.

---

Editor's note — Here's a little more about Matt:

Matt Watson is the product manager for the TI audio and automotive infotainment processors at Texas Instruments. At TI, Matt has also held roles managing software development for audio, multimedia, and infotainment groups over the last 12 years. Prior to joining TI, Matt held positions at Dolby Laboratories (focusing on low-bit rate audio coding) and Motorola Semiconductor (developing software for floating-point audio digital signal processors).

In good company: QNX partner solutions at 2014 CES

Guest post by Peter McCarthy of the QNX global partnerships team

Peter McCarthy

If anyone thinks that creating an infotainment system is easy, they obviously haven’t thought about it hard enough. It is, in fact, a massive undertaking that requires seamless integration of navigation engines, voice technologies, app environments, HMI tools, Internet music services, smartphone connectivity, automotive-hardened processors — the list goes on.

No single company could possibly offer all of these technologies. And even if it could, it still wouldn’t address the needs of automakers and tier one suppliers, who need the power of choice. Any company building an infotainment system needs the flexibility to combine Navigation Engine A with Processor B and Bluetooth Solution C.

Enabling customers to enjoy such choice without worrying about integration issues is something that QNX works very hard at. For evidence, look no further than our latest technology concept car, a modified Mercedes-Benz CLA45 AMG, which debuted this week at our CES booth. The car integrates an array of partner tech, including:

• voice recognition from Nuance
   
• navigation engines from Elektrobit (EB), Kotei Informatics, and HERE
   
• automotive processors from Texas Instruments
   
• MirrorLink smartphone-to-vehicle connectivity from RealVNC
   
• Android Jellybean version of iHeartRadio
   

Meanwhile, the head unit in our reference vehicle, also featured in the QNX booth, integrates several partner apps and holds the distinction of being the world’s first in-vehicle implementation of Qualcomm’s Snapdragon Automotive Solutions. And if that’s not enough, our booth contains demos of a navigation engine from Aisin AW and a digital instrument cluster built with HMI tools from HI Corporation.

Mind you, the action isn’t restricted to the QNX booth. Several partners have also gotten into the act and are demonstrating QNX-based systems in their CES booths and meeting rooms. For instance:

• Elektrobit — Demonstrating a new concept electric vehicle that sports an instrument cluster and infotainment system based on the QNX Neutrino Realtime Operating System.
   
• Freescale — Demonstrating the QNX CAR Platform for Infotainment on its i.MX 6 Applications Processors for Automotive.
   
• Gracenote — Demonstrating how its technology can personalize the in-vehicle music experience, using a system based on the QNX Neutrino OS.
   
• NVIDIA — Demonstrating Audi's newest infotainment system featuring the NVIDIA Tegra processor and the QNX Neutrino OS.
   
• Qualcomm — Demonstrating the QNX CAR Platform on Snapdragon Automotive Solutions.
   
• Red Bend Software — Demonstrating virtualization technology that runs the QNX CAR Platform and a digital instrument cluster on dual displays driven by a single processor.
   
• Texas Instruments — Demonstrating the QNX CAR Platform running on its latest Jacinto processors
  

For the fully skinny on QNX partner technology at CES, I invite you to check out our press release, along with joint announcements that we have issued with Aisin AW, HERE, HI Corporation, and Qualcomm.

---

About Peter
When he isn't talking on oversized mobile phones, Peter McCarthy serves as director of global partnerships at QNX Software Systems, where he is responsible for establishing and fostering partnerships with technology and services companies in all of the company's target industries.

So many cores — what to do with them all?

Multi-core processors are clearly becoming the mainstream for automotive infotainment systems. TI’s OMAP processors and their automotive derivatives use dual A15 cores, Freescale's i.MX 6 boasts up to four A9 cores, and other companies also have multi-core architectures in production or on near-term roadmaps. Quad-core A15 processors are just around the corner. As a percentage of overall die area, the CPU core is relatively small, so SoC producers can lay down multiple cores with little impact on cost. GPUs, on the other hand, represent a large percentage of the die area and, as such, are typically instantiated only once per SoC.

Realistically, infotainment systems should no longer be CPU bound. In fact, when looking at leading-edge SoCs available today, many are asking what to do with all that extra CPU just sitting there, waiting to do something. At first blush, the more obvious areas to merge are infotainment and ADAS, or infotainment and digital instrument clusters. This is, at the highest level, pretty much a no-brainer so long as the safety requirements mandated for clusters and ADAS can be achieved.

Thing is, automotive safety standards like ISO 26262 require system-level certifications. As such, the entire system needs to be certified. Already a daunting task for a standalone ADAS system or digital instrument cluster, the problem can become unmanageable when you include the full infotainment stack.

Think about your car. Your cluster does a handful of operations whereas your infotainment system runs a full navigation system, voice recognition, multimedia, device connectivity, and, in the connected case, cloud services. People don't get frustrated trying to figure out how your cluster works (I hope), and they don't give up trying to figure out how fast the car is moving. The same cannot be said for many infotainment systems shipping today. Ask your mother to pair her cell phone to her car. I dare you! The complexity involved in attempting to certify a system that combines infotainment and cluster functions is orders of magnitude higher than certifying a cluster alone.

All is not lost, however. Virtualization offers an elegant way to isolate multiple systems running on a single multi-core SoC. By using virtualization you could seek certification on the cluster without burdening yourself with the infotainment problem and collapse two formerly discrete systems onto one SoC. You would save money and probably get a promotion to boot. Just one thing: there is still only one GPU on the die and both the infotainment system and the cluster rely heavily on that single GPU.

Enter Red Bend Software, a long-time QNX CAR Platform partner for FOTA. They have taken the challenge of virtualizing the GPU head-on and successfully demonstrated the QNX CAR Platform and a Crank Software-based digital instrument cluster running on dual displays driven by a single OMAP 5 at Telematics Munich. I saw it and was impressed with how snappy performance was on the infotainment side and how smooth the needles were (60+ fps) on the cluster.


Using virtualization to drive dual displays from a single TI OMAP 5 processor.

According to Red Bend, they have designed a GPU-sharing architecture that enables multiple guest operating systems to access hardware accelerators, including the GPU, providing isolation between the operating systems while having a minimal impact on overall performance (which probably isn't a huge deal considering how many CPU cores are going to be shipping on a single SoC in the near term). It sounds impressive, but seeing is believing.

Red Bend will next show this demo in the TI Suite at CES (N115 in the North Hall). If system consolidation is something that keeps you up at night, you should really stop by to see what they have done.