close
close

Edge AI’s New App-Changing Device

Posted on by darion

Edge computing and edge artificial intelligence (AI) and machine learning (ML) applications are increasingly transforming the landscape for organizations across industries. Most customer interactions still take place in stores, offices, and amusement parks, which are edge locations. Similarly, many operational interactions take place in factories and warehouses, which are also edge locations. In these edge locations, an increasing amount of data is being produced using cameras, sensors, and other IoT devices. This has made the edge locations fertile ground for new AI/ML applications in retail, manufacturing, healthcare, and even the public sector.

In the healthcare industry, for example, CT scanners, video cameras, and many other sensors generate a significant amount of data. Due to consumer privacy concerns and stringent regulatory requirements, hospitals and healthcare organizations are actively exploring the use of edge AI/ML applications to offer on-site discovery and determination. In factories, high-speed cameras offer a new way to detect and remove defective products before they reach consumers and lead to costly recalls. As companies increasingly rely on AI and ML to drive innovation and deliver improved user experiences, the need for robust edge computing solutions has become more apparent.

Enabling AI and Machine Learning at the Edge

Manjul Sahay, a product manager at Google, leads the development of the Google Distributed Cloud Edge Appliance, an edge computing solution designed to help companies deploy AI and ML applications in remote locations, even in environments with unreliable network connectivity.

Edge Appliance offers an easy-to-use solution that integrates with cloud services, enabling enterprises to explore new possibilities and deliver unconventional user experiences, regardless of location. Google Distributed Cloud Edge Appliance falls into the same category as Azure Stack HCI and AWS Outposts. It is aimed at enterprises that need a modern, cloud-native platform to run data-driven, compute-intensive AI/ML workloads at the edge. Each The Edge Appliance is equipped with a 16-core processor, 64GB of RAM, an NVIDIA T4 GPU, and 3.6TB of usable storage. The device also has a pair of 10 Gigabit and 1 Gigabit Ethernet ports. With a 1U rack-mount form factor, it supports both landscape and portrait orientations, making it well-suited to a variety of edge deployment scenarios.

Sahay led the development team for the Google Edge Appliance, focusing on key advances in security, the AI/ML platform, and network connectivity. First, Sahay and the team focused on security features, as data security and privacy are key concerns for any edge technology. Edge devices are typically placed in unsecure locations, as opposed to a corporate data center with multiple layers of security. A device placed in a retail store, hospital, or factory can be easily removed by a determined attacker. The devices are also vulnerable to tampering during shipment to and from these locations. Sahay’s team identified and built in a set of advanced security features, such as double encryption of the security key, storing the key on a Trusted Platform module instead of on disk, and shipping with a hardened operating system.

Then, for an AI/ML platform, Sahay led the team in evaluating options and decided to use Kubernetes as the base platform. Sahay explained that using Kubernetes reduces the platform footprint and makes the majority of the compute power available to customer applications. Based on CPU benchmarks and sysbench tests, Kubernetes resource utilization, CPU and RAM, is up to 50% lower compared to a traditional VM approach. Additionally, Kubernetes applications can be easily built in the cloud and then quickly deployed to dozens or even hundreds of edge devices. This choice of Kubernetes platform is different from what other big cloud players like Amazon have made, and was a bold decision for the team.

The final tricky part was solving the problem of intermittent network connectivity. Sahay built on previous work by Storage Appliances to incorporate technology that allows these devices to operate without network connectivity for up to 90 days. Despite intermittent network connectivity, the device’s ability to operate with full AI and ML capabilities sets it apart from traditional edge computing solutions. This resilience is key for companies operating in remote or challenging environments where reliable connectivity isn’t always guaranteed.

Many applications in various industries

The Edge Appliance has potential AI/ML applications across industries, including telecommunications, manufacturing, autonomous vehicles, and retail. Google Cloud Sahay’s team has focused on several leading use cases and customers for these devices. One such use case is Advanced Driving Assistance System (ADAS). Nuro, a leading innovator in the ADAS space, collects a lot of data from remote environments, such as warehouses, that may not have continuous connectivity. Once the data is collected, it can be processed locally or transmitted to the cloud using these devices.

Another use case is for retail customers to redesign store management operations, such as monitoring store occupancy, line depth and wait times, slip and fall detection, and inventory compliance. Sahay and the team work closely with these customers to create industry-specific customizations, such as supporting in-car video protocols for devices in a car or rover, and enabling defect detection applications for devices for manufacturing customers.

Security concerns, industry sentiment and impact

While the Google Edge Appliance has stirred industry enthusiasm, some critics have raised concerns about the potential security risks associated with deploying AI and ML applications at the edge. Cybersecurity experts are concerned about the privacy implications of widespread data collection at the edge, data theft, ransomware attacks, distributed denial of service (DDoS) attacks, which can now span hundreds or even thousands of these devices across an organization. In this context, Sahaya’s work on dual-key encryption and enhanced OS security features has significantly improved the security posture of these devices.

Despite concerns, the general sentiment in the industry remains positive, with many experts predicting that edge computing will play an increasingly important role in the future of business technology. Sahaya’s work in creating and adapting the Edge Appliance has the potential to significantly impact a range of industries. Supporting the growth and development of ADAS is a game-changer in itself. Similarly, retail and manufacturing companies are significantly reducing costly store operations issues and manufacturing defect problems, potentially reducing costs by 10-20%. Telecom companies can create new revenue streams. The Appliance opens up new opportunities for innovation and growth across industries, making it easier for companies to deploy AI and ML applications at the edge. Early customer success points to an impactful future.

Faced with the challenges and opportunities that rapid technological change brings, product managers like Sahay help shape the future by helping companies adapt and thrive in an increasingly complex world.

Read more from Techbullion