This article was published on Microsoft Customer Stories on May 27th 2022

Early on, Scenera recognized that the greatest limitation on the deployment of video cameras is rooted not in technology issues, but privacy concerns. In the same way, the majority of perspectives on IoT have been focused on devices and the related technology, but not necessarily the supporting ecosystem that makes a system truly viable in the long-term.

Scenera is a co-founder of the Network of Intelligent Camera Ecosystem (NICE) Alliance. The NICE Alliance is committed to establishing global protocols for manufacturers worldwide to align their service and enhance product development.

Founding members of the alliance include industry leading electronics manufacturers and brands such as Nikon and Sony Semiconductor Solutions in Japan, along with Wistron and Foxconn in Taiwan. Image solution providers like these can utilize the evolving protocols ready for the evolution from color-rich images to rich scene information that enables new and powerful capabilities. 

At the intersection of AI, IoT, 5G and cloud, Scenera provides end-to-end orchestration from cloud to edge, enabling maximum possible use for edge processing for cost, efficiency and privacy.

Microsoft Azure Cloud was selected in 2020 to ensure that the IoT data for all partner companies is manageable, searchable, secure, and private within the scalable platform provided by Scenera.

New classes of applications and innovative standards for smart camera markets

Scenera aims to overcome the difficulties in achieving a system where no personal data is exposed, otherwise known as Zero Image Video Surveillance. To realize those goals, other necessary features include curation of Scene-based Event Capture, analysis of Structured & Organized Data, and processing with Distributed AI. 

Dr. William J. Raduchel, Scenera Strategic Advisor says, “Our goal is to achieve Zero Image Video Surveillance, recognizing that the creation of these systems fully exploits the power of vision while protecting privacy by keeping the images within the camera and controlling which events are signaled by the sensor. NICE makes that possible by enabling camera sensors to effectively tell them what to do and how. People have legitimate privacy concerns when they see cameras, and we already have many laws mandating disclosure and forbidding use in some circumstances. NICE changes the game.”

NICE simplifies the process for partner companies and enables them to focus on differentiation and development of their own suites of applications for intelligent cameras and intelligent cloud services. The collaboration with Microsoft as a development partner for NICE Cloud Services incorporates Azure Media Services, Azure Storage, Azure KeyVault, Azure Cognitive Services, Azure IoT Edge, along with many other Azure services.

In recalling the considerations for pursuing development, David Lee, Scenera CEO & Co-Founder says, “The reason we started to solve this five years ago is because there was so much focus on the IoT device itself, which nowadays, is highly powered and highly capable. However, we did not really think that building directly into the hardware was the most optimal decision.”

At the same time, Microsoft and other similar platforms were all creating powerful cloud technology. However, some companies only operate in closed systems, which locks in consumers. One of the key points is that consumers locked into the end-user license agreements don’t necessarily have access to whatever information is collected, whether listened to or viewed. 

Therefore, the ideal scenario is for consumers to be able to protect their data while also maximizing their choice of their devices and analytics.”

Since the start of its initiative, Scenera has been working with several Japanese companies, including Nikon and Sony Semiconductor Solutions, as a part of the NICE Alliance. In 2019, the need for a cloud service partnership became more apparent. Through a fortuitous introduction, directors from Scenera, Sony and Nikon visited and discussed development with executives at the headquarters of Microsoft in Redmond, Washington. A key individual in those discussions was involved with agile marketing as a corporate VP. He liked the story behind Scenera’s vision and that was a large part of securing a collaboration agreement with Microsoft in 2020.

Realizing edge processing with minimized costs and maximized efficiency and privacy

The end goal of the NICE Alliance is to allow participating member corporations to access cloud-to-edge operation, enabling the maximum possible leverage with edge processing and minimized costs while optimizing efficiency and privacy. Scenera provides the core solution for that intersection of AI, IoT and Cloud, with the infrastructure provided on a global scale by Microsoft Azure.

As a real-world case, faces of staff and frequent guests in an office can be uploaded while all other images are simultaneously ignored when those particular facial images are present. In terms of practical application, David says, “You could then use video surveillance to look for all anomalies without having to worry about any loss of privacy. Public cameras could look for specific faces under court order, but ignore everyone else. The opportunities are endless.”

Partners adopting the standards provided by the alliance can build their own software and devices around solid protocols, and ensure their solutions are connected to the cloud. 

A robust cloud infrastructure such as Azure Cloud Services was therefore essential. As explained by David, “Hosting the necessary resources on the cloud is so much better. As we couldn’t find any other resources available, we decided to solve this problem ourselves and our search led to the partnership with Microsoft.”

One of the first client partners Scenera engaged with was S&I Corp., the company that services building and facility management for LG Electronics Group in Korea. Scenera partnered with S&I to implement its AI-based surveillance system as a smart solution for over 150 buildings including factories, retail stories and other facilities. 

Microsoft and the lead team at Scenera worked together on this project extensively during the first half of 2020. The initial launch was introduced at one of the LGE facilities by the second half of 2020, and the progress in that development allowed its commercial launch in May 2021 at several additional buildings owned by LGE. Given their satisfaction with the roll-out, LGE now plans for further incorporation of the platform for their affiliated facilities throughout inside the group.

Hyungjun Son, Team Leader / INFRA Development team, S&I Corp. says, “Implementing and managing a cloud-based Al surveillance service can be very costly and time consuming. With Scenera’s Platform as a Service technology, we were able to customize our edge-to-cloud Al analytic workflow, employing various 3rd party Al models. This service enabled us to recognize the events of our interests and monitor analytic performance to improve and add new Al models. Supported by Microsoft Azure infrastructure and managed by Scenera and TnM, this system helps manage our business more cost efficiently.”

Creating a solid footprint to establish NICE Alliance’s presence among adopters and consumers, both Scenera and Microsoft are planning for additional projects including a proof-of-concept system for customer sites in Los Angeles.

One of the benefits of the Scenera solution is to enable the customer to customize the analytics for each camera and to reuse the camera for multiple different types of analytics.

Secure cloud-based service platform with global-standard key vault and encryption keys

While there were other platforms connecting cameras to the cloud, all were closed ecosystems. This meant that Scenera’s partner companies were left without any practical solutions for their own cloud solutions.

Another significant issue for partner companies was the issue of trust, whether dealing with other businesses and enterprises, or with consumers as end-users. In simple terms, there are several elements where trust – privacy and security – must be ensured as data is transferred from point-to-point. This starts with the camera, or IoT device, that generates or sends the data, along with a user at some point that wants access to that information. Next is the cloud service providing storage and processing, including AI. Finally, there are the related applications needed to handle the final data according to specific user-defined requirements.

Andrew Wajs, Scenera CTO & Co-Founder says, “We came up with a scheme to bind all these related support elements together but needed a very secure cloud-based service with corresponding security features including a key vault and encryption keys. Microsoft’s Azure Key Vault was ideal as it is already trusted and recognized globally by many in the industry. Based on that service, we were able to combine all these elements together.”

The NICE Alliance provides the necessary protocol to separate device, data, and application securities, enabling each to be managed independently while also ensuring maximized security. It was on this basis that Scenera convinced their customer base to trust the platform with their data, device, user accounts, and cloud applications. The initial background discussions with partners in the alliance distilled some of these key challenges. 

“We were collaborating with companies like Nikon and Sony Semiconductor Solutions. These companies are building numerous interesting and innovative products, but their eye was on advanced sensors. In the case of Sony, they have AI sensors which are not only sensing images but running AI right on the sensor.

As part of the integration process, they tried to connect their devices to the cloud, to realize that it was not an easy or straightforward process to decide which right cloud resources to use. It’s just not that simple,” says David.

Scenera has been able to rely on the research teams at Microsoft while exploring numerous use cases with Azure Cloud. The offloading of processing between each transfer stage is something where significant customization is required to match the needs of both smaller and enterprise-scale partners in the alliance.

The Microsoft Azure Key Vault is complementary to NICE security, as it further enables secure cloud processing and management. In addition, the Azure Key Vault is fully certified in accordance with several international security standards. This elevates the NICE ecosystem to a truly global level, while maintaining security and adhering to established industry security practices.

David explained, “Although you have a standard system running, there are needs for new ways of implementing things. For example, take the recognition of an event at an on-premises location with so many subsequent events happening at the hardware level that the device cannot handle the processing. We enable “curation” from multiple cameras by configuring each data stream with custom AI models for every IOT’s unique role in gathering and detect the right events.

Scenera can efficiently move the processing workflow on-premises, essentially offloading processing to the larger capacity of a next-level server where additional computing can also be applied. 

We have found the Microsoft team highly engaged in figuring out if we can move an application, or certain types of analytic processing, from one location to the other without losing any integrity. We are now able to successfully demonstrate to members of the alliance and our client partners that we can fully adjust downstream and upstream processing at all levels.

Leveraging benefits of edge computing for sustainability and public safety

Scenera is a smaller company but with the support of Microsoft in early 2022, they have joined another open-source project from Europe for CAMARA – The Telco Global API Alliance. The expectation is that this will lead to further opportunities for Scenera in a new market with an expanding range of international partners.

This style of expansion through alliance building is one that comes from previous experience with HDMI by Scenera’s leadership team. David says, “Our introduction and promotion of HDMI followed an untraditional path where we approached key content creators and Hollywood Studios that wanted to distribute digitally to provide the best entertainment experience. We embraced this direction instead of going a traditional path with TV manufacturers. Our next step was to approach large retailers and those respective distribution channels to discuss the potential for increased sales with the features from this new standard. This momentum is what encouraged hardware manufacturers to adopt the standards which we now see in so many devices including automobiles.”

Scenera has added an additional category to its objectives aligned with public safety, which is a concern for individuals, communities, and nations worldwide. One example is prevention or mitigation of natural disasters such as wildfires. 

David commented, “The system we envision to handle this will be driven by consumer and public demand, which will push adoption by municipalities. There are numerous difficulties to overcome including putting AI capabilities in reach at a reasonable price, with the right type of applications that can run on advanced networks like 5G.

Overcoming emerging obstacles like this is one of the real motivations for the NICE Alliance where our partner companies and development partners like Microsoft can create incredibly compelling technology that consumers and end-users really want to use. 

Azure IoT Edge by Microsoft is a toolkit that can enable the streamlined connections of such smart devices to the Microsoft Cloud. Thousands of devices are already registered for Azure IoT Edge that can also take advantage of our platform and standardized protocols.”

Another emerging category of potential client partners are energy utility companies. In numerous regions in the US, notably California, wildfires have become a significant issue with billions in damages over the years. This is a use case where cameras in public spaces such as mountains forests, ports, and power stations can be unified with an early-detection system to detect smoke or fires across a large area.

While these emerging opportunities align in many ways with Sustainable Development Goals (SDGs), there are also accompanying privacy concerns that must be addressed. David summarized, “Scenera is very interested in contributing to better health, public safety, and the environment. We are aiming to continue in this area through our collaboration with Microsoft. At the same time, the 5G companies are also important as they will play a great role in fast-tracking event notifications and device communication.”

Scenera’s teams are located globally in India, Europe, Korea, Japan, and the US. With Azure Cloud, operations can be optimized to balance the local processing of small edge computing devices, which are on site connected to cameras and other inexpensive “always on” bridge devices, to seamlessly connect to the cloud with Azure Stack Edge.

Where the introduction of HDMI allowed consumers to enjoy a better experience through wired connections, Scenera and the NICE Alliance are aiming to realize a wireless future enabled through the air with the power of cloud computing and 5G.

Our goal is to achieve Zero Image Video Surveillance, which is realized by creation of systems that fully exploit the power of vision but protect privacy by keeping the images within the camera and controlling which events are signaled by the sensor. 

Dr. William J. Raduchel: Strategic Advisor
Scenera, Inc.

Hosting the necessary resources on the cloud is so much better. As we couldn’t find any other resources available, we decided to solve this problem by ourselves, and our search led to the partnership with Microsoft. 

David Lee: CEO & Co-Founder
Scenera, Inc.

We came up with a scheme to bind all these related support elements together but needed a very secure cloud-based service with corresponding security features including a key vault and encryption keys. Microsoft’s Azure Key Vault was ideal as it is already trusted and recognized globally by many in the industry. So based on that service, we were able to combine all these elements together. Our goal is to achieve Zero Image Video Surveillance, which is realized by creation of systems that fully exploit the power of vision but protect privacy by keeping the images within the camera and controlling which events are signaled by the sensor. 

Andrew Wajs: CTO & Co-Founder
Scenera, Inc.

Hosting the necessary resources on the cloud is so much better. As we couldn’t find any other resources available, we decided to solve this problem by ourselves, and our search led to the partnership with Microsoft. 

David Lee: CEO & Co-Founder
Scenera, Inc.

Implementing and managing a cloud-based Al surveillance service can be very costly and time consuming. With Scenera’s Platform as a Service technology, we were able to customize our edge-to-cloud Al analytic workflow, employing various 3rd party Al models. This service enabled us to recognize the events of our interests and monitor analytic performance to improve and add new Al models. Supported by Microsoft Azure infrastructure and managed by Scenera and TnM, this system helps manage our business more cost efficiently.We came up with a scheme to bind all these related support elements together but needed a very secure cloud-based service with corresponding security features including a key vault and encryption keys. Microsoft’s Azure Key Vault was ideal as it is already trusted and recognized globally by many in the industry. So based on that service, we were able to combine all these elements together. Our goal is to achieve Zero Image Video Surveillance, which is realized by creation of systems that fully exploit the power of vision but protect privacy by keeping the images within the camera and controlling which events are signaled by the sensor. 

Mr. Hyungjun Son: Team Leader / INFRA Development team
S&I Corp.