//php echo do_shortcode(‘[responsivevoice_button voice=”US English Male” buttontext=”Listen to Post”]’) ?>

After years of nurturing by researchers, AI is springing up and has been experiencing significant growth. AI deployment encompasses various sectors such as automotive, industrial automation, healthcare, education, home and living, finance, and more. AI-powered solutions have become part and pieces of our surroundings without us realizing it, providing the intelligence to replace direct human intervention. AI technologies like machine learning, deep learning, computer vision, speech generation, natural language processing, marketing automation, etc., are generally uncovered at the system processing level for data crunching through their ability to process complex and massive data, whether on the edge or cloud server. Nevertheless, these AI-powered end devices may still consume power and time for collecting and processing raw information, such as user input information and sensor data signals. As most end devices are wireless and battery-operated, low power consumption has become a key concern in the system design in every segment, from the front-end collection and processing of raw information to back-end local host processing and interface communication to external systems. With PixArt’s expertise in the CMOS Image Sensor (CIS) technologies and design of Application Specific IC (ASIC), we renovate the conventional CIS by exerting AI in the image sensor pixels. We are bringing a new image sensor architecture to the next advanced level by integrating AI-driven pixels in the new SMART PIXEL OPTICAL SENSING TECHNOLOGY. The AI-driven pixels in the front-end first line sensing of a system empower capturing more detailed and accurate visual information at lower power consumption and shorter processing time. 

INTRODUCING THE NEW SMART PIXEL OPTICAL SENSING CHIP – PAC9001LU 

The PAC9001LU Smart Pixel Optical Sensing Chip is a Computer Vision ASIC that fits as an always-on motion sensor by leveraging the novel AI-driven pixel architecture into the sensor array design. Based on a CMOS Image Sensor rolling shutter structural design with an array of 36 x 16 pixels, it can support a high frame rate of up to 1000Hz to facilitate image capturing of fast-moving object applications. The design of AI in pixels integrates a frame comparing circuit with AI-powered algorithms to compute differences in pixel luminosity within a configurable image area. It directly provides analog frame differences and event info in Pixel Differences Mode and supports Smart Motion Detection Mode to eliminate the complex image signal processing in the processor. The partial array sensing, such as configurable ROI region, provides supple custom scene capturing for the needs of AIoT edge applications.

The PAC9001LU offers more than a conventional low-resolution image sensor that output raw images only. It is an application-based SoC sensor that integrates with an on-chip processing unit, memory, and GPIO, all on a single substrate. Its function is not limited to capturing indoor or outdoor scene images for motion/event detection. It also offers many programmable flexibilities for settings and parameters tuning to custom-made for different scenarios. This chip is built-in with self-learning intelligence by storing the past motion/event detection record in its memory for data mining. Based on our initial testing in-house, the motion detection rate of PAC9001LU could cover up to 95% under different situations and test conditions. In addition, the chip supports the setting conversion into new designs of specific applications to ease the design effort.  

The chip uses the I2C as the control interface and outputs image data through the I2C or SPI interface. The data output format depends on selecting the two operation modes available: Pixel Differences Mode and Smart Motion Detection Mode. The output data significantly differentiates the PAC9001LU from other image sensors. It can output motion event interrupt in the primitive data form to eliminate the data polling process by the backend host controller or ISP. The optimization of the raw image data conversion, processing, and analysis can reduce the data bandwidth, latency, processing resources, and power consumption to boost the overall system performance. The versatility of PAC9001LU as an AI-powered SoC Sensor is conducive to simplifying the system design complexity with the lowest possible power consumption.  

1. Pixel Differences Mode 

In this mode, the PAC9001LU directly handles the digital conversion of raw image signals. It computes the image subtraction difference between two frames internally in the chip to provide the difference of each pixel in an 8-bit data format. This 8-bit data size is relatively small compared to the raw image data of a whole pixel array, which can effectively reduce data transmission bandwidth and latency issues. Likewise, the motion event interrupt is also available by setting the threshold to the 8-bit pixel differences data converting to digital information of 0 or 1. The chip also supports several advanced custom-use control options for intricate scenarios. 

2. Smart Motion Detection Mode 

Through the in-built AI-powered algorithms, Smart Motion Detection Mode can generate a motion interrupt upon detecting motion or scene change based on the image data captured by continuous monitoring. The interrupt output is an event-based trigger used as a motion alert for direct readout by the host controller for next-level action judgment. The PAC9001LU chip also gains knowledge of the physical scene phenomena by analyzing all the images frame by frame in all circumstances with its self-learning intelligence.  

For example, the background continuous motion monitoring or luminance changes in the Smart Detection Mode can be an event triggering to wake up the primary system host without additional external resources at the low power consumption of ~70µW. This ultra-low power design is ideal for the power-concerned always-on wireless devices in on-the-edge applications such as home-use surveillance monitoring, intelligent sensing on mobile or IoT devices, event triggering in the automation industry, etc. The high frame rate of up to 1000 frames per second (FPS) makes it possible to capture images of fast-moving objects and perform on-chip image signal processing and analysis to provide more accurate information and reduce false alarms to interpret the physical scene better. Many motion monitoring applications could replace traditional PIR sensors with this PAC9001LU as an image-based motion sensor to enrich IoT products’ features.  

The PAC9001LU chip is in a W2.5 x L2.6 x H0.43 mm3 CSP package body (excluding solder balls). A recommended matching lens set, LST0-2621 is also available to form a complete module when assembled with the PAC9001LU chip and comes in a size of W3.79 x L3.63 x H1.67 mm3 (height is including guide pin). The miniature form factor, whether in a standalone chip or module, is attributed to the broad use of the PAC9001LU sensor in vast applications’ space-confined design. The module set provides optimized optical performance corresponding to the image sensor configuration to ease the optomechanical assembly design in custom module production. Customers can also design and assemble the module based on the design support tools. 

EMPOWERING FOR AI-MODERNIZED APPLICATIONS  

1. Replacing PIR Motion Sensing Function 

PIR sensors are playing a crucial role in the motion sensing market. Even though there are scenarios where there are false alarms or malfunctions, their usage has existed for decades. It is irreplaceable due to the advantages of simple use, low power consumption, and cost. As many motion sensing applications evolve with the call to upgrade as AIoT devices, introducing new technologies in offering smaller module forms and delivering Smart features that can provide more helpful information about the surroundings proliferates the phase-out of the traditional PIR. The PAC9001LU’s image-based optical sensing architecture incorporates the intelligence of signal processing between multiple frames of images at the pixel level and the integrated algorithm for motion detection. It is not only ideal as an event-based triggering motion sensor. Still, it has extended features to provide more valuable intelligence to transform into Smart applications, such as presence sensing of desired objects in the Smart home, building and personal gadgets, and detection switches in industrial automation.  

The table below compares several motion detection solutions that are commonly available nowadays and highlights the key strengths of PAC9001LU that outstand from others.  

• The low-power consumption during the Smart Motion Detection Mode that comes with intelligent informative is the most remarkable building block worthwhile in enabling AI applications. As compare to PIR or CMOS Image Sensor (CIS), higher power consumption is required for further data processing in system level. 
• The high report rate, which can go up to 1000Hz can achieve motion detection with fast-moving objects, which outperforms the PIR or conventional CIS.   
• The PAC9001LU is more robust with reliable performance. It has fewer false alarms detecting motion and higher immunity to temperature interference. The external environment factors, such as bright and hot sunlight from outdoors, the indoors thermal noise from heated devices are not affecting its sensing performance. The built-in algorithms can eliminate interferences like background noise too. 

• The small form factor of the complete PAC9001LU sensor module, including the lens set, can nicely fit into the slim bezel ID design.  
• The traditional PIR sensors are usually required not to be shielded by plastics or glass front-facing cover, which may impact the detection of thermal IR radiation. Whereas the PAC9001LU solution does not have the restraint of having a front cover of any materials and still can keep the motion sensing quality, even placing the motion sensing device indoors looking out from the glass window is possible. With the cover protection, the PAC9001LU is less prone to external damage.  

Using PixArt’s advanced pixel design, the PAC9001LU sensor can support low-light sensing in low or no-light conditions, which is very suitable for use in a dark environment, such as a basement. Typically, the maximum sensing distance in normal room lighting conditions is 5M with accurate motion sensing performance. However, under low-light conditions beyond at 5M, the sensor performance degrades with a higher false alarm occurrence. Referring to the in-house testing results shown below, at 0.1 LUX with a detecting distance of 4.5M away from the chip, the PAC9001LU can achieve a 100% motion detection rate. The chip can still actively capture images at near-dark conditions of <0.01 LUX. However, the failure rate of motion sensing hits 50% at a distance of 2M. The higher failure rate is due to the low image sensitivity that does not have much pixel contrast to be differentiated. This limitation is common to an imaging sensor that depends on ambient light (without external illumination) for sensing purposes. When at a nearer distance, the sensing successful rate is higher.  

2. In-line Monitoring in Factory Automation 

The transformation of factory automation for Industry 4.0 highly demands real-time monitoring to keep productivity at the best optimum throughput. The PAC9001LU can cater to the need for in-chip high-speed motion detection, eliminating the external controller processing. Although a conventional image sensor can provide higher-resolution images, it still requires more resources and bandwidth for raw image signal processing. As most of the production line is in a fixed background, only the object movement in the area of interest is the target subjected for the conversion into valuable data to pass on for the following action call in the real-time monitoring production line. 

As illustrated in the figure below, the goods on the moving conveyor in the manufacturing facilities are key target. By setting up the PAC9001LU sensor to in-line monitor the goods passing by the desired view of interest, when there is goods jam or slip in the conveyor belt halting the production, the PAC9001LU can instantly trigger the alarm to alert the immediate attention to the in-line operator. When any object motion occurs, it can prompt event change interrupt indicating the production line is in good health and the output data of pixels differences can be a mean of further offline data analysis. Since the data size of pixels differences output is relatively less than raw image data, the PAC9001LU can capture the images and on-chip processing at high frame rate up to1000Hz.  

3. Object Tracking  

In addition to motion sensing, the PAC9001LU sensor can provide the coordinate information of a targeted moving object that is in sync with each pixel differences image data. Based on the objects’ coordinates, the host controller can perform post data processing in analyzing the object’s movement to obtain the moving direction, relative tracking path, and count the number of objects moving in and out within the area of interest. The thresholds are configurable to optimize for the identification of a moving object. The on-chip algorithm has the trained intelligences to disregard the interference of light switching or flickering to avoid false alarm of luminance chances. 

DESIGN SUPPORT – HIGHLY INTEGRATED EVALUATION BOARD 

The PAC9001KE Evaluation Kit is available for evaluation and design research purposes. The system block diagram shows that the evaluation kit includes an evaluation board (EVB) that integrates with the PAC9001LU chip, LST0-2621 lens set, other passive components, jumper connectors, etc., and an MCU controller board. Customers can connect the EVB directly to the host controller board to access the PAC9001LU chip and read the raw image, pixel differences image, and motion event interrupt data from the sensor. We aim to ease the design resources with a plug-and-play approach using our ready-made evaluation board to integrate and fusion with other subsystems, such as PIR, high-resolution CIS, ToF, etc., to form a more complex system design catering to different use cases.  

PixArt can provide extensive support to customers in designing and developing their end systems. We look forward to building partnerships with customers embarking into the AI domain with Smart Pixel Optical Sensing technology.