Feed aggregator

Artificial intelligence (AI) is expanding rapidly to the edge. This generalization conceals many more specific advances—many kinds of applications, with different processing and memory requirements, moving to different kinds of platforms. One of the most exciting instances, happening soonest and with the most impact on users, is the appearance of TinyML inference models embedded at the extreme edge—in smart sensors and small consumer devices.

Figure 1 The TinyML inference models are being embedded at the extreme edge in smart sensors and small consumer devices. Source: PIMIC

This innovation is enabling valuable functions such as keyword spotting (detecting spoken keywords) or performing environmental-noise cancellation (ENC) with a single microphone. Users treasure the lower latency, reduced energy consumption, and improved privacy.

Local execution of TinyML models depends on the convergence of two advances. The first is the TinyML model itself. While most of the world’s attention is focused on enormous—and still growing—large language models (LLMs), some researchers are developing really small neural-network models built around hundreds of thousands of parameters instead of millions or billions. These TinyML models are proving very capable on inference tasks with predefined inputs and a modest number of inference outputs.

The second advance is in highly efficient embedded architectures for executing these tiny models. Instead of a server board or a PC, think of a die small enough to go inside an earbud and efficient enough to not harm battery life.

Several approaches

There are many important tasks involved in neural-network inference, but the computing workload is dominated by matrix multiplication operations. The key to implementing inference at the extreme edge is to perform these multiplications with as little time, power, and silicon area as possible. The key to launching a whole successful product line at the edge is to choose an approach that scales smoothly, in small increments, across the whole range of applications you wish to cover.

It is the nature of the technology that models get larger over time.

System designers are taking different approaches to this problem. For the tiniest of TinyML models in applications that are not particularly sensitive to latency, a simple microcontroller core will do the job. But even for small models, MCUs with their constant fetching, loading, and storing are not an energy-efficient approach. And scaling to larger models may be difficult or impossible.

For these reasons many choose DSP cores to do the processing. DSPs typically have powerful vector-processing subsystems that can perform hundreds of low-precision multiply-accumulate operations per cycle. They employ automated load/store and direct memory access (DMA) operations cleverly to keep the vector processors fed. And often DSP cores come in scalable families, so designers can add throughput by adding vector processor units within the same architecture.

But this scaling is coarse-grained, and at some point, it becomes necessary to add a whole DSP core or more to the design, and to reorganize the system as a multicore approach. And, not unlike the MCU, the DSP consumes a great deal of energy in shuffling data between instruction memory and instruction cache and instruction unit, and between data memory and data cache and vector registers.

For even larger models and more latency-sensitive applications, designers can turn to dedicated AI accelerators. These devices, generally either based on GPU-like SIMD processor arrays or on dataflow engines, provide massive parallelism for the matrix operations. They are gaining traction in data centers, but their large size, their focus on performance over power, and their difficulty in scaling down significantly make them less relevant for the TinyML world at the extreme edge.

Another alternative

There is another architecture that has been used with great success to accelerate matrix operations: processing-in-memory (PiM). In this approach, processing elements, rather than being clustered in a vector processor or pipelined in a dataflow engine, are strategically dispersed at intervals throughout the data memory. This has important benefits.

First, since processing units are located throughout the memory, processing is inherently highly parallel. And the degree of parallel execution scales smoothly: the larger the data memory, the more processing elements it will contain. The architecture needs not change at all.

In AI processing, 90–95% of the time and energy is consumed by matrix multiplication, as each parameter within a layer is computed with those in subsequent layers. PiM addresses this inefficiency by eliminating the constant data movement between memory and processors.

By storing AI model weights directly within memory elements and performing matrix multiplication inside the memory itself as input data arrives, PiM significantly reduces data transfer overhead. This approach not only enhances energy efficiency but also improves processing speed, delivering lower latency for AI computations.

To fully leverage the benefits of PiM, a carefully designed neural network processor is crucial. This processor must be optimized to seamlessly interface with PiM memory, unlocking its full performance potential and maximizing the advantages of this innovative technology.

Design case study

The theoretical advantages of PiM are well established for TinyML systems at the network edge. Take the case of Listen VL130, a voice-activated wake word inference chip,which is also PIMIC’s first product. Fabricated on TSMC’s standard 22-nm CMOS process, the chip’s always-on voice-detection circuitry consumes 20 µA.

This circuit triggers a PiM-based wake word-inference engine that consumes only 30 µA when active. In operation, that comes out to a 17-times reduction in power compared to an equivalent DSP implementation. And the chip is tiny, easily fitting inside a microphone package.

Figure 2 Listen VL130, connected to external MCU in the above diagram, is an ultra-low-power keyword-spotting AI chip designed for edge devices. Source: PIMIC

PIMIC’s second chip, Clarity NC100, takes on a more ambitious TinyML model: single-microphone ENC. Consuming less than 200 µA, which is up to 30 times more efficient than a DSP approach, it’s also small enough for in-microphone mounting. It is scheduled for engineering samples in January 2025.

Both chips depend for their efficiency upon a TinyML model fitting entirely within an SRAM-based PiM array. But this is not the only way to exploit PiM architectures for AI, nor is it anywhere near the limits of the technology.

LLMs at the far edge?

One of today’s undeclared grand challenges is to bring generative AI—small language models (SLMs) and even some LLMs—to edge computing. And that’s not just to a powerful PC with AI extensions, but to actual edge devices. The benefit to applications would be substantial: generative AI apps would have greater mobility while being impervious to loss of connectivity. They could have lower, more predictable latency; and they would have complete privacy. But compared to TinyML, this is a different order of challenge.

To produce meaningful intelligence, LLMs require training on billions of parameters. At the same time, the demand for AI inference compute is set to surge, driven by the substantial computational needs of agentic AI and advanced text-to-video generation models like Sora and Veo 2. So, achieving significant advancements in performance, power efficiency, and silicon area (PPA) will necessitate breakthroughs in overcoming the memory wall—the primary obstacle to delivering low-latency, high-throughput solutions.

Figure 3 Here is a view of the layout of Listen VL130 chip, which is capable of processing 32 wake words and keywords while operating in the tens of microwatts, delivering energy efficiency without compromising performance. Source: PIMIC

At this technology crossroads, PiM technology is still important, but to a lesser degree. With these vastly larger matrices, the PiM array acts more like a cache, accelerating matrix multiplication piecewise. But much of the heavy lifting is done outside the PiM array, in a massively parallel dataflow architecture. And there is a further issue that must be resolved.

At the edge, in addition to facilitate model execution, it’s of primary importance to resolve the bandwidth and energy issues that come with scaling to massive memory sizes. Meeting all these challenges can improve an edge chip’s power-performance-area efficiency by more than 15 times.

PIMIC’s studies indicate that models with hundreds of millions to tens of billions of parameters can in fact be executed on edge devices. It will require 5-nm or 3-nm process technology, PiM structures, and most of all a deep understanding of how data moves in generative-AI models and how it interacts with memory.

PiM is indeed a silver bullet for TinyML at the extreme edge. But it’s just one tool, along with dataflow expertise and deep understanding of model dynamics, in reaching the point where we can in fact execute SLMs and some LLMs effectively at the far edge.

Subi Krishnamuthy is the founder and CEO of PIMIC, an AI semiconductor company developing processing-in-memory (PiM) technology for ultra-low-power AI solutions.

 

Related Content

• Getting a Grasp on AI at the Edge
• Tiny machine learning brings AI to IoT devices
• Why MCU suppliers are teaming up with TinyML platforms
• Open-Source Development Comes to Edge AI/ML Applications
• Edge AI: The Future of Artificial Intelligence in embedded systems

The post AI at the edge: It’s just getting started appeared first on EDN.

Aehr Test Systems of Fremont, CA, USA has received an initial production order from a top-tier automotive semiconductor supplier for a FOX-XP wafer-level test and burn-in system with fully integrated FOX WaferPak Aligner for production test of its gallium nitride (GaN) power semiconductor devices. The system was scheduled to ship immediately...

Keysight Technologies announces the expansion of its Novus portfolio with the Novus mini automotive, a quiet small form-factor pluggable (SFP) network test platform that addresses the needs of automotive network engineers as they deploy software defined vehicles (SDV). Keysight is expanding the capability of the Novus platform by offering a next generation vehicle interface that includes 10BASE-T1S, and multi-gigabyte BASE-T1 support for 100 megabytes per second, 2.5 gigabits per second (Gbit/s), 5Gbit/s, and 10Gbit/s. Keysight’s SFP architecture provides a flexible platform to mix and match speeds for each port with modules plugging into existing cards rather than requiring a separate card, as many current test solutions necessitate.

As vehicles move to zonal architectures, connected devices are a critical operational component.  As a result, any system failures caused by connectivity and network issues can impact safety and potentially create life-threatening situations. To mitigate this risk, engineers must thoroughly test the conformance and performance of every system element before deploying them.

Key benefits of the Novus mini automotive platform include:

• Streamlines testing – The combined solution offers both traffic generation and protocol testing on one platform. With both functions on a single platform, engineers can optimize the testing process, save time, and simplify workflows without requiring multiple tools. It also accelerates troubleshooting and facilitates efficient remediation of issues.
• Helps lower costs and simplify wiring – Supports native automotive interfaces BASE-T1 and BASE-T1S that help lower costs and simplify wiring for automotive manufacturers, reducing the amount of required cabling and connectors. BASE-T1 and BASE-T1S offer a scalable and flexible single-pair Ethernet solution that can adapt to different vehicle models and configurations. These interfaces support higher data rates compared to traditional automotive communication protocols for faster, more efficient data transmission as vehicles become more connected.
• Compact, quiet, and affordable – Features the smallest footprint in the industry with outstanding cost per port, and ultra-quiet, fan-less operation.
• Validates layers 2-7 in complex automotive networks– Provides comprehensive performance and conformance testing that covers everything from data link and network protocols to transport, session, presentation, and application layers. Validating the interoperability of disparate components across layers is necessary in complex automotive networks where multiple systems must seamlessly work together.

• Protects networks from unauthorized access – Supports full line rate and automated conformance testing for TSN 802.1AS 2011/2020, 802.1Qbv, 802.1Qav, 802.1CB, and 802.1Qci. The platform tests critical timing standards for automotive networking, as precise timing and synchronization are crucial for the reliable and safe operation of ADAS and autonomous vehicle technologies. Standards like 802.1Qci help protect networks from unauthorized access and faulty or unsecure devices.

Ram Periakaruppan, Vice President and General Manager, Network Test & Security Solutions, Keysight, said: “The Novus mini automotive provides real-world validation and automated conformance testing for the next generation of software defined vehicles. Our customers must trust that their products consistently meet quality standards and comply with regulatory requirements to avoid costly fines and penalties. The Novus mini allows us to deliver this confident assurance with a compact, integrated network test solution that can keep pace with constant innovation.”

Keysight will demonstrate its portfolio of test solutions for automotive networks, including the Novus mini automotive, at the Consumer Technology Show (CES), January 7-10th in Las Vegas, NV, West Hall, booth 4664 (Inside the Intrepid Controls booth).

The post Keysight Expands Novus Portfolio with Compact Automotive Software Defined Vehicle Test Solution appeared first on ELE Times.

Soft soldering is a popular technique in metal joining, known for its simplicity and versatility. It involves the use of a low-melting-point alloy to bond two or more metal surfaces. The process is widely used in electronics, plumbing, and crafting due to its ease of application and the reliability of the joints it produces.

What is Soft Soldering?

Soft soldering refers to the process of joining metals using a filler material, known as solder, that melts and flows at temperatures below 450°C (842°F). Unlike brazing or welding, the base metals are not melted during this process. The bond is achieved by the solder adhering to the surface of the base metals, which must be clean and properly prepared to ensure a strong joint.

The solder typically consists of tin-lead alloys, although lead-free alternatives are now common due to health and environmental concerns. Flux is often used alongside solder to remove oxides from the metal surfaces, promoting better adhesion and preventing oxidation during heating.

How Soft Soldering Works

Soft soldering is a straightforward process that follows these basic steps:

1. Preparation:
   • Clean the surfaces to be joined by removing dirt, grease, and oxidation. This can be done using sandpaper, a wire brush, or chemical cleaners.
   • Apply flux to the cleaned surfaces to prevent oxidation during heating and enhance solder flow.
2. Heating:
   • Utilize a soldering iron, soldering gun, or any appropriate heat source to warm the joint. Make sure the temperature is adequate to liquefy the solder while keeping the base metals intact.
3. Application of Solder:
   • After heating the joint, introduce the solder to the targeted area. The solder will melt and flow into the joint by capillary action, creating a strong bond upon cooling.
4. Cooling:
   • Let the joint cool down gradually without being disturbed. This ensures the integrity of the bond and prevents the formation of weak spots.

Soft Soldering Process 1. Tools and Materials

The essential tools and materials for soft soldering include:

• Soldering iron or gun
• Solder (tin-lead or lead-free)
• Flux
• Cleaning tools (e.g., sandpaper, wire brush)
• Heat-resistant work surface

2. Detailed Steps

1. Surface Preparation: Clean the metal surfaces thoroughly. Apply flux to prevent oxidation and enhance solder adherence.
2. Preheating: Warm the area to ensure uniform heating and improve solder flow.
3. Solder Application: Melt the solder onto the heated joint, ensuring it flows evenly.
4. Inspection: Examine the joint for uniformity and proper adhesion.
5. Cleanup: Remove excess flux residue to prevent corrosion.

Uses and Applications

Soft soldering is widely employed in various industries and applications, including:

1. Electronics:
   • Circuit board assembly
   • Wire connections
   • Repair of electrical components
2. Plumbing:
   • Joining copper pipes
   • Creating watertight seals in plumbing joints for water supply systems
3. Jewellery Making:
   • Crafting and repairing delicate metal items
4. Arts and Crafts:
   • Creating stained glass
   • Assembling small metal models
5. Automotive Repairs:
   • Fixing radiators and other small components

Advantages of Soft Soldering

1. Ease of Use: The process is simple and does not require extensive training.
2. Low Temperature: Operates at lower temperatures, reducing the risk of damaging components.
3. Versatility: Capable of accommodating diverse materials and a variety of applications..
4. Cost-Effective: Requires minimal equipment and materials.
5. Repairability: Joints can be easily reworked or repaired.

Disadvantages of Soft Soldering

1. Weak Joint Strength: The bond is not as strong as those produced by welding or brazing.
2. Temperature Limitations: Joints may fail under high temperatures.
3. Toxicity: Lead-based solders pose health risks, necessitating the use of proper ventilation and safety measures.
4. Corrosion Risk: Residual flux can lead to corrosion if not cleaned properly.
5. Limited Material Compatibility: Not suitable for all types of metals, especially those with high melting points.

Conclusion

Soft soldering remains a valuable technique for joining metals in numerous applications, particularly where ease of use and low-temperature operation are essential. Its advantages make it ideal for delicate tasks in electronics, plumbing, and crafting, while its limitations must be considered when high strength or temperature resistance is required. With advancements in soldering materials and techniques, soft soldering continues to be a reliable and accessible method for metal joining.

The post Soft Soldering Definition, Process, Working, Uses & Advantages appeared first on ELE Times.