Tae-hwan Kim

This study proposes a novel technique to improve the efficiency of the inference based on binary neural networks (BNNs). In the proposed technique, the threshold, used to binarize feature elements, is adaptively scaled by considering the spatial localities inherent in feature maps, to skip redundant operations without degrading inference accuracy. A BNN inference processor supporting the proposed technique is designed and implemented, and the effectiveness of the proposed technique is evaluated with the implementation results of the processor. The proposed technique reduces the overall latency by 17.0%, while the inference accuracy is degraded by less than 1.56% for the CIFAR10 classification task. The resource efficiency is enhanced by up to 36.7%.

With the growing demand for efficient deployment of deep learning models on resource-constrained devices, designing neural network processors that balance performance, resource usage, and speed has become a critical challenge. In this context, an efficient low-resource inference processor for recurrent neural networks (RNNs) is presented. The RNN models are quantized, representing each weight by a single bit with a power-of-two magnitude. The inference process is performed on a per-lane basis to reduce resource usage, with speculative pruning of some operations to increase speed. The microarchitecture is designed to execute threads across lanes using an interleaved scheduling, enhancing efficiency. The functionality of the proposed processor is verified under a fully-integrated FPGA-based inference system. For the Penn Treebank task with a binary-weight long short-term memory model, the proposed processor achieves a perplexity of 72.19, which is close to the quality result without pruning, while achieving a 1.56× speedup through pruning. The LUT resource usage is as low as 1.44k, with an efficiency of 2.415 step/s/LUT.

게임 제작에서 알고리즘 성능은 게임의 전반적인 품질과 사용자 경험에 많은 영향을 준다. 최적화된 알고리즘은 높은 프레임 속도를 유지하게 하고, 부드럽게 실행되게 하여 사용자에게 더 나은 경험과 몰입감을 제공한다. 게임은 그래픽, 음향, 인공지능, 네트워크, 물리 등 다양한 기술적 요소들을 포함하고 있고, 각각의 요소들은 많은 자원을 소모하기 때문에, 요소들의 효율적인 구현은 게임의 재미 증진과 현실감에 중요한 역할을 한다. 이징 함수는 시간 흐름에 따른 매개변수의 변화율로 게임 애니메이션의 움직임을 더욱 부드럽고 자연스럽게 표현할 수 있는 도구로 시각적 효과에 자주 사용된다. 본 연구에서는 게임 애니메이션 연출을 위한 이징 함수의 성능을 최적화하는 방법을 제시하였고, 이징 함수의 일반적인 구현 방식과 양자화 기법을 적용한 구현 방식을 비교 분석하였다. 실험결과, PC 성능에 따라 구분된 실험 환경에서 31종의 이징 함수는 평균 45%에서 81%까지 향상된 결과를 보였으며, PC 성능보다는 이징 함수 수식의 복잡도에 따라 연산량이 많고, 복잡한 수식일수록 성능향상이 높게 나타났다.

An efficient processor to perform inference of binary neural networks (BNNs) is presented. The proposed processor, named BiNPU, is designed based on a unified architecture that can efficiently process BNN modules of various types, including those with group convolution and global average pooling, in a consistent output-parallel mechanism without resource overhead. Implemented in a 28 nm FPGA, BiNPU shows the resource efficiency as high as 33.0 MOP/s/LUT, 35.3% higher than the previous state-of-the-art processor that supports even fewer module types. BiNPU performs the CIFAR10 classification task achieving 88.26% accuracy with 1.9 Mbit parameters entirely stored in on-chip memories. The BRAM usage for implementing the on-chip memories is rather smaller than those of the previous processors stored some of the parameters in off-chip memories.

Abstract This letter presents a novel technique to optimize latency and accuracy for the inference based on binary neural network (BNN). The effects of the spatial locality in feature maps on latency and accuracy are analyzed in the BNN inference with the previous operation‐skipping method. A regularization‐based technique is proposed to adjust the locality with the aim of further reducing latency and improving accuracy of the previous method. In the CIFAR10 classification task, 11.62% latency reduction or 0.77% accuracy increase can be achieved when optimizing for each individually. When optimizing for both simultaneously, 5.58% latency reduction and 0.59% accuracy increase can be achieved.

This paper presents an efficient error-correction technique aiming at error-resilient inference for binary neural networks. The proposed technique is designed to achieve error-resilience by recompute-based error correction against transient errors potentially encountered in inference systems. The recompute is performed selectively by exploiting the spatial locality in a feature map to minimize the overhead. For the CIFAR10 classification task, the BNN inference with the proposed technique achieves an accuracy of 80.97% at a bit-error rate of 3%, which is 41.48% higher than that achieved without error resilience. The compute overhead is 64.61% of that of the conventional triple recompute technique in terms of latency.

This brief presents an efficient binary neural network inference processor, that is, resilient to soft errors caused by potential circuit faults. The proposed processor, SR-BIP, achieves error resilience based on a recompute-based error correction technique. The recompute is selectively performed by exploiting spatial locality inherent in a feature map, to minimize overhead. For the CIFAR10 task at 0.1% bit error rate, SR-BIP achieves 84.42% accuracy, which is 17.59% higher than that without any error resilience. Despite this error resilience, SR-BIP exhibits a resource efficiency of 75.27 MOP/s/LUT in a 28-nm FPGA, which is as high as that of the previous state-of-the-art processor designed without considering error resilience.

With the increasing adoption of the SDMX (Statistical Data and Metadata Exchange) standard by major national statistical institutes and public authorities, researchers and general users now have greater access to high-quality and reliable statistical data. However, collecting and managing data from various SDMX open data sources presents significant challenges. Firstly, despite operating under the same version of the SDMX standard, discrepancies in data exchange formats exist across different SDMX web services. Secondly, SDMX data requires accompanying metadata for accurate interpretation, yet there has been a lack of robust consideration for a data model capable of concurrently storing both statistical value and its associated metadata. To address these challenges, we propose an scalable data collection framework for SDMX. Structurally, the proposed framework is designed as a flexible and scalable architecture that can be seamlessly extended to target various SDMX open data sources. By deploying dedicated response parsers with standardized in/out interfaces, it can dynamically accommodate a wide range of data sources, providing a scalable solution for diverse statistical data collection. It retrieves data from SDMX open data sources and constructs a integrated SDMX data model within local systems. This model facilitates the retrieval, storage, and management of statistical data while preserving the integrity of the Data Structure Definitions (DSD) as specified by data providers. Additionally, our framework offers advanced data management capabilities, enabling users to efficiently request data CRUD (Collect, Read, Update, and Delete). We validated the functionality and efficacy of the framework by applying it to several prominent SDMX web services.

최근 대두되는 도심항공교통의 추세에 따라 항공기는 소형화되고 소프트웨어 집약적 시스템으로 변화하고 있다. 이에 따라 항공기 안전성 평가 프로세스인 ARP4761 중심의 안전성 분석 기법 및 표준도 소프트웨어를 중심으로 변화할 필요가 있다. 본 논문에서는 소프트웨어 집약적 시스템인 항공 모빌리티의 안전성 분석을 위해, STPA 방법과 FTA와 FMEA의 안전성 분석 기법을 연계하였다. 또한 STPA, FTA, FMEA의 연계 가능성과 효과를 파악하기 위하여 소형 항공기의 충돌 회피 소프트웨어를 중심으로 안전성 분석 사례를 진행하였다. 이러한 제안 방법과 사례 분석을 통해, STPA, FTA, 그리고 FMEA 안전성 기법의 상호 연계 효과를 확인하고 안전 요구사항을 도출하였다.

본 논문은 장단기 메모리 (Long Short-Term Memory; LSTM)의 추론 과정을 효율적으로 처리하는 방법을 제안한다. 제안하는 방법은 추론 과정 중 중간 연산 결과를 미리 예측하여 불필요한 연산을 생략함으로써 LSTM의 추론 과정을 효율적으로 처리한다. Non-Zero Pruning을 기반으로 근사 추론을 수행하면서 연산 생략 효과를 높였으며 제안하는 방법의 효율적인 적용을 위해 전체 연산을 다수의 레인으로 구성된 벡터 단위가 아닌 단일 레인 단위로 처리한다. 제안하는 방법을 적용한 LSTM 모델은 Sequential MNIST 태스크 및 IMDB 리뷰 감성 분석 태스크에서 각각 11.44% － 37.37%, 1.07% － 38.52% 연산량 감소 효과를 보이며 정확도의 하락은 각각 최대 3.41%, 2.54%로 미미하다.

This paper presents a real-time keyword spotting system in an FPGA. The proposed system performs the entire KWS task based on a binary convolutional neural network (BCNN) without involving any other complicated processing. The BCNN inference is efficiently carried out by skipping redundant operations. With all the essential components integrated, the proposed system has been implemented with only 8475 look-up tables in an FPGA. The proposed system processes one-second frame in 19.8 ms, exhibiting the spotting accuracy of 91.64%.

This paper proposes a resource-efficient keyword spotting (KWS) system based on a convolutional neural network (CNN). The end-to-end KWS process is performed based solely on 1D-CNN inference, where features are first extracted from a few convolutional blocks, and then the keywords are classified using a few fully connected blocks. The 1D-CNN model is binarized to reduce resource usage, and its inference is executed by employing a dedicated engine. This engine is designed to skip redundant operations, enabling high inference speed despite its low complexity. The proposed system is implemented using 6895 ALUTs in an Intel Cyclone V FPGA by integrating the essential components for performing the KWS process. In the system, the latency required to process a frame is 22 ms, and the spotting accuracy is 91.80% in an environment where the signal-to-noise ratio is 10 dB for Google speech commands dataset version 2.

Spin transfer torque MRAM (STT-MRAM) based digital in-memory computing (IMC) has been recently proposed for energy efficient processing of convolutional neural network (CNN). The conventional MRAM based IMC architecture suffers from excessive storage area since a large number of intermediate sum and carry bits should be stored for the following successive additions during multiply-accumulate (MAC) operations. In this paper, we propose an area efficient bi-directional MRAM digital IMC (BiMDiM) scheme, where the size of memory cells storing the intermediate sums and carries can be efficiently reduced by repetitively using the same memory cells during MAC operations. In addition, to reduce the number of inefficient half-additions, which can process only two inputs with almost same hardware cost, the addition re-scheduling is also presented to further improve the energy and latency of BiMDiM. The proposed BiMDiM architecture has been simulated using 28nm CMOS process. When compared to the baseline architecture, the proposed BiMDiM improves area efficiency up to 53%.

Recurrent neural networks (RNNs) are extensively employed to perform inference based on the temporal features of the input data. However, their computational workload and power consumption involved in inference are prohibitively high in practice, which may be problematic to achieve a high-speed inference in devices with tight limitations in the available silicon resources and power supply. This paper presents an efficient inference processor for RNNs, named ROSETTA. ROSETTA supports multiple data formats programmable for each vector operand to achieve a wide range or high precision with a limited data size. ROSETTA consistently performs every vector operation based on homogeneous processing units with a high utilization rate. Moreover, ROSETTA skips operations and reduces memory accesses to achieve high energy efficiency by pruning the activation elements in a fine-grained manner. Implemented in a low-cost 28 nm field-programmable gate array, ROSETTA exhibits a resource and energy efficiency as high as 2.51 – 1.14 MOP/s/LUT and 434.01 – 113.29 GOP/s/W, respectively, while producing near-floating-point inference results. The resource and energy efficiency of ROSETTA are higher than those of the previous processor implemented in the same device by up to 206.1% and 304.0%, respectively. The functionality has been verified for several RNN models of various types under a fully-integrated inference system.

A binary convolutional neural network (BCNN) is a neural network promising to realize analysis of visual imagery in low-cost resource-limited devices. This study presents an efficient inference processor for BCNNs, named TORRES. TORRES performs inference efficiently, skipping operations based on the spatial locality inherent in feature maps. The training process is regularized with the objective of skipping more operations. The microarchitecture is designed to skip operations and generate addresses efficiently with low resource usage. A prototype inference system based on TORRES has been implemented in a 28 nm field-programmable gate array, and its functionality has been verified for practical inference tasks. Implemented with 2.31 K LUTs, TORRES achieves the inference speed of 291.2 GOP/s, exhibiting the resource efficiency of 126.06 MOP/s/LUT. The resource efficiency of TORRES is 1.45 times higher than that of the state-of-the-art work.

Abstract With an increasing demand for artificial intelligence, the emulation of the human brain in neuromorphic computing has led to an extraordinary result in not only simulating synaptic dynamics but also reducing complex circuitry systems and algorithms. In this work, an artificial electronic synaptic device based on a synthesized MoS2 memristor array (4 × 4) is demonstrated; the device can emulate synaptic behavior with the simulation of deep neural network (DNN) learning. MoS2 film is directly synthesized onto a patterned bottom electrode (Pt) with high crystallinity using sputtering and CVD. The proposed MoS2 memristor exhibits excellent memory operations in terms of endurance (up to 500 sweep cycles) and retention (~ 104) with a highly uniform memory performance of crossbar array (4 × 4) up to 16 memristors on a scalable level. Next, the proposed MoS2 memristor is utilized as a synaptic device that demonstrates close linear and clear synaptic functions in terms of potentiation and depression. When providing consecutive multilevel pulses with a defined time width, long-term and short-term memory dynamics are obtained. In addition, an emulation of the artificial neural network of the presented synaptic device showed 98.55% recognition accuracy, which is 1% less than that of software-based neural network emulations. Thus, this work provides an enormous step toward a neural network with a high recognition accuracy rate.

본 논문은 파라미터 수가 적은 경량의 이진화된 신경망 모델을 제안한다. 이진화된 신경망 모델의 파라미터를 줄이는 방법으로 Depthwise-separable Convolution, Grouped Convolution, Global Average Pooling기법을 이진화된 신경망의 기본 블록의 연산 구조에 적합하도록 수정하였다. 이에 따라 기본 블록의 모든 입출력을 이진화된 형태로 유지하고, 추론과 관련된 모든 연산을 이진화된 영역에서 수행하면서도 위와 같은 기법들로 인한 장점들을 효과적으로 달성할 수 있게 되었다. SVHN과 CIFAR-10 이미지 분류 문제를 위한 현실적인 모델에 대하여 제안한 기법을 적용한 결과 전체 파라미터의 수를 82.1%, 82.2% 줄일 수 있었고, 각각의 데이터 세트에 대해 1.7%, 1.8%의 정확도의 하락을 보였다.

An efficient inference processor for recurrent neural networks is designed and implemented in an FPGA. The proposed processor is designed to be reconfigurable for various models and perform every vector operation consistently utilizing a single array of multiply-accumulate units with the aim of achieving a high resource efficiency. The data format is programmable per operand. The resource and energy efficiency are 1.89MOP/LUT and 263.95GOP/J, respectively, in Intel Cyclone-V FPGA. The functionality has been verified successfully under a fully-integrated inference system.

This letter presents an efficient technique to reduce the computational complexity involved in training binary convolutional neural networks (BCNN). The BCNN training shall be conducted focusing on the optimization of the sign of each weight element rather than the exact value itself in convention; in which, the sign of an element is not likely to be flipped anymore after it has been updated to have such a large magnitude to be clipped out. The proposed technique does not update such elements that have been clipped out and eliminates the computations involved in their optimization accordingly. The complexity reduction by the proposed technique is as high as 25.52% in training the BCNN model for the CIFAR-10 classification task, while the accuracy is maintained without severe degradation.

Processing-In-Memory (PIM) 기술 기반의 효율적인 구현을 목적으로 새로운 Binary Neural Network 모델을 제안한다. PIM 기술 구현을 위해 메모리 내부에 Analog-To-Digital Converter (ADC) 를 내장할 필요가 있는데, 해당 ADC는 Truncation 현상이 발생하지 않도록 충분히 높은 해상도를 가져야 한다. 제안하는 모델은 해상도 요구조건을 만족하지 못하는 ADC에서도 높은 성능을 발생할 수 있도록, ADC 내부에서 발생하는 Truncation 현상을 고려하여 설계되었다. Back Propagation 과정에서 다항 함수 형태의 새로운 전달함수로 미분 불가능한 Truncation의 전달함수를 대체했다. SVHN과 CIFAR-10 분류에 대해 Truncation현상을 고려하지 않은 모델 대비 각각 최대 13.97%, 30.41%의 정확도 향상을 보인다. 제안하는 모델을 구현하기 위해 별도의 회로를 추가적으로 요구되지 않기 때문에 메모리 밀집도의 저하를 최소화할 수 있다.

Logic device for AI-inference needs high band width and low latency characteristics to increase the response speed. In order to overcome the size limitation of a single logic chip and secure these characteristics, it is inevitable to separate the SRAM function to increase the memory capacity and apply a 3D package structure that directly stacks with logic. The structure of stacking logic and memory can be implemented in four cases; face to face and back to face (B2F), Logic on SRAM and SRAM on Logic. Among them, thermal characteristics in SRAM on Logic with B2F are not stronger than other structures because in a server environment where most of the heat is forcibly discharged through the cooler installed on the top of package, a lot of heat generated from the logic front side does not go directly to the cooler through Si alone, but passes through the micro-bump bonding layer and the entire SRAM chip. In this study, it was presented that a detailed method for reducing the thermal resistance of the micro-bump junction in order to improve the thermal characteristics in the SRAM on Logic stack package structure. Test vehicle consisted of top chip (93mm2) and bottom chip (103mm2) with micro-bump connections of under 40 μm in pitch and under 20 μm in diameter. The main influence factors were analyzed in terms of the joint structure, material, and layout design, and thermal resistance was measured and compared after achieving actual package to confirm exactly the effect of each major factor on reducing package thermal resistance.

Abstract Spin-orbit coupling effect in structures with broken inversion symmetry, known as the Rashba effect, facilitates spin-orbit torques (SOTs) in heavy metal/ferromagnet/oxide structures, along with the spin Hall effect. Electric-field control of the Rashba effect is established for semiconductor interfaces, but it is challenging in structures involving metals owing to the screening effect. Here, we report that the Rashba effect in Pt/Co/AlO x structures is laterally modulated by electric voltages, generating out-of-plane SOTs. This enables field-free switching of the perpendicular magnetization and electrical control of the switching polarity. Changing the gate oxide reverses the sign of out-of-plane SOT while maintaining the same sign of voltage-controlled magnetic anisotropy, which confirms the Rashba effect at the Co/oxide interface is a key ingredient of the electric-field modulation. The electrical control of SOT switching polarity in a reversible and non-volatile manner can be utilized for programmable logic operations in spintronic logic-in-memory devices.