GSF, using grouped spatial gating, partitions the input tensor, and consequently, unifies the decomposed parts with channel weighting. Existing 2D CNN architectures can be adapted to extract spatio-temporal features using GSF, demonstrating superior performance with negligible overhead in terms of parameters and computation. We meticulously examine GSF, leveraging two prominent 2D CNN families, and attain state-of-the-art or comparable results across five standard action recognition benchmarks.
The trade-offs inherent in edge inference using embedded machine learning models involve a delicate balancing act between resource metrics, such as energy consumption and memory usage, and performance indicators like computation speed and precision. Departing from traditional neural network approaches, this work investigates Tsetlin Machines (TM), a rapidly developing machine learning algorithm. The algorithm utilizes learning automata to formulate propositional logic rules for classification. Naphazoline agonist We introduce a novel methodology for TM training and inference, leveraging algorithm-hardware co-design. By utilizing independent training and inference techniques for transition machines, the REDRESS methodology seeks to shrink the memory footprint of the resultant automata, facilitating their use in low-power and ultra-low-power applications. The array of Tsetlin Automata (TA) maintains learned information encoded in binary format, where 0 represents excludes and 1 represents includes. REDRESS's novel include-encoding method, designed for lossless TA compression, focuses solely on storing included information, enabling over 99% compression. Serratia symbiotica A novel, computationally economical training process, termed Tsetlin Automata Re-profiling, enhances the accuracy and sparsity of TAs, thereby diminishing the number of inclusions and consequently, the memory burden. The REDRESS inference algorithm, intrinsically bit-parallel and operating on the optimally trained TA within its compressed representation, effectively eliminates decompression during runtime, showcasing significant speed advantages over current-generation Binary Neural Network (BNN) models. This investigation reveals that the REDRESS method yields superior performance for TM models compared to BNN models, achieving better results on all design metrics for five benchmark datasets. Machine learning research frequently utilizes the datasets MNIST, CIFAR2, KWS6, Fashion-MNIST, and Kuzushiji-MNIST. Speedups and energy savings obtained through REDRESS, running on the STM32F746G-DISCO microcontroller, ranged from a factor of 5 to 5700 when contrasted with distinct BNN models.
Image fusion tasks have seen promising results from deep learning-based fusion approaches. The fusion process's results are profoundly influenced by the network architecture's substantial contribution. However, establishing a suitable fusion architecture is frequently difficult, and thus, the design of fusion networks is still a form of applied artistry, not a scientific procedure. To tackle this issue, we mathematically frame the fusion task, and demonstrate a link between its optimal solution and the network architecture capable of executing it. This approach results in the creation of a novel, lightweight fusion network, as outlined in the paper's method. Rather than engaging in a tedious empirical network design process based on trial and error, it employs an alternative technique. We employ a learnable representation approach to the fusion task, the structure of the fusion network being determined by the optimization algorithm that creates the learnable model. The low-rank representation (LRR) objective serves as the cornerstone of our learnable model. Central to the solution, the matrix multiplications are converted into convolutional operations, and the iterative optimization process is replaced by a specialized feed-forward network architecture. A lightweight end-to-end fusion network is implemented based on this novel network architecture, combining infrared and visible light images. Its successful training hinges upon a detail-to-semantic information loss function, meticulously designed to maintain the image details and augment the significant characteristics of the original images. Experiments performed on public datasets show that the proposed fusion network achieves superior fusion performance relative to the prevailing state-of-the-art fusion methods. Our network, interestingly, utilizes a smaller quantity of training parameters than other existing methods.
Deep learning models for visual tasks face the significant challenge of long-tailed data, requiring the training of well-performing deep models on a large quantity of images exhibiting this characteristic class distribution. Deep learning, in its prominence over the last decade, has emerged as a formidable recognition model for learning and acquiring high-quality image representations, marking notable progress in the domain of generic visual recognition. Nonetheless, the problem of class imbalance, a frequent challenge in real-world visual recognition tasks, frequently limits the usability of deep learning-based recognition models, as these models tend to be biased towards the more common classes and underperform on less prevalent classes. A plethora of studies have been performed in recent years to address this concern, showcasing encouraging strides in the field of deep long-tailed learning. Due to the substantial progress in this area, this paper undertakes a detailed examination of recent breakthroughs in the realm of deep long-tailed learning. We have segmented existing deep long-tailed learning research into three key groups: class re-balancing, data augmentation, and module improvement. Our subsequent analysis will thoroughly examine these approaches within this organizational framework. We then empirically investigate several leading-edge methods, scrutinizing their handling of class imbalance based on a newly proposed evaluation metric: relative accuracy. Hip flexion biomechanics To conclude the survey, we emphasize the significant applications of deep long-tailed learning and pinpoint prospective research avenues.
Objects in the same visual field exhibit a spectrum of interconnections, but only a limited portion of these connections are noteworthy. In the light of the Detection Transformer's exceptional object detection skills, we perceive scene graph generation as a task focused on predicting sets. In this research paper, a novel scene graph generation model, Relation Transformer (RelTR), is proposed, leveraging an encoder-decoder architecture. The visual feature context is processed by the encoder, and the decoder, utilizing varied attention mechanisms, infers a fixed-size set of subject-predicate-object triplets employing coupled subject and object queries. In the context of end-to-end training, a set prediction loss is constructed for the purpose of aligning predicted triplets with their respective ground truth values. Differing from conventional scene graph generation methods, RelTR implements a one-step procedure to predict sparse scene graphs, utilizing only visual input and avoiding the integration of entities and the comprehensive labeling of all potential predicates. Through extensive experiments on the Visual Genome, Open Images V6, and VRD datasets, we observe the superior performance and fast inference of our model.
Many vision applications heavily rely on the identification and description of local features, meeting considerable industrial and commercial demands. These tasks, within the context of large-scale applications, impose stringent demands on the precision and celerity of local features. Local feature learning studies are often preoccupied with the isolated descriptors of keypoints, failing to account for the interconnectedness of these keypoints as determined from a comprehensive global spatial awareness. This paper introduces AWDesc, characterized by a consistent attention mechanism (CoAM), thereby granting local descriptors the capacity for image-level spatial awareness in both their training and matching stages. Local feature detection, enhanced by a feature pyramid, is employed to achieve more stable and accurate localization of keypoints. Addressing varying needs for accuracy and speed in describing local features, we offer two versions of AWDesc. Context Augmentation addresses the inherent locality limitation of convolutional neural networks by injecting non-local contextual information, enabling local descriptors to perceive a wider range of information and thus describe better. The Adaptive Global Context Augmented Module (AGCA) and the Diverse Surrounding Context Augmented Module (DSCA) are innovative modules for building robust local descriptors, enriching them with global and surrounding context information. Instead, an ultra-lightweight backbone network, paired with the suggested knowledge distillation strategy, provides the optimal trade-off between speed and accuracy. In addition, we execute extensive experiments on image matching, homography estimation, visual localization, and 3D reconstruction tasks, and the results clearly indicate that our method exhibits superiority over current state-of-the-art local descriptors. On the platform GitHub, the project AWDesc has its code accessible at https//github.com/vignywang/AWDesc.
The establishment of consistent associations between points within separate point clouds is vital for 3D vision tasks, such as registration and object recognition. Employing a mutual voting mechanism, we present a technique for ranking 3D correspondences in this paper. The key to trustworthy scoring results in a mutual voting scheme for correspondences lies in the simultaneous improvement of both the candidates and the voters. A graph is generated using the initial correspondence set and applying the pairwise compatibility restriction. Subsequently, nodal clustering coefficients are employed to initially identify and remove a segment of outlier data points, thereby expediting the subsequent voting operation. Graph nodes are represented as candidates and edges as voters in our third model. Mutual voting within the graph ultimately determines the scoring of correspondences. Ultimately, the correspondences are ordered by their voting scores, with the highest-scoring ones designated as inliers.