HighlightCurated by Aramai Editorial

Deep visual-semantic alignments for generating image descriptions

Presents a model that aligns image regions with sentence fragments to generate natural language descriptions of images and their regions.

This work introduces a model that generates natural-language descriptions of images and their regions by learning correspondences between language and visual data. Its alignment model combines CNNs over image regions, bidirectional RNNs over sentences, and a structured objective linking the two via a multimodal embedding. A Multimodal RNN then uses the inferred alignments to generate novel region descriptions. The alignment model sets state-of-the-art retrieval results on Flickr8K, Flickr30K, and MSCOCO, and generated descriptions beat retrieval baselines.

Based on: Deep visual-semantic alignments for generating image descriptions · Computer Vision and Pattern Recognition

HighlightCurated by Aramai Editorial

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

Introduces an efficient sub-pixel convolutional network that extracts features in low-resolution space for real-time image and video super-resolution.

Prior deep super-resolution methods first upscale the low-resolution input with a fixed bicubic filter and reconstruct in high-resolution space, which the authors show is sub-optimal and costly. This paper presents the first CNN to super-resolve 1080p video in real time on a single K2 GPU by extracting feature maps in low-resolution space. An efficient sub-pixel convolution layer then learns an array of upscaling filters to produce the high-resolution output. It gains +0.15dB on images and +0.39dB on videos while running an order of magnitude faster than prior CNNs.

Based on: Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network · Computer Vision and Pattern Recognition

HighlightCurated by Aramai Editorial

A Survey on Bias and Fairness in Machine Learning

Surveys sources of bias in AI systems and builds a taxonomy of fairness definitions and mitigation approaches across machine learning subdomains.

As AI systems increasingly make consequential decisions in sensitive settings, ensuring they do not discriminate against particular groups has become critical. This survey examines real-world applications that exhibited bias and catalogs the sources of bias that can affect AI. It organizes proposed fairness definitions into a taxonomy and reviews how various AI domains have observed unfair outcomes and tried to address them. The authors highlight open directions to motivate further work on mitigating bias.

Based on: A Survey on Bias and Fairness in Machine Learning · ACM Computing Surveys

HighlightCurated by Aramai Editorial

Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments

Presents a multi-agent actor-critic method that conditions on other agents' policies to learn coordination in mixed cooperative-competitive settings.

Deep reinforcement learning in multi-agent domains is hard: Q-learning faces environment non-stationarity, and policy gradients suffer variance that grows with the number of agents. The authors adapt actor-critic methods so each agent considers other agents' action policies, learning behaviors that require complex coordination. Training with an ensemble of policies per agent yields more robust results. In both cooperative and competitive scenarios, agent populations discover physical and informational coordination strategies, beating existing methods.

Based on: Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments · Neural Information Processing Systems

HighlightCurated by Aramai Editorial

Prefix-Tuning: Optimizing Continuous Prompts for Generation

Proposes prefix-tuning, a lightweight alternative to fine-tuning that freezes the language model and optimizes continuous task-specific prefix vectors.

Fine-tuning adapts large pretrained language models but modifies all parameters, requiring a full model copy per task. Prefix-tuning instead keeps the model frozen and optimizes a small sequence of continuous, task-specific vectors, the prefix, that later tokens attend to as virtual tokens. Applied to GPT-2 for table-to-text and BART for summarization, it learns only 0.1% of the parameters yet matches full fine-tuning with full data, beats it in low-data settings, and extrapolates better to unseen topics.

Based on: Prefix-Tuning: Optimizing Continuous Prompts for Generation · Annual Meeting of the Association for Computational Linguistics

HighlightCurated by Aramai Editorial

R-FCN: Object Detection via Region-based Fully Convolutional Networks

Introduces region-based fully convolutional networks (R-FCN) that share computation across the image for accurate, efficient object detection.

R-FCN is a region-based, fully convolutional detector that shares nearly all computation across the whole image, unlike Fast/Faster R-CNN which run a costly per-region subnetwork many times. Position-sensitive score maps reconcile translation-invariance in classification with translation-variance in detection, letting the model adopt fully convolutional backbones like ResNets. On PASCAL VOC 2007 it reaches 83.6% mAP with a 101-layer ResNet at 170ms per image, 2.5-20x faster than Faster R-CNN. Code is publicly released.

Based on: R-FCN: Object Detection via Region-based Fully Convolutional Networks · Neural Information Processing Systems

HighlightCurated by Aramai Editorial

Deep Reinforcement Learning from Human Preferences

Shows how to train reinforcement learning agents from non-expert human preferences over trajectory-segment pairs, without a reward function.

Communicating complex goals to reinforcement learning systems is hard. This work defines goals via non-expert human preferences between pairs of trajectory segments, and shows the approach can solve complex RL tasks, including Atari games and simulated robot locomotion, without access to the reward function. Feedback is required on less than one percent of the agent's interactions, reducing human oversight cost enough for practical use on state-of-the-art systems. The method can also train novel complex behaviors with about an hour of human time.

Based on: Deep Reinforcement Learning from Human Preferences · Neural Information Processing Systems

HighlightCurated by Aramai Editorial

Improved Denoising Diffusion Probabilistic Models

Improves denoising diffusion probabilistic models with simple modifications for competitive log-likelihoods, faster sampling, and better scaling.

Denoising diffusion probabilistic models (DDPMs) produce excellent samples, and a few simple modifications also let them achieve competitive log-likelihoods while keeping high sample quality. Learning the variances of the reverse diffusion process enables sampling with an order of magnitude fewer forward passes at negligible quality cost, aiding deployment. Using precision and recall, the authors compare how well DDPMs and GANs cover the target distribution. Sample quality and likelihood scale smoothly with model capacity and training compute.

Based on: Improved Denoising Diffusion Probabilistic Models · International Conference on Machine Learning

HighlightCurated by Aramai Editorial

OpenAI Gym

Describes OpenAI Gym, a toolkit for reinforcement learning research offering benchmark problems with a common interface and a results-sharing site.

OpenAI Gym is a toolkit for reinforcement learning research. It provides a growing collection of benchmark problems that share a common interface, along with a website where people can share results and compare algorithm performance. The whitepaper describes the components of OpenAI Gym and the design decisions behind the software.

Based on: OpenAI Gym · arXiv.org

HighlightCurated by Aramai Editorial

ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes

Introduces ScanNet, an RGB-D video dataset of 2.5M views across 1513 indoor scenes with 3D poses, surface reconstructions, and semantic labels.

Supervised deep learning needs large labeled datasets, but RGB-D scene understanding data is scarce, covering few views with limited annotations. ScanNet is an RGB-D video dataset of 2.5M views across 1513 scenes, annotated with 3D camera poses, surface reconstructions, and semantic segmentations. The authors built a scalable, easy-to-use capture pipeline with automated surface reconstruction and crowdsourced annotation. Using this data achieves state-of-the-art performance on 3D object classification, semantic voxel labeling, and CAD model retrieval.

Based on: ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes · Computer Vision and Pattern Recognition

HighlightCurated by Aramai Editorial

Longformer: The Long-Document Transformer

Introduces Longformer, a transformer whose attention scales linearly with sequence length to process documents of thousands of tokens.

Standard transformers cannot process long sequences because self-attention scales quadratically with length. Longformer introduces attention that scales linearly, combining local windowed attention with task-motivated global attention as a drop-in replacement for self-attention. It reaches state-of-the-art results on character-level language modeling and, when pretrained and finetuned, consistently outperforms RoBERTa on long-document tasks, with new records on WikiHop and TriviaQA. A Longformer-Encoder-Decoder variant supports generative tasks like arXiv summarization.

Based on: Longformer: The Long-Document Transformer · arXiv.org

HighlightCurated by Aramai Editorial

Robust principal component analysis?

Proves that a matrix that is the sum of a low-rank and a sparse component can be exactly recovered by convex Principal Component Pursuit.

The paper asks whether a data matrix formed as the superposition of a low-rank component and a sparse component can be decomposed into its parts. Under suitable assumptions, it proves both can be recovered exactly by solving a convex program, Principal Component Pursuit, which minimizes a weighted combination of the nuclear norm and the l1 norm. This yields a principled robust PCA that recovers principal components even when a fraction of entries are arbitrarily corrupted or missing, with applications in video surveillance and face recognition.

Based on: Robust principal component analysis? · JACM