HighlightCurated by Aramai Editorial

Billion-Scale Similarity Search with GPUs

A GPU design for k-selection that accelerates exact, approximate, and compressed similarity search, enabling billion-scale nearest-neighbor graphs.

Similarity search over complex data such as images and videos needs high-dimensional features and specialized indexing. This paper improves GPU use: GPUs excel at parallel distance computation, but prior methods were limited by low-parallelism k-min selection or poor memory use. They propose a new k-selection design applied to brute-force, approximate, and product-quantization compressed search, beating prior art by wide margins. It builds a k-NN graph over 95M Yfcc100M images in 35 minutes and connects 1 billion vectors in under 12 hours on four Titan X GPUs.

Based on: Billion-Scale Similarity Search with GPUs · IEEE Transactions on Big Data

HighlightCurated by Aramai Editorial

Sentiment Analysis and Opinion Mining

A comprehensive survey and introductory text on sentiment analysis and opinion mining in natural language processing.

This book is a comprehensive survey of sentiment analysis and opinion mining, the study of people's opinions, sentiments, attitudes, and emotions in written language. An active NLP area, it also spans data, Web, and text mining and has spread to management and social sciences due to its business and societal importance. Its growth parallels the rise of social media such as reviews, blogs, and Twitter. The text covers key topics and recent developments with over 400 references, suited to students, researchers, and practitioners.

Based on: Sentiment Analysis and Opinion Mining · Synthesis Lectures on Human Language Technologies

HighlightCurated by Aramai Editorial

DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks

Presents DeepFool, an algorithm that efficiently computes minimal adversarial perturbations to quantify and improve deep classifier robustness.

State-of-the-art deep networks excel at image classification yet are unstable to small, carefully chosen input perturbations. Despite its significance, no effective method existed to accurately measure classifier robustness to such perturbations on large-scale datasets. The paper fills this gap with DeepFool, an algorithm that efficiently computes perturbations that fool deep networks and reliably quantifies their robustness. Extensive experiments show it outperforms recent methods at computing adversarial perturbations and making classifiers more robust.

Based on: DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks · Computer Vision and Pattern Recognition

HighlightCurated by Aramai Editorial

Fast Graph Representation Learning with PyTorch Geometric

Introduces PyTorch Geometric, a PyTorch library for deep learning on graphs, point clouds, and manifolds with fast sparse GPU operations.

PyTorch Geometric is a library, built on PyTorch, for deep learning on irregularly structured data such as graphs, point clouds, and manifolds. Alongside general graph data structures and processing routines, it implements many recently published methods from relational learning and 3D data processing. It attains high throughput via sparse GPU acceleration, dedicated CUDA kernels, and efficient mini-batching of variable-sized inputs. The paper presents the library and reports a comparative study of the implemented methods under homogeneous evaluation settings.

Based on: Fast Graph Representation Learning with PyTorch Geometric · arXiv.org

HighlightCurated by Aramai Editorial

Federated Learning: Strategies for Improving Communication Efficiency

Proposes structured and sketched update methods that cut federated learning's uplink communication cost by up to two orders of magnitude.

Federated learning trains a central model while data stays on many clients with slow, unreliable links, so communication efficiency is key. Each round, clients compute updates on local data and a server aggregates them into a new global model. To reduce uplink cost, it proposes structured updates, learned in a restricted low-parameter space such as low-rank or random-mask, and sketched updates, a full update compressed via quantization, random rotations, and subsampling. Experiments on convolutional and recurrent nets cut communication by two orders of magnitude.

Based on: Federated Learning: Strategies for Improving Communication Efficiency · arXiv.org

HighlightCurated by Aramai Editorial

Comparative Protein Structure Modeling Using MODELLER

A protocol for comparative (homology) protein structure modeling with MODELLER, covering the full pipeline from alignment to model evaluation.

Comparative protein structure modeling predicts the 3D structure of a target sequence chiefly from its alignment to template proteins of known structure. The workflow has four stages: fold assignment, target-template alignment, model building, and model evaluation. This protocol shows how to compute models with the MODELLER program and use the ModBase database, discussing each stage, common errors, and applications. Modeling lactate dehydrogenase from Trichomonas vaginalis (TvLDH) is the worked example, and software installation is also covered.

Based on: Comparative Protein Structure Modeling Using MODELLER · Current Protocols in Bioinformatics

HighlightCurated by Aramai Editorial

Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing

Surveys and organizes prompt-based learning in NLP, offering unified notation and a typology of prompting methods, models, and tuning strategies.

This article surveys prompt-based learning in NLP. Rather than training a model to map input x to output y as P(y|x), it uses language models over text: a template reshapes the input into a prompt with empty slots the LM fills, and the answer is read from the completed string. Because the LM is pre-trained on massive raw text, defining new prompts enables few-shot or zero-shot adaptation with little or no labeled data. The authors give unified notation, organize work by choice of pre-trained model, prompt, and tuning strategy, and release supporting resources.

Based on: Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing · ACM Computing Surveys

HighlightCurated by Aramai Editorial

CIDEr: Consensus-based image description evaluation

Introduces CIDEr, a consensus-based automatic metric for evaluating image descriptions, plus new annotation methods and datasets.

Evaluating automatically generated image descriptions remains difficult despite progress in vision and language tasks. This work proposes a human-consensus paradigm with three parts: a triplet-based annotation method for measuring consensus, an automated metric that captures it, and two datasets, PASCAL-50S and ABSTRACT-50S, each with 50 sentences per image. The metric matches human judgment better than prior measures, and five leading methods are benchmarked. A variant, CIDEr-D, ships with the MS COCO evaluation server.

Based on: CIDEr: Consensus-based image description evaluation · Computer Vision and Pattern Recognition

HighlightCurated by Aramai Editorial

KEGG for linking genomes to life and the environment

KEGG, a database integrating genomic, chemical, and functional information, links genomes to life via PATHWAY mapping and to the environment via BRITE.

KEGG is a database integrating genomic, chemical, and systemic functional information. It is a reference knowledge base linking genomes to life through PATHWAY mapping, which maps genomic or transcriptomic gene content onto reference pathways to infer systemic behavior of a cell or organism. It also links genomes to the environment via BRITE mapping, an ontology of functional hierarchies over molecules, cells, organisms, diseases, and drugs. It adds a global metabolic map combining about 120 maps and is expanding with KEGG MODULE, DRUG, and DISEASE.

Based on: KEGG for linking genomes to life and the environment · Nucleic Acids Res.

HighlightCurated by Aramai Editorial

Moses: Open Source Toolkit for Statistical Machine Translation

Introduces Moses, an open-source statistical machine translation toolkit with linguistic factors, confusion network decoding, and efficient data formats.

This paper describes Moses, an open-source toolkit for statistical machine translation. Its novel contributions are support for linguistically motivated factors, confusion network decoding, and efficient data formats for translation models and language models. Beyond the SMT decoder, the toolkit bundles a wide range of tools for training, tuning, and applying the system across many translation tasks.

Based on: Moses: Open Source Toolkit for Statistical Machine Translation · Annual Meeting of the Association for Computational Linguistics

HighlightCurated by Aramai Editorial

Zero-Shot Text-to-Image Generation

Models text and image tokens as one autoregressive transformer stream for zero-shot text-to-image generation competitive with domain-specific models.

Text-to-image generation has traditionally relied on better modeling assumptions for a fixed dataset, sometimes via complex architectures, auxiliary losses, or side information like object part labels or segmentation masks during training. This paper describes a simple alternative: a transformer that autoregressively models text and image tokens together as a single stream of data. Given sufficient data and scale, the approach is competitive with prior domain-specific models when evaluated in a zero-shot setting.

Based on: Zero-Shot Text-to-Image Generation · International Conference on Machine Learning

HighlightCurated by Aramai Editorial

DeepFace: Closing the Gap to Human-Level Performance in Face Verification

DeepFace uses 3D alignment and a nine-layer, 120M-parameter deep network to reach 97.35% on LFW face verification, nearly matching human performance.

Face recognition typically runs a pipeline of detect, align, represent, and classify. DeepFace revisits alignment and representation, using explicit 3D face modeling for a piecewise affine transformation and a nine-layer deep network. The network has over 120 million parameters and uses locally connected layers without weight sharing rather than standard convolutions, trained on four million images of over 4,000 identities. It reaches 97.35% accuracy on LFW, cutting the prior state-of-the-art error by over 27% and approaching human performance.

Based on: DeepFace: Closing the Gap to Human-Level Performance in Face Verification · 2014 IEEE Conference on Computer Vision and Pattern Recognition