Highlight

Learning Entity and Relation Embeddings for Knowledge Graph Completion

Proposes TransR, a knowledge graph embedding model that learns entity and relation embeddings in separate spaces for link prediction.

This paper tackles knowledge graph completion via graph embeddings. Prior models like TransE and TransH treat a relation as a translation from head to tail entity but place entities and relations in one space. Since an entity has multiple aspects that different relations emphasize, the authors propose TransR to build entity and relation embeddings in separate spaces, projecting entities into a relation-specific space before translating. On link prediction, triple classification, and relational fact extraction, TransR gives consistent gains over TransE and TransH.

Based on: Learning Entity and Relation Embeddings for Knowledge Graph Completion · AAAI Conference on Artificial Intelligence

Curated by Aramai Editorial

Read summary →
Highlight

piecewiseSEM: Piecewise structural equation modelling in r for ecology, evolution, and systematics

Presents piecewiseSEM, an open-source R package implementing confirmatory path analysis for ecology, evolution, and systematics.

piecewiseSEM is an open-source R package implementing confirmatory path analysis, a form of structural equation modeling (SEM), for ecology, evolution, and systematics. Unlike covariance-based SEM, local estimation allows non-normal distributions, random effects, and varied correlation structures, but had not been automated. The package extends piecewise SEM to generalized linear, phylogenetic least-squares, and mixed effects models with familiar R syntax, and includes worked examples using random effects, temporal autocorrelation, and phylogenetic contrasts.

Based on: piecewiseSEM: Piecewise structural equation modelling in r for ecology, evolution, and systematics · arXiv

Curated by Aramai Editorial

Read summary →
Highlight

CodeBERT: A Pre-Trained Model for Programming and Natural Languages

Presents CodeBERT, a bimodal Transformer pre-trained on natural language and programming language for code search and documentation.

CodeBERT is a bimodal pre-trained model for programming and natural language that learns general-purpose representations supporting tasks like code search and code documentation generation. Built on a Transformer architecture, it is trained with a hybrid objective including replaced token detection, letting it use both bimodal NL-PL pairs and unimodal data. Fine-tuned, CodeBERT achieves state-of-the-art results on natural language code search and code documentation generation, and it outperforms prior models on a zero-shot NL-PL probing dataset.

Based on: CodeBERT: A Pre-Trained Model for Programming and Natural Languages · Findings

Curated by Aramai Editorial

Read summary →
Highlight

FiLM: Visual Reasoning with a General Conditioning Layer

Introduces FiLM, a feature-wise linear modulation layer that conditions neural network computation for visual reasoning tasks.

FiLM (Feature-wise Linear Modulation) is a general-purpose conditioning method that influences neural network computation through a simple feature-wise affine transformation driven by conditioning information. Applied to visual reasoning, where answering image questions requires multi-step high-level processing, FiLM layers halve the state-of-the-art error on the CLEVR benchmark. They modulate features coherently, remain robust to ablations and architectural changes, and generalize well to new data from few examples or even zero-shot.

Based on: FiLM: Visual Reasoning with a General Conditioning Layer · AAAI Conference on Artificial Intelligence

Curated by Aramai Editorial

Read summary →
Highlight

Program Synthesis with Large Language Models

Evaluates large language models on program synthesis using the new MBPP and MathQA-Python benchmarks across few-shot and fine-tuning regimes.

This paper probes how well current large language models (244M to 137B parameters) synthesize short Python programs from natural language, using two new benchmarks: MBPP with 974 entry-level tasks and MathQA-Python with 23,914 problems. Synthesis performance scales log-linearly with model size; the largest model solves 59.6% of MBPP via few-shot prompting and reaches 83.8% on MathQA-Python after fine-tuning. Human dialog feedback halves the error rate, but even the best models struggle to predict program execution outputs.

Based on: Program Synthesis with Large Language Models · arXiv.org

Curated by Aramai Editorial

Read summary →
Highlight

TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension

Introduces TriviaQA, a large-scale distantly supervised reading comprehension dataset of over 650K question-answer-evidence triples.

TriviaQA is a reading comprehension dataset of over 650K question-answer-evidence triples, including 95K QA pairs written by trivia enthusiasts plus independently gathered evidence documents averaging six per question for distant supervision. Its questions are more compositional, show greater syntactic and lexical variability, and demand more cross-sentence reasoning than prior datasets. Two baselines, a feature-based classifier and a strong neural network, reach only 23% and 40%, far below human performance of 80%.

Based on: TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension · Annual Meeting of the Association for Computational Linguistics

Curated by Aramai Editorial

Read summary →
Highlight

Finetuned Language Models Are Zero-Shot Learners

This paper shows instruction tuning, finetuning a large language model on many tasks phrased as instructions, greatly improves zero-shot generalization.

This paper studies instruction tuning as a simple way to improve the zero-shot abilities of large language models. A 137B-parameter pretrained model is finetuned on over 60 NLP tasks expressed via natural language instruction templates, producing a model called FLAN. On held-out task types, FLAN clearly beats its unmodified counterpart and outperforms zero-shot 175B GPT-3 on 20 of the 25 evaluated tasks, even topping few-shot GPT-3 on benchmarks like ANLI, RTE, and BoolQ. Ablations show the number of finetuning tasks, model scale, and instruction phrasing are all crucial.

Based on: Finetuned Language Models Are Zero-Shot Learners · International Conference on Learning Representations

Curated by Aramai Editorial

Read summary →
Highlight

SRILM - an extensible language modeling toolkit

SRILM is an extensible C++ toolkit for building and experimenting with statistical N-gram language models for speech recognition and related tasks.

SRILM bundles C++ libraries, executable programs, and helper scripts for building and experimenting with statistical language models used in speech recognition and other applications. Freely available for noncommercial use, it supports creating and evaluating a variety of N-gram-based language model types, along with related tasks like statistical tagging and operations on N-best lists and word lattices. The paper summarizes the toolkit's functionality and discusses a design emphasizing rapid prototyping, reusability, and combinability of tools.

Based on: SRILM - an extensible language modeling toolkit · Interspeech

Curated by Aramai Editorial

Read summary →
Highlight

The R*-tree: an efficient and robust access method for points and rectangles

The R*-tree improves the R-tree by jointly optimizing area, margin, and overlap of bounding rectangles, giving a robust spatial access method.

The R*-tree is a spatial access method for points and rectangles that refines the classic R-tree. Whereas the R-tree only heuristically minimizes enclosing-rectangle area, the R*-tree jointly optimizes area, margin, and overlap of directory rectangles, tuned via extensive experiments over varied data and queries. In benchmarks it clearly outperforms existing R-tree variants, including Guttman's linear and quadratic versions and Greene's variant, for rectangles and multidimensional points. It supports point and spatial data together at slightly higher implementation cost.

Based on: The R*-tree: an efficient and robust access method for points and rectangles · ACM SIGMOD Conference

Curated by Aramai Editorial

Read summary →
Highlight

NLTK: The Natural Language Toolkit

NLTK is a Python, open-source suite of modules, datasets, and tutorials for research and teaching in computational linguistics and NLP.

The Natural Language Toolkit (NLTK) is a suite of program modules, data sets, and tutorials that support research and teaching in computational linguistics and natural language processing. Written in Python and distributed under the GPL open-source license, the toolkit was rewritten over the past year to simplify many of its linguistic data structures and to exploit recent enhancements in the Python language. This paper reports on the simplified toolkit and explains how it is used in teaching NLP.

Based on: NLTK: The Natural Language Toolkit · Annual Meeting of the Association for Computational Linguistics

Curated by Aramai Editorial

Read summary →
Highlight

Adversarial Discriminative Domain Adaptation

ADDA unifies adversarial domain adaptation methods, combining discriminative modeling, untied weight sharing, and a GAN loss for unsupervised adaptation.

Adversarial Discriminative Domain Adaptation (ADDA) tackles unsupervised domain adaptation by aligning source and target feature distributions via adversarial learning. They propose a general framework casting prior adversarial adaptation methods as special cases differing in modeling, weight sharing, and loss. Within it they introduce ADDA, combining discriminative modeling, untied weight sharing, and a GAN-based loss. Simpler yet more effective than competing methods, it exceeds prior state-of-the-art on standard benchmarks and a hard cross-modality classification task.

Based on: Adversarial Discriminative Domain Adaptation · Computer Vision and Pattern Recognition

Curated by Aramai Editorial

Read summary →
Highlight

Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

DeepLab combines deep convolutional networks with fully connected CRFs to sharpen boundary localization in semantic image segmentation.

This work unites deep convolutional neural networks (DCNNs) with probabilistic graphical models for semantic image segmentation. The invariance that aids DCNN classification leaves final-layer responses too poorly localized for segmentation, so the authors couple them with a fully connected Conditional Random Field (CRF). The resulting DeepLab system localizes boundaries more precisely than prior methods, setting a new state of the art on PASCAL VOC-2012 (71.6% IOU). Network re-purposing and the wavelet 'hole' algorithm enable dense computation at 8 fps on a GPU.

Based on: Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs · International Conference on Learning Representations

Curated by Aramai Editorial

Read summary →