đŸȘŽ Anil's Garden

GLUE Benchmark

02 Apr 20255 min read

GLUE Benchmark

Excerpt

The General Language Understanding Evaluation (GLUE) benchmark is a collection of resources for training, evaluating, and analyzing natural language understanding systems

1

MultiNLI Matched-Accuracy

Microsoft Alexander v-team

MultiNLI Matched-Accuracy

Turing ULR v691.373.397.594.2/92.393.5/93.176.4/90.992.592.196.793.697.955.42

MultiNLI Matched-Accuracy

JDExplore d-team

MultiNLI Matched-Accuracy

Vega v191.373.897.994.5/92.693.5/93.176.7/91.192.191.996.792.497.951.43

MultiNLI Matched-Accuracy

Microsoft Alexander v-team

MultiNLI Matched-Accuracy

Turing NLR v5 91.272.697.693.8/91.793.7/93.376.4/91.192.692.497.994.195.957.04

MultiNLI Matched-Accuracy

DIRL Team

MultiNLI Matched-Accuracy

DeBERTa + CLEVER91.174.797.693.3/91.193.4/93.176.5/91.092.191.896.793.296.653.35

MultiNLI Matched-Accuracy

ERNIE Team - Baidu

MultiNLI Matched-Accuracy

ERNIE91.175.597.893.9/91.893.0/92.675.2/90.992.391.797.392.695.951.76

MultiNLI Matched-Accuracy

AliceMind & DIRL

MultiNLI Matched-Accuracy

StructBERT + CLEVER91.075.397.793.9/91.993.5/93.175.6/90.891.791.597.492.595.249.17

MultiNLI Matched-Accuracy

DeBERTa Team - Microsoft

MultiNLI Matched-Accuracy

DeBERTa / TuringNLRv490.871.597.594.0/92.092.9/92.676.2/90.891.991.699.293.294.553.28

MultiNLI Matched-Accuracy

HFL iFLYTEK

MultiNLI Matched-Accuracy

MacALBERT + DKM90.774.897.094.5/92.692.8/92.674.7/90.691.391.197.892.094.552.69

MultiNLI Matched-Accuracy

PING-AN Omni-Sinitic

MultiNLI Matched-Accuracy

ALBERT + DAAF + NAS90.673.597.294.0/92.093.0/92.476.1/91.091.691.397.591.794.551.210

MultiNLI Matched-Accuracy

T5 Team - Google

MultiNLI Matched-Accuracy

T590.371.697.592.8/90.493.1/92.875.1/90.692.291.996.992.894.553.111

MultiNLI Matched-Accuracy

Microsoft D365 AI & MSR AI & GATECH

MultiNLI Matched-Accuracy

MT-DNN-SMART89.969.597.593.7/91.692.9/92.573.9/90.291.090.899.289.794.550.212

MultiNLI Matched-Accuracy

Huawei Noah’s Ark Lab

MultiNLI Matched-Accuracy

NEZHA-Large89.871.797.393.3/91.092.4/91.975.2/90.791.591.396.290.394.547.913

MultiNLI Matched-Accuracy

LG AI Research

MultiNLI Matched-Accuracy

ANNA89.868.797.092.7/90.193.0/92.875.3/90.591.891.696.091.895.951.814

MultiNLI Matched-Accuracy

Zihang Dai

MultiNLI Matched-Accuracy

Funnel-Transformer (Ensemble B10-10-10H1024)89.770.597.593.4/91.292.6/92.375.4/90.791.491.195.890.094.551.615

MultiNLI Matched-Accuracy

ELECTRA Team

MultiNLI Matched-Accuracy

ELECTRA-Large + Standard Tricks89.471.797.193.1/90.792.9/92.575.6/90.891.390.895.889.891.850.716

MultiNLI Matched-Accuracy

David Kim

MultiNLI Matched-Accuracy

2digit LANet89.371.897.392.4/89.693.0/92.775.5/90.591.891.696.491.188.454.617

MultiNLI Matched-Accuracy

ć€Ș仕文

MultiNLI Matched-Accuracy

DropAttack-RoBERTa-large88.870.396.792.6/90.192.1/91.875.1/90.591.190.995.389.989.748.218

MultiNLI Matched-Accuracy

Microsoft D365 AI & UMD

MultiNLI Matched-Accuracy

FreeLB-RoBERTa (ensemble)88.468.096.893.1/90.892.3/92.174.8/90.391.190.795.688.789.050.119

MultiNLI Matched-Accuracy

Junjie Yang

MultiNLI Matched-Accuracy

HIRE-RoBERTa88.368.697.193.0/90.792.4/92.074.3/90.290.790.495.587.989.049.320

MultiNLI Matched-Accuracy

Shiwen Ni

MultiNLI Matched-Accuracy

ELECTRA-large-M (bert4keras)88.369.395.892.2/89.691.2/91.175.1/90.591.190.993.887.991.848.221

MultiNLI Matched-Accuracy

Facebook AI

MultiNLI Matched-Accuracy

RoBERTa88.167.896.792.3/89.892.2/91.974.3/90.290.890.295.488.289.048.722

MultiNLI Matched-Accuracy

Microsoft D365 AI & MSR AI

MultiNLI Matched-Accuracy

MT-DNN-ensemble87.668.496.592.7/90.391.1/90.773.7/89.987.987.496.086.389.042.823

MultiNLI Matched-Accuracy

GLUE Human Baselines

MultiNLI Matched-Accuracy

GLUE Human Baselines87.166.497.886.3/80.892.7/92.659.5/80.492.092.891.293.695.9-24

MultiNLI Matched-Accuracy

kk xx

MultiNLI Matched-Accuracy

ELECTRA-Large-NewSCL(single)85.673.397.292.7/90.292.0/91.775.3/90.690.890.395.686.960.350.025

MultiNLI Matched-Accuracy

Adrian de Wynter

MultiNLI Matched-Accuracy

Bort (Alexa AI)83.663.996.294.1/92.389.2/88.366.0/85.988.187.892.382.771.251.926

MultiNLI Matched-Accuracy

Lab LV

MultiNLI Matched-Accuracy

ConvBERT base83.267.895.791.4/88.390.4/89.773.0/90.088.387.493.277.965.142.927

MultiNLI Matched-Accuracy

Stanford Hazy Research

MultiNLI Matched-Accuracy

Snorkel MeTaL83.263.896.291.5/88.590.1/89.773.1/89.987.687.293.980.965.139.928

MultiNLI Matched-Accuracy

XLM Systems

MultiNLI Matched-Accuracy

XLM (English only)83.162.995.690.7/87.188.8/88.273.2/89.889.188.594.076.071.944.729

MultiNLI Matched-Accuracy

WATCH ME

MultiNLI Matched-Accuracy

ConvBERT-base-paddle-v1.183.166.395.491.6/88.690.0/89.273.9/90.088.287.793.378.265.19.230

MultiNLI Matched-Accuracy

Zhuosheng Zhang

MultiNLI Matched-Accuracy

SemBERT82.962.394.691.2/88.387.8/86.772.8/89.887.686.394.684.565.142.431

MultiNLI Matched-Accuracy

Jun Yu

MultiNLI Matched-Accuracy

mpnet-base-paddle82.960.595.991.6/88.990.8/90.372.5/89.787.686.693.382.465.19.232

MultiNLI Matched-Accuracy

Danqi Chen

MultiNLI Matched-Accuracy

SpanBERT (single-task training)82.864.394.890.9/87.989.9/89.171.9/89.588.187.794.379.065.145.133

MultiNLI Matched-Accuracy

GAL team

MultiNLI Matched-Accuracy

distilRoBERTa+GAL (6-layer transformer single model)82.660.095.391.9/89.290.0/89.673.3/90.087.486.592.781.865.10.034

MultiNLI Matched-Accuracy

Kevin Clark

MultiNLI Matched-Accuracy

BERT + BAM82.361.595.291.3/88.388.6/87.972.5/89.786.685.893.180.465.140.735

MultiNLI Matched-Accuracy

Nitish Shirish Keskar

MultiNLI Matched-Accuracy

Span-Extractive BERT on STILTs82.363.294.590.6/87.689.4/89.272.2/89.486.585.892.579.865.128.336

MultiNLI Matched-Accuracy

LV NUS

MultiNLI Matched-Accuracy

LV-BERT-base82.164.094.790.9/87.989.4/88.872.3/89.586.686.192.677.065.139.537

MultiNLI Matched-Accuracy

Jason Phang

MultiNLI Matched-Accuracy

BERT on STILTs82.062.194.390.2/86.688.7/88.371.9/89.486.485.692.780.165.128.338

MultiNLI Matched-Accuracy

gao jie

MultiNLI Matched-Accuracy

182.066.896.590.9/87.291.4/90.872.9/89.690.256.494.782.862.39.239

MultiNLI Matched-Accuracy

Gino Tesei

MultiNLI Matched-Accuracy

RobustRoBERTa81.963.696.891.6/88.690.3/89.673.2/89.790.089.495.150.380.150.540

MultiNLI Matched-Accuracy

Karen Hambardzumyan

MultiNLI Matched-Accuracy

WARP with RoBERTa81.653.996.388.2/83.989.5/88.868.6/87.788.088.293.584.365.141.241

MultiNLI Matched-Accuracy

Junxiong Wang

MultiNLI Matched-Accuracy

Bigs-128-1000k81.564.494.988.7/84.287.8/87.571.2/89.286.185.091.677.665.136.242

MultiNLI Matched-Accuracy

Huawei Noah’s Ark Lab MTL

MultiNLI Matched-Accuracy

CombinedKD-TinyRoBERTa (6 layer 82M parameters, MATE-KD + AnnealingKD)81.558.695.191.2/88.188.5/88.473.0/89.786.285.692.476.665.120.243

MultiNLI Matched-Accuracy

Richard Bai

MultiNLI Matched-Accuracy

segaBERT-large81.462.694.889.7/86.188.6/87.772.5/89.487.987.794.071.665.10.044

MultiNLI Matched-Accuracy

ć»–äșż

MultiNLI Matched-Accuracy

u-PMLM-R (Huawei Noah’s Ark Lab)81.356.994.290.7/87.789.7/89.172.2/89.486.185.492.178.565.140.045

MultiNLI Matched-Accuracy

Xinsong Zhang

MultiNLI Matched-Accuracy

AMBERT-BASE81.060.095.290.6/87.186.3/88.272.2/89.587.286.592.672.665.139.446

MultiNLI Matched-Accuracy

Mikita Sazanovich

MultiNLI Matched-Accuracy

Routed BERTs80.756.193.688.6/84.788.0/87.671.0/88.885.284.592.680.065.19.247

MultiNLI Matched-Accuracy

USCD-AI4Health Team

MultiNLI Matched-Accuracy

CERT80.758.994.689.8/85.987.9/86.872.5/90.387.286.493.071.265.139.648

MultiNLI Matched-Accuracy

Jacob Devlin

MultiNLI Matched-Accuracy

BERT: 24-layers, 16-heads, 1024-hidden80.560.594.989.3/85.487.6/86.572.1/89.386.785.992.770.165.139.649

MultiNLI Matched-Accuracy

Chen Qian

MultiNLI Matched-Accuracy

KerasNLP XLM-R80.456.396.189.8/86.388.4/87.772.3/89.087.787.192.869.265.140.650

MultiNLI Matched-Accuracy

Chen Qian

MultiNLI Matched-Accuracy

KerasNLP RoBERTa80.456.396.189.8/86.388.4/87.772.3/89.087.787.192.869.265.140.651

MultiNLI Matched-Accuracy

Jinliang LU

MultiNLI Matched-Accuracy

MULTIPLE_ADAPTER_T5_BASE80.354.193.890.1/86.887.9/87.671.8/88.986.185.793.576.862.39.252

MultiNLI Matched-Accuracy

Yoshitomo Matsubara

MultiNLI Matched-Accuracy

HF bert-large-uncased (default fine-tuning)80.261.594.689.2/85.286.4/85.072.2/89.386.485.792.468.965.136.953

MultiNLI Matched-Accuracy

Neil Houlsby

MultiNLI Matched-Accuracy

BERT + Single-task Adapters80.259.294.388.7/84.387.3/86.171.5/89.485.485.092.471.665.19.254

MultiNLI Matched-Accuracy

KI BERT

MultiNLI Matched-Accuracy

KI-BERT80.055.694.588.2/83.986.3/85.171.5/88.985.283.791.269.373.335.655

MultiNLI Matched-Accuracy

Xiangyang Liu

MultiNLI Matched-Accuracy

elasticbert-large-12L79.957.092.989.4/86.089.7/88.672.7/89.685.484.992.371.862.39.256

MultiNLI Matched-Accuracy

战搑阳

MultiNLI Matched-Accuracy

roberta-large-12L79.859.494.689.1/85.889.8/89.171.5/89.486.485.291.667.362.39.257

MultiNLI Matched-Accuracy

Zhuohan Li

MultiNLI Matched-Accuracy

Macaron Net-base79.757.694.088.4/84.487.5/86.370.8/89.085.484.591.670.565.138.758

MultiNLI Matched-Accuracy

shi To

MultiNLI Matched-Accuracy

GAT-bert-base79.656.894.089.4/85.387.9/86.872.4/89.485.784.591.870.562.39.259

MultiNLI Matched-Accuracy

teerapong saelim

MultiNLI Matched-Accuracy

WT-VAT-BERT (Base)79.556.094.489.2/85.587.3/86.272.9/89.885.584.891.470.462.39.260

MultiNLI Matched-Accuracy

Anshuman Singh

MultiNLI Matched-Accuracy

Bert-n-Pals79.152.293.489.5/85.686.6/85.971.4/89.084.183.590.675.462.333.861

MultiNLI Matched-Accuracy

ANSHUMAN SINGH (RA1811003010460)

MultiNLI Matched-Accuracy

DeepPavlov Multitask PalBert78.848.193.488.9/85.687.0/86.771.4/89.083.983.490.876.762.333.862

MultiNLI Matched-Accuracy

xiaok Liu

MultiNLI Matched-Accuracy

BERT-EMD(6-layer; Single model; No DA)78.747.593.389.8/86.487.6/86.872.0/89.384.783.590.771.765.19.263

MultiNLI Matched-Accuracy

è˜‡ć€§éˆž

MultiNLI Matched-Accuracy

SesameBERT-Base78.652.794.288.9/84.886.5/85.570.8/88.883.783.691.067.665.135.864

MultiNLI Matched-Accuracy

xinge ma

MultiNLI Matched-Accuracy

ReptileDistil78.547.992.889.2/85.487.1/85.971.0/89.083.682.990.473.565.133.265

MultiNLI Matched-Accuracy

MobileBERT Team

MultiNLI Matched-Accuracy

MobileBERT78.551.192.688.8/84.586.2/84.870.5/88.384.383.491.670.465.134.366

MultiNLI Matched-Accuracy

Linyuan Gong

MultiNLI Matched-Accuracy

StackingBERT-Base78.456.293.988.2/83.984.2/82.570.4/88.784.484.290.167.065.136.667

MultiNLI Matched-Accuracy

TinyBERT Team

MultiNLI Matched-Accuracy

TinyBERT (6-layer; Single model)78.151.193.187.3/82.685.0/83.771.6/89.184.683.290.470.065.19.268

MultiNLI Matched-Accuracy

SqueezeBERT Team

MultiNLI Matched-Accuracy

SqueezeBERT (4.3x faster than BERT-base on smartphone)78.146.591.489.5/86.087.0/86.371.5/89.082.081.190.173.265.135.369

MultiNLI Matched-Accuracy

Anshuman Singh

MultiNLI Matched-Accuracy

CAMTL77.953.092.688.3/84.486.6/85.970.0/88.582.382.090.572.858.233.870

MultiNLI Matched-Accuracy

ć‚…è–›æž—

MultiNLI Matched-Accuracy

KRISFU77.852.492.589.0/84.883.7/82.270.4/88.684.383.490.965.965.136.171

MultiNLI Matched-Accuracy

王侊

MultiNLI Matched-Accuracy

s077.846.892.988.9/84.887.2/86.571.9/89.184.583.490.870.960.335.372

MultiNLI Matched-Accuracy

Stark Tony

MultiNLI Matched-Accuracy

Pocket GLUE77.649.392.489.0/84.684.9/84.070.1/88.784.082.890.167.265.136.173

MultiNLI Matched-Accuracy

Pavan Kalyan Reddy Neerudu

MultiNLI Matched-Accuracy

Pavan Neerudu - BERT77.656.193.587.6/83.285.3/83.870.6/88.884.083.490.864.060.334.674

MultiNLI Matched-Accuracy

NLC MSR Asia

MultiNLI Matched-Accuracy

BERT-of-Theseus (6-layer; single model)77.147.892.287.6/83.285.6/84.171.6/89.382.482.189.666.265.19.275

MultiNLI Matched-Accuracy

Hanxiong Huang

MultiNLI Matched-Accuracy

Hanxiong Huang75.949.393.387.1/81.983.3/81.771.5/89.184.883.891.064.153.49.276

MultiNLI Matched-Accuracy

YeonTaek Oh

MultiNLI Matched-Accuracy

EL-BERT(6-Layer, Single model)75.647.791.087.8/83.081.2/80.269.9/88.181.881.090.259.965.131.877

MultiNLI Matched-Accuracy

EVS Team

MultiNLI Matched-Accuracy

Anonymous74.752.693.487.6/83.261.2/59.171.8/89.383.783.289.965.062.335.678

MultiNLI Matched-Accuracy

Chen Money

MultiNLI Matched-Accuracy

KerasNLP 12/05/2022 Trial 274.652.293.587.8/82.684.5/83.171.3/89.382.381.689.361.743.832.979

MultiNLI Matched-Accuracy

Sinx

MultiNLI Matched-Accuracy

ZHIYUAN74.157.095.291.4/88.491.1/90.824.2/23.787.787.392.581.747.90.380

MultiNLI Matched-Accuracy

Tirana Noor Fatyanosa

MultiNLI Matched-Accuracy

distilbert-base-uncased73.645.892.387.6/83.171.0/71.069.6/88.281.681.388.854.165.131.881

MultiNLI Matched-Accuracy

Haiqin YANG

MultiNLI Matched-Accuracy

RefBERT73.147.992.986.9/81.975.0/76.361.6/84.480.980.387.361.754.8-10.382

MultiNLI Matched-Accuracy

Haiqin Yang

MultiNLI Matched-Accuracy

RefBERT73.147.992.986.9/81.975.0/76.361.4/84.280.980.387.361.754.8-10.383

MultiNLI Matched-Accuracy

Haiqin Yang

MultiNLI Matched-Accuracy

RefBERT71.836.392.986.9/81.975.0/76.361.6/83.880.980.387.361.754.8-10.384

MultiNLI Matched-Accuracy

Haiqin Yang

MultiNLI Matched-Accuracy

RefBERT71.836.392.986.9/81.975.0/76.361.3/83.680.980.387.361.754.8-10.385

MultiNLI Matched-Accuracy

ć…Źèƒœć…Źèƒœ

MultiNLI Matched-Accuracy

111171.435.890.183.2/75.781.0/79.368.5/87.577.577.186.758.056.89.286

MultiNLI Matched-Accuracy

Jack Hessel

MultiNLI Matched-Accuracy

Bag-of-words only BoW-BERT (Base)70.014.386.782.9/75.281.8/80.368.3/87.579.879.786.260.465.131.087

MultiNLI Matched-Accuracy

GLUE Baselines

MultiNLI Matched-Accuracy

BiLSTM+ELMo+Attn70.033.690.484.4/78.074.2/72.363.1/84.374.174.579.858.965.121.7

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

BiLSTM+ELMo67.732.189.384.7/78.070.3/67.861.1/82.667.267.975.557.465.121.3

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

Single Task BiLSTM+ELMo+Attn66.535.090.280.2/68.855.5/52.566.1/86.576.976.776.750.365.127.9

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

Single Task BiLSTM+ELMo66.435.090.280.8/69.064.0/60.265.6/85.772.973.471.750.165.119.5

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

GenSen66.17.783.183.0/76.679.3/79.259.8/82.971.471.378.659.265.120.6

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

BiLSTM+Attn65.618.683.083.9/76.272.8/70.560.1/82.467.668.374.358.465.117.8

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

BiLSTM64.211.682.881.8/74.370.3/67.862.5/84.265.666.174.657.465.120.3

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

InferSent63.94.585.181.2/74.175.9/75.359.1/81.766.165.772.758.065.118.3

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

Single Task BiLSTM63.715.785.979.4/69.366.0/62.861.4/81.770.370.875.752.862.321.0

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

Single Task BiLSTM+CoVe63.614.588.581.4/73.467.2/64.159.4/83.364.564.875.453.561.620.6

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

BiLSTM+CoVe+Attn63.18.380.780.0/71.869.8/68.460.5/83.468.168.672.956.065.118.3

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

Single Task BiLSTM+CoVe+Attn63.114.588.579.7/68.657.2/53.660.1/84.171.671.574.552.764.423.8

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

BiLSTM+CoVe62.918.581.978.7/71.564.4/62.760.6/84.965.465.770.852.765.117.6

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

Single Task BiLSTM+Attn62.815.785.980.3/68.559.3/55.862.9/83.574.273.877.251.955.524.9

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

DisSent61.94.983.781.7/74.166.1/64.859.5/82.658.759.173.956.465.115.9

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

Skip-Thought61.30.081.880.8/71.771.8/69.756.4/82.262.962.872.953.165.112.2

MultiNLI Matched-Accuracy

MultiNLI Matched-Accuracy

CBOW58.60.080.081.5/73.461.2/58.751.4/79.156.056.472.154.162.39.288

MultiNLI Matched-Accuracy

XLNet Team

MultiNLI Matched-Accuracy

XLNet (ensemble)-70.297.192.9/90.593.0/92.674.7/90.490.990.9-88.592.548.489

MultiNLI Matched-Accuracy

ALBERT-Team Google Language

MultiNLI Matched-Accuracy

ALBERT (Ensemble)-69.197.193.4/91.292.5/92.074.2/90.591.391.0-89.291.850.2

Graph View

Backlinks

  • No backlinks found