Librispeech: Difference between revisions

From MSc Voice Technology
Jump to navigation Jump to search
No edit summary
Line 21: Line 21:
== Performance Difference in WER between APIs ==
== Performance Difference in WER between APIs ==


Material source: LibriSpeech-dev-clean-1272-128104
Material source: LibriSpeech/dev-clean/1272/128104


=== WER Overview ===
{| class="wikitable"
{| class="wikitable"
|
|
|Google WER
|'''Google WER'''
|Whisper WER
|'''Whisper WER'''
|-
|-
|1272-128104-0000.flac
|1272-128104-0000.flac
|23.52941176
|17.65
|17.64705882
|5.88
|-
|-
|1272-128104-0001.flac
|1272-128104-0001.flac
|40
|30.0
|20
|10.0
|-
|-
|1272-128104-0002.flac
|1272-128104-0002.flac
|21.875
|15.62
|12.5
|3.12
|-
|-
|1272-128104-0003.flac
|1272-128104-0003.flac
|29.16666667
|29.17
|16.66666667
|8.33
|-
|-
|1272-128104-0004.flac
|1272-128104-0004.flac
|33.82352941
|29.41
|26.47058824
|16.18
|-
|-
|1272-128104-0005.flac
|1272-128104-0005.flac
|16.66666667
|11.11
|16.66666667
|5.56
|-
|-
|1272-128104-0006.flac
|1272-128104-0006.flac
|50
|41.67
|33.33333333
|16.67
|-
|-
|1272-128104-0007.flac
|1272-128104-0007.flac
|5.263157895
|5.26
|21.05263158
|5.26
|-
|-
|1272-128104-0008.flac
|1272-128104-0008.flac
|45.45454545
|36.36
|45.45454545
|18.18
|-
|-
|1272-128104-0009.flac
|1272-128104-0009.flac
|37.20930233
|37.21
|13.95348837
|9.3
|-
|-
|1272-128104-0010.flac
|1272-128104-0010.flac
|26.66666667
|20.0
|20
|6.67
|-
|-
|1272-128104-0011.flac
|1272-128104-0011.flac
|52.94117647
|50.0
|14.70588235
|5.88
|-
|-
|1272-128104-0012.flac
|1272-128104-0012.flac
|33.33333333
|22.22
|33.33333333
|0.0
|-
|-
|1272-128104-0013.flac
|1272-128104-0013.flac
|33.33333333
|27.78
|16.66666667
|5.56
|-
|-
|1272-128104-0014.flac
|1272-128104-0014.flac
|75
|80.0
|25
|40.0
|}
 
=== Detailed Transcription ===
{| class="wikitable"
|
|'''Reference'''
|'''Google transcription'''
|'''Whisper transcription'''
|-
|1272-128104-0000.flac
|MISTER QUILTER IS THE APOSTLE OF THE MIDDLE CLASSES AND WE ARE GLAD TO WELCOME HIS GOSPEL
|Mr quilter is the apostle of the middle classes and we're glad to welcome his gospel
|Mr. Quilter is the apostle of the middle classes, and we are glad to welcome his gospel.
|-
|1272-128104-0001.flac
|NOR IS MISTER QUILTER'S MANNER LESS INTERESTING THAN HIS MATTER
|nor is Mr coaches Manor less interesting than his matter
|Nor is Mr. Quilter's manner less interesting than his matter.
|-
|1272-128104-0002.flac
|HE TELLS US THAT AT THIS FESTIVE SEASON OF THE YEAR WITH CHRISTMAS AND ROAST BEEF LOOMING BEFORE US SIMILES DRAWN FROM EATING AND ITS RESULTS OCCUR MOST READILY TO THE MIND
|he tells us that at this festive season of the Year with Christmas roast beef looming before us similies drawn from eating and it's results are coronavirus ready to the mind
|He tells us that at this festive season of the year, with Christmas and roast beef looming before us, similarly drawn from eating and its results occur most readily to the mind.
|-
|1272-128104-0003.flac
|HE HAS GRAVE DOUBTS WHETHER SIR FREDERICK LEIGHTON'S WORK IS REALLY GREEK AFTER ALL AND CAN DISCOVER IN IT BUT LITTLE OF ROCKY ITHACA
|is grave doubts whether sir Frederic Layton's work is really Greek after all and can discover in it but little of Rocky is a car
|He has graved doubts whether Sir Frederick Layton's work is really Greek after all, and can discover in it but little of Rocky Ithaca.
|-
|1272-128104-0004.flac
|LINNELL'S PICTURES ARE A SORT OF UP GUARDS AND AT EM PAINTINGS AND MASON'S EXQUISITE IDYLLS ARE AS NATIONAL AS A JINGO POEM MISTER BIRKET FOSTER'S LANDSCAPES SMILE AT ONE MUCH IN THE SAME WAY THAT MISTER CARKER USED TO FLASH HIS TEETH AND MISTER JOHN COLLIER GIVES HIS SITTER A CHEERFUL SLAP ON THE BACK BEFORE HE SAYS LIKE A SHAMPOOER IN A TURKISH BATH NEXT MAN
|Lozells pictures of gods and Adam paintings and Masons exquisite Angels national as a jingle poem Mr broken Fosters landscapes smile at one match in the same way that Mr Kaka used to flash his teeth and Mr John Collier gives his sister a cheerful slap on the back before he said like a shampooer in a Turkish bath next Man
|Lynelle's pictures are a sort of upgards and atom paintings, and Mason's exquisite ittles are as national as a jingo poem. Mr. Birkett Foster's landscapes smile at one much in the same way that Mr. Carker used to flash his teeth. And Mr. John Collier gives his sitter a cheerful slap on the back, before he says, like a shampooer and a Turkish bath. Next man!
|-
|1272-128104-0005.flac
|IT IS OBVIOUSLY UNNECESSARY FOR US TO POINT OUT HOW LUMINOUS THESE CRITICISMS ARE HOW DELICATE IN EXPRESSION
|is obviously unnecessary for us to point out how luminous these criticisms are how delicate an expression
|It is obviously unnecessary for us to point out how luminous these criticisms are, how delicate and expression.
|-
|1272-128104-0006.flac
|ON THE GENERAL PRINCIPLES OF ART MISTER QUILTER WRITES WITH EQUAL LUCIDITY
|and the general principles of artistic water rates with equal lucidity
|On the general principles of art, Mr. Krilter writes with equal lucidity.
|-
|1272-128104-0007.flac
|PAINTING HE TELLS US IS OF A DIFFERENT QUALITY TO MATHEMATICS AND FINISH IN ART IS ADDING MORE FACT
|painting he tells us is of a different quality to Mathematics and finish in art is adding more fat
|Painting, he tells us, is of a different quality to mathematics, and finish in art is adding more factor.
|-
|1272-128104-0008.flac
|AS FOR ETCHINGS THEY ARE OF TWO KINDS BRITISH AND FOREIGN
|as for itching there are two kinds of British and foreign
|As for etchings, there are two kinds, British and foreign.
|-
|1272-128104-0009.flac
|HE LAMENTS MOST BITTERLY THE DIVORCE THAT HAS BEEN MADE BETWEEN DECORATIVE ART AND WHAT WE USUALLY CALL PICTURES MAKES THE CUSTOMARY APPEAL TO THE LAST JUDGMENT AND REMINDS US THAT IN THE GREAT DAYS OF ART MICHAEL ANGELO WAS THE FURNISHING UPHOLSTERER
|elements most beautiful that has been made between decorative part and what we usually call pictures makes a customer appeal to The Last Judgement and reminds us that integrate days of art Michelangelo was the furnishing a poster
|He laments most bitterly the divorce that has been made between decorative art and what we usually call pictures. Mixed a customary appeal to the last judgment and reminds us that in the great days of art Michelangelo was the furnishing upholsterer.
|-
|1272-128104-0010.flac
|NEAR THE FIRE AND THE ORNAMENTS FRED BROUGHT HOME FROM INDIA ON THE MANTEL BOARD
|near the fire any ornaments Fred brought home from India on the mental board
|near the fire, and the ornaments Fred brought home from India on the mental board.
|-
|1272-128104-0011.flac
|IN FACT HE IS QUITE SEVERE ON MISTER RUSKIN FOR NOT RECOGNISING THAT A PICTURE SHOULD DENOTE THE FRAILTY OF MAN AND REMARKS WITH PLEASING COURTESY AND FELICITOUS GRACE THAT MANY PHASES OF FEELING
|in fact is quite severe Mr Ruskin from not recognising that I picked you shouldn't up the 3rd of man and remarks with please increase the infelicitous Grace that many faces is a feeling
|In fact, he is quite severe on Mr. Ruskin for not recognizing that a picture should denote the frailty of man, and remarks with pleasing courtesy and felicitous grace that many phases of feeling
|-
|1272-128104-0012.flac
|ONLY UNFORTUNATELY HIS OWN WORK NEVER DOES GET GOOD
|only unfortunately is only work never does get good
|Only, unfortunately, his own work never does get good.
|-
|1272-128104-0013.flac
|MISTER QUILTER HAS MISSED HIS CHANCE FOR HE HAS FAILED EVEN TO MAKE HIMSELF THE TUPPER OF PAINTING
|Mr Coulter has missed his chance for his failed even to make himself the tower of painting
|Mr. Quilter has missed his chance, for he has failed even to make himself the Tupper of Painting.
|-
|1272-128104-0014.flac
|BY HARRY QUILTER M A
|hi Harry quarter ma
|by Harry quilter MA.
|}
|}



Revision as of 19:48, 27 September 2023

Dataset name Type of data Annotation remarks License Metadata
LibriSpeech FLAC
  • The audio file and the transcription file are stored under the same directory.
  • There are multiple audio files and only one txt file which is the transcription.
  • And in the transcription file, the transcription for each audio is in separate lines.
Creative Commons Attribution 4.0 International License.
  • Sampling rate: 16 kHz
  • Download size: 57.14 GiB

Performance Difference in WER between APIs

Material source: LibriSpeech/dev-clean/1272/128104

WER Overview

Google WER Whisper WER
1272-128104-0000.flac 17.65 5.88
1272-128104-0001.flac 30.0 10.0
1272-128104-0002.flac 15.62 3.12
1272-128104-0003.flac 29.17 8.33
1272-128104-0004.flac 29.41 16.18
1272-128104-0005.flac 11.11 5.56
1272-128104-0006.flac 41.67 16.67
1272-128104-0007.flac 5.26 5.26
1272-128104-0008.flac 36.36 18.18
1272-128104-0009.flac 37.21 9.3
1272-128104-0010.flac 20.0 6.67
1272-128104-0011.flac 50.0 5.88
1272-128104-0012.flac 22.22 0.0
1272-128104-0013.flac 27.78 5.56
1272-128104-0014.flac 80.0 40.0

Detailed Transcription

Reference Google transcription Whisper transcription
1272-128104-0000.flac MISTER QUILTER IS THE APOSTLE OF THE MIDDLE CLASSES AND WE ARE GLAD TO WELCOME HIS GOSPEL Mr quilter is the apostle of the middle classes and we're glad to welcome his gospel Mr. Quilter is the apostle of the middle classes, and we are glad to welcome his gospel.
1272-128104-0001.flac NOR IS MISTER QUILTER'S MANNER LESS INTERESTING THAN HIS MATTER nor is Mr coaches Manor less interesting than his matter Nor is Mr. Quilter's manner less interesting than his matter.
1272-128104-0002.flac HE TELLS US THAT AT THIS FESTIVE SEASON OF THE YEAR WITH CHRISTMAS AND ROAST BEEF LOOMING BEFORE US SIMILES DRAWN FROM EATING AND ITS RESULTS OCCUR MOST READILY TO THE MIND he tells us that at this festive season of the Year with Christmas roast beef looming before us similies drawn from eating and it's results are coronavirus ready to the mind He tells us that at this festive season of the year, with Christmas and roast beef looming before us, similarly drawn from eating and its results occur most readily to the mind.
1272-128104-0003.flac HE HAS GRAVE DOUBTS WHETHER SIR FREDERICK LEIGHTON'S WORK IS REALLY GREEK AFTER ALL AND CAN DISCOVER IN IT BUT LITTLE OF ROCKY ITHACA is grave doubts whether sir Frederic Layton's work is really Greek after all and can discover in it but little of Rocky is a car He has graved doubts whether Sir Frederick Layton's work is really Greek after all, and can discover in it but little of Rocky Ithaca.
1272-128104-0004.flac LINNELL'S PICTURES ARE A SORT OF UP GUARDS AND AT EM PAINTINGS AND MASON'S EXQUISITE IDYLLS ARE AS NATIONAL AS A JINGO POEM MISTER BIRKET FOSTER'S LANDSCAPES SMILE AT ONE MUCH IN THE SAME WAY THAT MISTER CARKER USED TO FLASH HIS TEETH AND MISTER JOHN COLLIER GIVES HIS SITTER A CHEERFUL SLAP ON THE BACK BEFORE HE SAYS LIKE A SHAMPOOER IN A TURKISH BATH NEXT MAN Lozells pictures of gods and Adam paintings and Masons exquisite Angels national as a jingle poem Mr broken Fosters landscapes smile at one match in the same way that Mr Kaka used to flash his teeth and Mr John Collier gives his sister a cheerful slap on the back before he said like a shampooer in a Turkish bath next Man Lynelle's pictures are a sort of upgards and atom paintings, and Mason's exquisite ittles are as national as a jingo poem. Mr. Birkett Foster's landscapes smile at one much in the same way that Mr. Carker used to flash his teeth. And Mr. John Collier gives his sitter a cheerful slap on the back, before he says, like a shampooer and a Turkish bath. Next man!
1272-128104-0005.flac IT IS OBVIOUSLY UNNECESSARY FOR US TO POINT OUT HOW LUMINOUS THESE CRITICISMS ARE HOW DELICATE IN EXPRESSION is obviously unnecessary for us to point out how luminous these criticisms are how delicate an expression It is obviously unnecessary for us to point out how luminous these criticisms are, how delicate and expression.
1272-128104-0006.flac ON THE GENERAL PRINCIPLES OF ART MISTER QUILTER WRITES WITH EQUAL LUCIDITY and the general principles of artistic water rates with equal lucidity On the general principles of art, Mr. Krilter writes with equal lucidity.
1272-128104-0007.flac PAINTING HE TELLS US IS OF A DIFFERENT QUALITY TO MATHEMATICS AND FINISH IN ART IS ADDING MORE FACT painting he tells us is of a different quality to Mathematics and finish in art is adding more fat Painting, he tells us, is of a different quality to mathematics, and finish in art is adding more factor.
1272-128104-0008.flac AS FOR ETCHINGS THEY ARE OF TWO KINDS BRITISH AND FOREIGN as for itching there are two kinds of British and foreign As for etchings, there are two kinds, British and foreign.
1272-128104-0009.flac HE LAMENTS MOST BITTERLY THE DIVORCE THAT HAS BEEN MADE BETWEEN DECORATIVE ART AND WHAT WE USUALLY CALL PICTURES MAKES THE CUSTOMARY APPEAL TO THE LAST JUDGMENT AND REMINDS US THAT IN THE GREAT DAYS OF ART MICHAEL ANGELO WAS THE FURNISHING UPHOLSTERER elements most beautiful that has been made between decorative part and what we usually call pictures makes a customer appeal to The Last Judgement and reminds us that integrate days of art Michelangelo was the furnishing a poster He laments most bitterly the divorce that has been made between decorative art and what we usually call pictures. Mixed a customary appeal to the last judgment and reminds us that in the great days of art Michelangelo was the furnishing upholsterer.
1272-128104-0010.flac NEAR THE FIRE AND THE ORNAMENTS FRED BROUGHT HOME FROM INDIA ON THE MANTEL BOARD near the fire any ornaments Fred brought home from India on the mental board near the fire, and the ornaments Fred brought home from India on the mental board.
1272-128104-0011.flac IN FACT HE IS QUITE SEVERE ON MISTER RUSKIN FOR NOT RECOGNISING THAT A PICTURE SHOULD DENOTE THE FRAILTY OF MAN AND REMARKS WITH PLEASING COURTESY AND FELICITOUS GRACE THAT MANY PHASES OF FEELING in fact is quite severe Mr Ruskin from not recognising that I picked you shouldn't up the 3rd of man and remarks with please increase the infelicitous Grace that many faces is a feeling In fact, he is quite severe on Mr. Ruskin for not recognizing that a picture should denote the frailty of man, and remarks with pleasing courtesy and felicitous grace that many phases of feeling
1272-128104-0012.flac ONLY UNFORTUNATELY HIS OWN WORK NEVER DOES GET GOOD only unfortunately is only work never does get good Only, unfortunately, his own work never does get good.
1272-128104-0013.flac MISTER QUILTER HAS MISSED HIS CHANCE FOR HE HAS FAILED EVEN TO MAKE HIMSELF THE TUPPER OF PAINTING Mr Coulter has missed his chance for his failed even to make himself the tower of painting Mr. Quilter has missed his chance, for he has failed even to make himself the Tupper of Painting.
1272-128104-0014.flac BY HARRY QUILTER M A hi Harry quarter ma by Harry quilter MA.

Comparison Between Transcription and Original Transcript using Google API

  • Transcription (Reader: Xueying Liu)
  • Actual sentence and comparison
  • Transcription (Reader: Jingxuan Yue)
  • Actual sentence and comparison
  • Transcription (Reader: M.Tepei)
  • Actual sentence and comparison

Contributors

  • Yuxing Ouyang
  • Xiaoling Lin
  • Xueying Liu
  • Jingxuan Yue
  • M.Tepei