Grading rubrics: Difference between revisions

From MSc Voice Technology
Jump to navigation Jump to search
 
(14 intermediate revisions by the same user not shown)
Line 2: Line 2:


== Introduction to Voice Technology ==
== Introduction to Voice Technology ==
This section is dedicated to describing how scoring in the [[Intro to Voice Technology syllabus|Intro to Voice Tech]] course is performed.  
This section describes how scoring in the [[Intro to Voice Technology syllabus|Intro to Voice Tech]] course is performed.  


=== Wiki pages ===
=== Wiki pages ===
Line 70: Line 70:
* 7 = LLM used productively and reported well
* 7 = LLM used productively and reported well
* 10 = LLM used in a highly productive or interesting way and reported very well
* 10 = LLM used in a highly productive or interesting way and reported very well


=== Talking clock ===
=== Talking clock ===
This assignment is assessed along the following 4 criteria, each of which is described under the table.
{| class="wikitable"
{| class="wikitable"
|+
!Criteria
!Maximum points
|-
|-
|Recording quality
! Criteria !! Description !! Maximum Points
|3
|-
| Interface Design || Aesthetics and layout of the clock's GUI. Incl. aspects like color scheme, font choices, visual consistency, and user-friendliness. || 2
|-
| Usability || Ease of use and intuitiveness. Is it straightforward for a user to interact with and customize? Does it require a steep learning curve? || 2
|-
|-
|User experience
| Audio Quality || Quality of the audio recordings used for time announcements. Incl. clarity, pronunciation, volume, and overall audio experience. || 2
|3
|-
|-
|Non-tech doc
| Documentation || Clarity and completeness of the documentation provided. Is everything properly explained? If there are multiple languages in the talking clock, is there some reflection on relevant differences between the languages? Are ethical issues mentioned in the documentation (if there are none, this should be motivated)? || 2
|3
|-
|-
|Extra functionalities
| Customization Options || Variety and utility of customization options. This may include language selection, voice options, snooze, alarm, time zones, and other features. || 2
|1
|-
|-
|TOTAL
| colspan="2" |Total
|10
|10
|}
|}
Key:
* 0.0: Not addressed at all.
* 0.5: Addressed in a very minimal way
* 1.0: Addressed to a limited extent
* 1.5: Addressed very well
* 2.0: Addressed perfectly, nothing can be improved.


* Recording quality:
=== Talking clock video presentation ===
** 1 = The time is expressed in a glitchy or unpleasant way.
{| class="wikitable"
** 2 = The time is expressed in a reasonably clear way.
|-
** 3 = The time is expressed in a natural, lifelike (or purposefully creative) way.
! Criteria !! Description !! Maximum Points
* User experience
|-
** 0 = The interface is not functional
| Functionality || The extent to which the basic functionalities of the clock are adequately presented as per instructions || 2
** 1 = The interface is mostly functional, minor issues notwithstanding
|-
** 2 = The interface is completely functional
| Customization + interface || The extent to which the customized features and interface of the clock are adequately presented, with attention given to aesthetics, creativity, and intuitive aspects || 2
** 3 = The interface is completely functional and there is evidence of some creative flourish
|-
* Non-Technical Documentation (including team organization and project workflow, GUI user manual, and Licensing statement / reflection FAIR data):
| Shortcomings / challenges || The extent to which any shortcomings or room for improvement are adequately discussed in a constructive and practical way || 2
** 0 = Absent
|-
** 1 = Very incomplete
| Quality of the video and presentation || Extent to which the video content is professionally made. Overall professional impression || 2
** 2 = Complete, but lacking detail
|-
** 3 = Complete, all expected information is provided
| Engagement || The extent to which the video  engages the viewer, sustaining  interest. The point is to make something that is interesting to watch through e.g. sharing your commitment, interest, enthusiasm, or struggles in a way that hooks the viewer -- not just simply reading from a script in a monotone voice. || 2
* Are there any extra functionalities?
|-
** 0 = No
| colspan="2" | TOTAL || 10
** 1 = Yes
|}
Key'':''
* 0.0: Not at all
* 0.5: In a minimal way
* 1.0: To a limited extent
* 1.5: Very well
* 2.0: Perfectly


=== Participation activities ===
=== Participation activities ===
Line 129: Line 139:


== Speech Sounds ==
== Speech Sounds ==
== Thesis Design ==
This section describes how scoring in the [[Thesis Design]] course is performed.
==== Participation ====
Participation will be assessed via three metrics: 1) Your delivery of in class exercises (assessed P/F);  2) your self-assessment of participation as in I2VT; 3) my assessment of your participation (the extent to which you engage with the content presented by me and your peers)
==== Peer review 1 (RQ + hypothesis) ====
* Beginning (0 points)
** The RQ is not adequately precise. More work is necessary to sharpen it.
** The link between the hypothesis and the RQ is too fuzzy. More work is necessary.
** The scope of the research is such that there will likely be no meaningful novel contribution to science.
* Emerging (1 point)
** The RQ is relatively precise, though there is room for improvement. Relevance and/or importance are not entirely convincing.
** The link between the hypothesis and the RQ could be made more logical or coherent.
** The scope of the research is such that it is not yet entirely clear if there is a meaningful novel contribution to science, no matter how small (regardless of whether the hypothesis is validated or not).
* Proficient (2 points)
** The RQ is somewhat precise. It is likely answerable and relevant.
** It seems that the hypothesis follows logically from the RQ, (in)validating the hypothesis would answer the RQ - though the link could still be sharpened.
** The scope of the research is such that there may be a meaningful novel contribution to science, no matter how small (regardless of whether the hypothesis is validated or not).
* Experienced (3 points)
** The RQ is very precise and sharp. It is obviously answerable, relevant, and important.
** The clearly hypothesis follows logically from the RQ, (in)validating the hypothesis would directly answer the RQ.
** The scope of the research is such that there is meaningful novel contribution to science, no matter how small (regardless of whether the hypothesis is validated or not).
==== Peer review 2 ====
Similar as to peer review 1
==== Peer review 3 ====
Similar as to peer review 1
==== Article presentation ====
'''1.''' '''Content accuracy and comprehensiveness''' '''(40 points)'''
* Excellent (40 points): Precise, thorough summary of key findings, methodology, and relevance to research. Speaker demonstrates high degree of familiarity with the text and critical thinking.
* Good (30 points): Covers most key points with minor omissions. Speaker demonstrates good degree of familiarity with the text.
* Satisfactory (20 points): Basic summary but misses significant details. Speaker demonstrates moderate degree of familiarity with the text. Lack of critical analysis, more of a "book report".
* Needs Improvement (0-10 points): Lacks important information, significant inaccuracies. Speaker demonstrates low degree of familiarity with the text.
'''2. Presentation skills  assessed by instructor (30 points)'''
* Excellent (30 points): Clear, well-organized, within time limit, effective communication. Speaker makes the relevance very clear for those unfamiliar with the topic through avoiding jargon and giving examples. Useful visualizations created by the student.
* Good (20 points): Mostly clear, minor issues in delivery or time management. Speaker makes the relevance clear for those unfamiliar with the topic through avoiding jargon and giving examples (though there is room for improvement).
* Satisfactory (15 points): Understandable but lacks organization or has delivery issues. Speaker makes the relevance moderately clear for those unfamiliar with the topic, though there may be too much jargon or too few examples.
* Needs Improvement (0-5 points): Poorly organized, difficult to follow, major issues in delivery
'''3. Contribution to the Wiki  (30 points)'''
* Excellent (30 points): Excellent summary in very clear and accessible language. This Wiki page is a very useful resource for those interested in the article.
* Good (15 points): Good summary in clear and accessible language. This Wiki page is a useful resource for those interested in the article.
* Satisfactory (10 points): Basic summary, but room for improvement. This Wiki page is a somewhat useful resource for those interested in the article.
* Needs Improvement (0-10 points): Inadequate summary. This Wiki page is a not a particularly useful resource for those interested in the article.
==== Collaborative article resource assignment ====
'''Individual contributions (Total: 40 Points)'''
# '''Depth and insight of article analysis and annotations (40 points)'''
#* 0-10 points: Annotations lack depth or critical engagement with the article. Minimal or superficial evaluations are provided.
#* 11-20 points: Annotations show some critical engagement, but analyses are inconsistent or lack depth in places.
#* 21-30 points: Annotations are detailed, demonstrating a good level of critical engagement and insight into the methodologies and findings.
#* 31-40 points: Annotations are exceptionally detailed and insightful, offering deep analyses of methodologies, findings, and relevance, with clear connections to the thesis theme.
'''Group Contributions (Total: 60 Points)'''
# '''Relevance and coherence of synthesized findings in the report (40 points)'''
#* 0-10 points: The report lacks coherence; synthesized findings are not clearly relevant to the thematic focus.
#* 11-20 points: The report shows some relevance and coherence, but the synthesis of findings is uneven or lacks depth.
#* 21-30 points: The report is coherent and relevant, with a clear synthesis of findings that are well-aligned with the thematic focus.
#* 31-40 points: The report is highly coherent and relevant, with a comprehensive and insightful synthesis of findings, clearly illuminating trends, debates, or future directions.
# '''Organization and clarity of the report (20 points)'''
#* 0-5 points: The report is poorly organized and difficult to follow, with significant issues in clarity or adherence to the template.
#* 6-10 points: The report has some organizational structure and clarity, but there are notable issues in adherence to the template or writing quality.
#* 11-15 points: The report is well-organized and clear, with minor issues in adherence to the template or writing quality.
#* 16-20 points: The report is excellently organized and clear, adhering closely to the template with high-quality writing throughout.
==== Interactive poster pitch ====
'''#BetterPoster Slide Evaluation (Total: 7 Points)'''
# '''Content Quality (3 points)'''
#* 0 points: Lacks depth in content; critical information is missing or incorrect.
#* 1 point: Contains relevant information but lacks depth or some content may be inaccurate.
#* 2 points: Very good content quality which is very accurate but not very interesting (or may have too much jargon)
#* 3 points: Excellent content quality which balances attention-grabbing interest and scientific accuracy..
# '''Alignment with #BetterPoster Principles (4 points)'''
#* 0 points: Does not follow #BetterPoster principles; lacks a clear main message.
#* 1 point: Partially follows #BetterPoster principles with a main message that could be clearer. May lack a functional QR code.
#* 2 points: Aligns with #BetterPoster principles but may lack some aspects of design or clarity.
#* 3 points: Well aligned with #BetterPoster principles. Easy to read, uncluttered, and effectively communicates the main message.
#* 4 points: Exemplary alignment with #BetterPoster principles. Easy to read, uncluttered, particularly aesthetic, and includes functional enhancements and/or creative / interactive elements.
'''3-Minute Pitch Evaluation (Total: 8 Points)'''
# '''Delivery (3 points)'''
#* 0 points: Unclear, rushed, or monotone delivery that fails to engage the audience.
#* 1 point: Basic delivery with some clarity, but generally lacking in engagement.
#* 2 points: Good delivery with clear articulation and moderate engagement.
#* 3 points: Excellent delivery with engaging, clear, and well-paced articulation.
# '''Content and Structure (3 points)'''
#* 0 points: Lacks a clear structure; content is disorganized and main points are not conveyed.
#* 1 point: Some organization with main points somewhat articulated.
#* 2 points: Well-organized content with clear main points.
#* 3 points: Exceptionally structured pitch with a compelling narrative and clear, insightful main points.
# '''Engagement and Response to Questions (2 points)'''
#* 0 points: Does not engage the audience and fails to address questions adequately.
#* 1 point: Engages the audience with acceptable responses to most questions.
#* 2 points: Highly engages the audience and provides thorough, thoughtful responses to all questions.

Latest revision as of 15:23, 17 March 2024

Below are grading rubrics for some courses.

Introduction to Voice Technology[edit | edit source]

This section describes how scoring in the Intro to Voice Tech course is performed.

Wiki pages[edit | edit source]

The Wiki pages will be assessed according to the criteria below. A total score will be given assigned as an average.

Criteria Points
Topic 10
Presentation 10
Underlying research 10
Content 10
Internal linking 10
LLM review 10

Topic: Appropriateness and relevance

  • 0 = Inappropriate
  • 5 = Somewhat Appropriate
  • 7 = Good
  • 10 = Completely Appropriate

Presentation: Adherence to layout, clarity of language

  • 0 = Unacceptable
  • 5 = Sufficient but rushed (lots of language may be difficult to understand or the flow may be incoherent)
  • 7 = Good
  • 10 = Excellent

Underlying Research: Evidence of scholarly research

  • 0 = No citations
  • 5 = Citations only to popular content and/or citations are incorrect / missing
  • 7 = Some Citations, mainly reporting outcomes
  • 10 = Excellent, evidence of synthesis of complex ideas and reflection

Content

  • 0 = Incoherent
  • 5 = Coherent but superficial
  • 7 = Good, some depth
  • 10 = Excellent

Internal linking to other Wiki articles produced by peers

  • 0 = No Links
  • 5 = Some links missing
  • 7 = Well-linked to other articles
  • 10 = Well-linked to other articles in an enriching way which demonstrates a deep understanding

LLM Review

  • 0 = No effort made
  • 5 = Minimal effort, but not transparent, insufficient detail
  • 7 = LLM used productively and reported well
  • 10 = LLM used in a highly productive or interesting way and reported very well


Talking clock[edit | edit source]

Criteria Description Maximum Points
Interface Design Aesthetics and layout of the clock's GUI. Incl. aspects like color scheme, font choices, visual consistency, and user-friendliness. 2
Usability Ease of use and intuitiveness. Is it straightforward for a user to interact with and customize? Does it require a steep learning curve? 2
Audio Quality Quality of the audio recordings used for time announcements. Incl. clarity, pronunciation, volume, and overall audio experience. 2
Documentation Clarity and completeness of the documentation provided. Is everything properly explained? If there are multiple languages in the talking clock, is there some reflection on relevant differences between the languages? Are ethical issues mentioned in the documentation (if there are none, this should be motivated)? 2
Customization Options Variety and utility of customization options. This may include language selection, voice options, snooze, alarm, time zones, and other features. 2
Total 10

Key:

  • 0.0: Not addressed at all.
  • 0.5: Addressed in a very minimal way
  • 1.0: Addressed to a limited extent
  • 1.5: Addressed very well
  • 2.0: Addressed perfectly, nothing can be improved.

Talking clock video presentation[edit | edit source]

Criteria Description Maximum Points
Functionality The extent to which the basic functionalities of the clock are adequately presented as per instructions 2
Customization + interface The extent to which the customized features and interface of the clock are adequately presented, with attention given to aesthetics, creativity, and intuitive aspects 2
Shortcomings / challenges The extent to which any shortcomings or room for improvement are adequately discussed in a constructive and practical way 2
Quality of the video and presentation Extent to which the video content is professionally made. Overall professional impression 2
Engagement The extent to which the video engages the viewer, sustaining interest. The point is to make something that is interesting to watch through e.g. sharing your commitment, interest, enthusiasm, or struggles in a way that hooks the viewer -- not just simply reading from a script in a monotone voice. 2
TOTAL 10

Key:

  • 0.0: Not at all
  • 0.5: In a minimal way
  • 1.0: To a limited extent
  • 1.5: Very well
  • 2.0: Perfectly

Participation activities[edit | edit source]

For the most part, participation activities are scored on a three-point scale:

  • 0 = incomplete
  • 1 = minimal attempt to deliver
  • 2 = rushed or incomplete delivery
  • 3 = complete delivery

Some activities are worth 2 points:

  • 0 = incomplete
  • 1 = minimal or incomplete delivery
  • 2 = complete delivery

Programming[edit | edit source]

Speech Sounds[edit | edit source]

Thesis Design[edit | edit source]

This section describes how scoring in the Thesis Design course is performed.

Participation[edit | edit source]

Participation will be assessed via three metrics: 1) Your delivery of in class exercises (assessed P/F); 2) your self-assessment of participation as in I2VT; 3) my assessment of your participation (the extent to which you engage with the content presented by me and your peers)

Peer review 1 (RQ + hypothesis)[edit | edit source]

  • Beginning (0 points)
    • The RQ is not adequately precise. More work is necessary to sharpen it.
    • The link between the hypothesis and the RQ is too fuzzy. More work is necessary.
    • The scope of the research is such that there will likely be no meaningful novel contribution to science.
  • Emerging (1 point)
    • The RQ is relatively precise, though there is room for improvement. Relevance and/or importance are not entirely convincing.
    • The link between the hypothesis and the RQ could be made more logical or coherent.
    • The scope of the research is such that it is not yet entirely clear if there is a meaningful novel contribution to science, no matter how small (regardless of whether the hypothesis is validated or not).
  • Proficient (2 points)
    • The RQ is somewhat precise. It is likely answerable and relevant.
    • It seems that the hypothesis follows logically from the RQ, (in)validating the hypothesis would answer the RQ - though the link could still be sharpened.
    • The scope of the research is such that there may be a meaningful novel contribution to science, no matter how small (regardless of whether the hypothesis is validated or not).
  • Experienced (3 points)
    • The RQ is very precise and sharp. It is obviously answerable, relevant, and important.
    • The clearly hypothesis follows logically from the RQ, (in)validating the hypothesis would directly answer the RQ.
    • The scope of the research is such that there is meaningful novel contribution to science, no matter how small (regardless of whether the hypothesis is validated or not).

Peer review 2[edit | edit source]

Similar as to peer review 1

Peer review 3[edit | edit source]

Similar as to peer review 1


Article presentation[edit | edit source]

1. Content accuracy and comprehensiveness (40 points)

  • Excellent (40 points): Precise, thorough summary of key findings, methodology, and relevance to research. Speaker demonstrates high degree of familiarity with the text and critical thinking.
  • Good (30 points): Covers most key points with minor omissions. Speaker demonstrates good degree of familiarity with the text.
  • Satisfactory (20 points): Basic summary but misses significant details. Speaker demonstrates moderate degree of familiarity with the text. Lack of critical analysis, more of a "book report".
  • Needs Improvement (0-10 points): Lacks important information, significant inaccuracies. Speaker demonstrates low degree of familiarity with the text.

2. Presentation skills assessed by instructor (30 points)

  • Excellent (30 points): Clear, well-organized, within time limit, effective communication. Speaker makes the relevance very clear for those unfamiliar with the topic through avoiding jargon and giving examples. Useful visualizations created by the student.
  • Good (20 points): Mostly clear, minor issues in delivery or time management. Speaker makes the relevance clear for those unfamiliar with the topic through avoiding jargon and giving examples (though there is room for improvement).
  • Satisfactory (15 points): Understandable but lacks organization or has delivery issues. Speaker makes the relevance moderately clear for those unfamiliar with the topic, though there may be too much jargon or too few examples.
  • Needs Improvement (0-5 points): Poorly organized, difficult to follow, major issues in delivery

3. Contribution to the Wiki (30 points)

  • Excellent (30 points): Excellent summary in very clear and accessible language. This Wiki page is a very useful resource for those interested in the article.
  • Good (15 points): Good summary in clear and accessible language. This Wiki page is a useful resource for those interested in the article.
  • Satisfactory (10 points): Basic summary, but room for improvement. This Wiki page is a somewhat useful resource for those interested in the article.
  • Needs Improvement (0-10 points): Inadequate summary. This Wiki page is a not a particularly useful resource for those interested in the article.

Collaborative article resource assignment[edit | edit source]

Individual contributions (Total: 40 Points)

  1. Depth and insight of article analysis and annotations (40 points)
    • 0-10 points: Annotations lack depth or critical engagement with the article. Minimal or superficial evaluations are provided.
    • 11-20 points: Annotations show some critical engagement, but analyses are inconsistent or lack depth in places.
    • 21-30 points: Annotations are detailed, demonstrating a good level of critical engagement and insight into the methodologies and findings.
    • 31-40 points: Annotations are exceptionally detailed and insightful, offering deep analyses of methodologies, findings, and relevance, with clear connections to the thesis theme.

Group Contributions (Total: 60 Points)

  1. Relevance and coherence of synthesized findings in the report (40 points)
    • 0-10 points: The report lacks coherence; synthesized findings are not clearly relevant to the thematic focus.
    • 11-20 points: The report shows some relevance and coherence, but the synthesis of findings is uneven or lacks depth.
    • 21-30 points: The report is coherent and relevant, with a clear synthesis of findings that are well-aligned with the thematic focus.
    • 31-40 points: The report is highly coherent and relevant, with a comprehensive and insightful synthesis of findings, clearly illuminating trends, debates, or future directions.
  2. Organization and clarity of the report (20 points)
    • 0-5 points: The report is poorly organized and difficult to follow, with significant issues in clarity or adherence to the template.
    • 6-10 points: The report has some organizational structure and clarity, but there are notable issues in adherence to the template or writing quality.
    • 11-15 points: The report is well-organized and clear, with minor issues in adherence to the template or writing quality.
    • 16-20 points: The report is excellently organized and clear, adhering closely to the template with high-quality writing throughout.

Interactive poster pitch[edit | edit source]

#BetterPoster Slide Evaluation (Total: 7 Points)

  1. Content Quality (3 points)
    • 0 points: Lacks depth in content; critical information is missing or incorrect.
    • 1 point: Contains relevant information but lacks depth or some content may be inaccurate.
    • 2 points: Very good content quality which is very accurate but not very interesting (or may have too much jargon)
    • 3 points: Excellent content quality which balances attention-grabbing interest and scientific accuracy..
  2. Alignment with #BetterPoster Principles (4 points)
    • 0 points: Does not follow #BetterPoster principles; lacks a clear main message.
    • 1 point: Partially follows #BetterPoster principles with a main message that could be clearer. May lack a functional QR code.
    • 2 points: Aligns with #BetterPoster principles but may lack some aspects of design or clarity.
    • 3 points: Well aligned with #BetterPoster principles. Easy to read, uncluttered, and effectively communicates the main message.
    • 4 points: Exemplary alignment with #BetterPoster principles. Easy to read, uncluttered, particularly aesthetic, and includes functional enhancements and/or creative / interactive elements.

3-Minute Pitch Evaluation (Total: 8 Points)

  1. Delivery (3 points)
    • 0 points: Unclear, rushed, or monotone delivery that fails to engage the audience.
    • 1 point: Basic delivery with some clarity, but generally lacking in engagement.
    • 2 points: Good delivery with clear articulation and moderate engagement.
    • 3 points: Excellent delivery with engaging, clear, and well-paced articulation.
  2. Content and Structure (3 points)
    • 0 points: Lacks a clear structure; content is disorganized and main points are not conveyed.
    • 1 point: Some organization with main points somewhat articulated.
    • 2 points: Well-organized content with clear main points.
    • 3 points: Exceptionally structured pitch with a compelling narrative and clear, insightful main points.
  3. Engagement and Response to Questions (2 points)
    • 0 points: Does not engage the audience and fails to address questions adequately.
    • 1 point: Engages the audience with acceptable responses to most questions.
    • 2 points: Highly engages the audience and provides thorough, thoughtful responses to all questions.