ELSNET-List Message
Elsnet
 
   


ELSNET-List Message

Subject: [ E-Announce ] THE JOINT STUDENT RESPONSE ANALYSIS AND 8TH RECOGNIZING TEXTUAL ENTAILMENT CHALLENGE
From: <giampiccolo_(on)_celct.it>
Date received: 16 Dec 2012
Deadline: 15 Feb 2013
Start date: -





Please distribute widely - Apologies for cross-posting

*******************************************

CALL FOR PARTICIPATION

THE JOINT STUDENT RESPONSE ANALYSIS AND 8TH RECOGNIZING TEXTUAL
ENTAILMENT CHALLENGE (SemEval-2013 Task 7)

We are pleased to invite participants to the Joint Student
Response Analysis and 8th Recognizing Textual Entailment
Challenge at SemEval 2013, a common effort of both educational
technology and textual inference communities in order to present
a unified scientific challenge addressing researchers in both
fields. It will be run together with the SEMEVAL-2013 Semantic
Evaluation Exercise, co-located with the *SEM and NAACL-2013
conferences.

MAIN TASK

The goal of the task is to produce an assessment of student
answers to explanation and definition questions typically asked
in practice exercises, tests or tutorial dialogue. It is linked
to the tasks of semantic analysis and recognizing textual
entailment in the textual inference community, and essay grading
and short answer assessment in the educational NLP community.

Specifically, given a question, a known correct "reference
answer" and a 1- or 2-sentence "student answer", the Main task
consists of assessing the correctness of a student's answer at
different levels of granularity - either 5-way (correct,
partially correct, contradictory, irrelevant, not in the domain),
3-way task (correct, contradictory, incorrect), or 2-way task
(correct, incorrect). Participants can opt to carry out the task
at any level of granularity.

This task is similar to essay grading and short answer
assessment. However, meaningful labels are used instead of
numerical scores as a way to extract information necessary to
provide effective tutorial feedback with respect to the specific
problems that can be found in student answers. Educational NLP
and essay grading system developers are encouraged to adapt their
scoring techniques to the categorical labeling problem.

The Student Response Analysis task is also closely related to the
notion of textual entailment. In most correct answers at least a
substantial portion of the reference answer will be entailed by
the student answer text, even if not all of it. Thus textual
entailment engine developers are encouraged to exploit textual
entailment techniques to test the potential contributions of
their systems to the student response assessment problem.

PILOT TASK ON PARTIAL ENTAILMENT

An additional pilot task on partial entailment is offered as part
of the challenge, where systems may recognize that specific parts
of the hypothesis are entailed by the text, even though
entailment might be not recognized for the hypothesis as a whole.
Such recognition of partial entailment may have various uses in
the educational setting based on identifying the missing parts in
the student answer, and may similarly have value in other
applications such as summarization or question answering.

PROPOSED SCHEDULE:

- November 9, 2012 Training Data available for participants
- February 15, 2013 Registration Deadline
- March 1 - 15, 2013 Evaluation period
- April 9, 2013 Paper submission deadline [TBC]
- April 23, 2013 Reviews Due [TBC]
- May 4, 2013 Camera ready Due [TBC]

More details on the Main and Pilot tasks are available at:
http://www.cs.york.ac.uk/semeval-2013/task7/.

If you are interested in the task, please register at SemEval
2013
(http://www.cs.york.ac.uk/semeval-2013/index.php?id=registration).
__________________________________________
- ELSNET mailing list Elsnet-list_(at)_elsnet.org
- To manage your subscription go to:
  http://mailman.elsnet.org/mailman/listinfo/elsnet-list


	

[ Search | Events calendar | Deadline calendar ]

 

Page generated 21-12-2014 by Steven Krauwer Disclaimer / Contact ELSNET