The trained marker is then applied to the remaining diagrams (the test set consisting of 70 diagrams) and the results reported for this test set. Following the approach adopted for ERDs (Thomas et al., 2007c), we determine suitable values for the thresholds using a training set of diagrams in this case the first 30 diagrams in our corpus. Thresholds are determined by experiment and represent limiting values for example, two objects will be considered the same if the measure of their similarity is above a certain threshold. The weights represent the significance, for marking purposes, of diagram features in a particular domain and are set by the user. The similarity measures are modified by certain weights and thresholds which have to be determined for the data set under test. We then compare the results of the automatic marker to the moderated human marks.

We mark diagrams by comparing a student’s attempt with a gold standard – a specimen solution (there could be more than one acceptable solution) – to produce a set of similarity measures for certain diagram features known as minimal meaningful units (MMUs) (Smith et al., 2004) to which the mark scheme used by the human markers is applied. Of course, the user may not be able to fix the errors and can submit the syntactically incorrect diagram to the automatic marker which has to operate in the face of such errors. If a repaired diagram is re-checked, errors may still be detected, but these are the irreparable complex errors which the user should attempt to fix prior to submitting the diagram to the automatic marker. Other errors will be repaired on the basis of the most likely error – there are some patterns which occur frequently and have highly likely causes (see Figure 4). Simple errors, by definition, are those that can be repaired without changing the essential meaning of the diagram. Not all errors are repaired – only those that can be dealt with with reasonable certainty. Having identified errors, the tool can be asked to repair the errors (see Figure 3). A textual description of an error can be obtained by right-clicking on the erroneous element and requesting feedback as shown in Figure 2. The syntax checker identifies errors by shading (objects and activations) and change of colour (messages) – although this mechanism may change after usability testing. This facility can be disabled if required, for example if the tool is used in an exam. tool shown in Figure 1 incorporates a syntax checking facility (within the Tools menu).

Sequence diagram online draw