D
Dwayne Anius
This is the results of the survey conducted by Dwayne Anius and Brian
Dobing on the topic PROGRAMMERS' VIEWS OF THE USEFULNESS OF UML
DIAGRAMS. Thank You to all who participated in the survey.
Please feel free to take part in the survey as the research is ongoing
and any additional data provided would be included in further
analysis.
http://www.surveymonkey.com/s/uml - Survey Link
Survey Monkey collected 94 responses from February 16 to May 6, 2010.
This analysis is based on the 47 complete responses and another 5 that
provided responses to at least seven of the UML diagram items. The
experience profile of the respondents is shown in Table 1. Presumably
those reporting 15 and 20 years UML programming experience were
including object-oriented programming prior to release of the UML.
About 21% reported 10 years experience or more. The two experience
measures were correlated (0.64). About 58% reported using C++ and 50%
have used Java, with 13% having used both. Respondents could list as
many languages as they had used; the next most frequently mentioned
was C# (13%).
Table 1: Respondent Experience
Mean Median Min Max N
Programming Experience (yrs) 14.3 15 2.0 40 51
Programming Experience with UML (yrs) 5.4 5 0.1 20 52
Respondents were also asked if they played “a role in developing
system requirements before or while programming the applications.”
Almost half (25 of 52) selected “Major Role” while 11 chose “Moderate
Role” and 14 “Some Role.” Greater overall programming experience, but
not specifically UML experience, was weakly correlated with playing a
greater role.
Table 2 shows the percentage of respondents who have used each
diagram, the average percentage of time they had access to them (when
relevant), and the average ratings for usefulness and accuracy. The
final column shows the number of responses to the first question
(Used). When the percentage using the diagram is low, the N for the
remaining questions is correspondingly low as well because those not
using the diagram do not see those questions.
Table 2: Respondent Views of UML Diagrams
Used (%) Access Accuracy
Usefulness N
Activity 75 27 2.85 3.41 52
Class 98 65 3.39 4.06 52
Collaboration/Communication 50 28 2.82 3.39 52
Component 54 32 2.54 2.75 52
Deployment 35 27 2.61 2.50 52
Object 50 30 2.69 3.08 52
Package 46 23 2.67 2.92 52
Sequence 82 43 3.33 3.76 50
State Chart/Machine 69 35 3.59 3.94 49
Timing 6 36 2.00 3.00 47
Use Case Diagrams 87 47 3.12 3.00 47
Use Case Narratives 47 51 3.59 3.41 47
Table 2 shows that the Class Diagram has been almost universally used.
However, it must be noted the definition was “ever used,” not
typically used, so we cannot assume that Class Diagrams are used on
98% of projects. Sequence (82%) and Activity (75%) Diagrams have been
also used by a solid majority of the programmers in our sample. The
Timing Diagram is rarely used, but it is not often needed and is newer
than many of the others. The Component, Deployment and Package
Diagrams are used more at the architectural level and thus might not
be relevant to some of our respondents. Use Case Narratives also had a
lower usage rate, which is consistent with claims that programmers
find them less useful because they don’t understand the user domain
and the terminology. But there may be other factors at work as well.
The Access column also contains generally low numbers, with only the
Class Diagram (65%) and Use Case Narratives (51%) over the 50% mark.
Based on the question wording, which included the qualification “when
they would be relevant,” the numbers should be much higher. The
results suggest that many respondents may have missed that part. For
example, two of the three respondents who have used the Timing Diagram
reported that they are available when relevant only 2% and 5% of the
time. This seems unlikely so we may be asking some respondents for
clarification.
Respondents reported low Accuracy ratings for many of the diagrams.
The highest ratings are for State Chart/Machine Diagrams (3.59), which
are not heavily used according earlier surveys, and Use Case
Narratives (3.59). The Class (3.39), Sequence (3.33) and Activity
(2.85) Diagrams were all heavily used by respondents but apparently
not that well written. Use Case Diagrams (3.12) also received accuracy
ratings above the midpoint of 3.0, but still rather low for something
that provides a high level overview. The survey did not address the
reasons for these ratings (although some respondents provided
comments). Requirements can change, but that is not the fault of the
diagram. Diagrams can also be syntactically incorrect, but this is
less likely when appropriate modeling tools are used. So the main
problem is likely to be specifications that don’t match the true
requirements, perhaps incorrect or incomplete.
The Usefulness item included the qualification “when reasonably
accurate and complete,” so the ratings are presumably higher than they
would be for the diagrams the respondents are actually given. Two of
the diagrams, Class (4.07) and State Chart/Machine (3.94) received
average scores around the “Very Good” level. These ratings tend to be
higher than the accuracy ratings.
CONCLUSIONS
There has been considerable attention paid to how well the UML serves
as a requirement specifications language, which is its primary
intended purpose. However, at some point specifications need to be
turned into code and the results of this survey suggest that the UML
is not working as well as it could or should in this task.
We are continuing to analyze the data and welcome your comments, ideas
or questions. Please send them to Professor Brian Dobing at the
University of Lethbridge at (e-mail address removed).
And the survey remains open, so feel free to encourage colleagues and
other IT professionals you know to participate and share their views.
The survey can be found at: http://www.surveymonkey.com/s/uml
Dobing on the topic PROGRAMMERS' VIEWS OF THE USEFULNESS OF UML
DIAGRAMS. Thank You to all who participated in the survey.
Please feel free to take part in the survey as the research is ongoing
and any additional data provided would be included in further
analysis.
http://www.surveymonkey.com/s/uml - Survey Link
Survey Monkey collected 94 responses from February 16 to May 6, 2010.
This analysis is based on the 47 complete responses and another 5 that
provided responses to at least seven of the UML diagram items. The
experience profile of the respondents is shown in Table 1. Presumably
those reporting 15 and 20 years UML programming experience were
including object-oriented programming prior to release of the UML.
About 21% reported 10 years experience or more. The two experience
measures were correlated (0.64). About 58% reported using C++ and 50%
have used Java, with 13% having used both. Respondents could list as
many languages as they had used; the next most frequently mentioned
was C# (13%).
Table 1: Respondent Experience
Mean Median Min Max N
Programming Experience (yrs) 14.3 15 2.0 40 51
Programming Experience with UML (yrs) 5.4 5 0.1 20 52
Respondents were also asked if they played “a role in developing
system requirements before or while programming the applications.”
Almost half (25 of 52) selected “Major Role” while 11 chose “Moderate
Role” and 14 “Some Role.” Greater overall programming experience, but
not specifically UML experience, was weakly correlated with playing a
greater role.
Table 2 shows the percentage of respondents who have used each
diagram, the average percentage of time they had access to them (when
relevant), and the average ratings for usefulness and accuracy. The
final column shows the number of responses to the first question
(Used). When the percentage using the diagram is low, the N for the
remaining questions is correspondingly low as well because those not
using the diagram do not see those questions.
Table 2: Respondent Views of UML Diagrams
Used (%) Access Accuracy
Usefulness N
Activity 75 27 2.85 3.41 52
Class 98 65 3.39 4.06 52
Collaboration/Communication 50 28 2.82 3.39 52
Component 54 32 2.54 2.75 52
Deployment 35 27 2.61 2.50 52
Object 50 30 2.69 3.08 52
Package 46 23 2.67 2.92 52
Sequence 82 43 3.33 3.76 50
State Chart/Machine 69 35 3.59 3.94 49
Timing 6 36 2.00 3.00 47
Use Case Diagrams 87 47 3.12 3.00 47
Use Case Narratives 47 51 3.59 3.41 47
Table 2 shows that the Class Diagram has been almost universally used.
However, it must be noted the definition was “ever used,” not
typically used, so we cannot assume that Class Diagrams are used on
98% of projects. Sequence (82%) and Activity (75%) Diagrams have been
also used by a solid majority of the programmers in our sample. The
Timing Diagram is rarely used, but it is not often needed and is newer
than many of the others. The Component, Deployment and Package
Diagrams are used more at the architectural level and thus might not
be relevant to some of our respondents. Use Case Narratives also had a
lower usage rate, which is consistent with claims that programmers
find them less useful because they don’t understand the user domain
and the terminology. But there may be other factors at work as well.
The Access column also contains generally low numbers, with only the
Class Diagram (65%) and Use Case Narratives (51%) over the 50% mark.
Based on the question wording, which included the qualification “when
they would be relevant,” the numbers should be much higher. The
results suggest that many respondents may have missed that part. For
example, two of the three respondents who have used the Timing Diagram
reported that they are available when relevant only 2% and 5% of the
time. This seems unlikely so we may be asking some respondents for
clarification.
Respondents reported low Accuracy ratings for many of the diagrams.
The highest ratings are for State Chart/Machine Diagrams (3.59), which
are not heavily used according earlier surveys, and Use Case
Narratives (3.59). The Class (3.39), Sequence (3.33) and Activity
(2.85) Diagrams were all heavily used by respondents but apparently
not that well written. Use Case Diagrams (3.12) also received accuracy
ratings above the midpoint of 3.0, but still rather low for something
that provides a high level overview. The survey did not address the
reasons for these ratings (although some respondents provided
comments). Requirements can change, but that is not the fault of the
diagram. Diagrams can also be syntactically incorrect, but this is
less likely when appropriate modeling tools are used. So the main
problem is likely to be specifications that don’t match the true
requirements, perhaps incorrect or incomplete.
The Usefulness item included the qualification “when reasonably
accurate and complete,” so the ratings are presumably higher than they
would be for the diagrams the respondents are actually given. Two of
the diagrams, Class (4.07) and State Chart/Machine (3.94) received
average scores around the “Very Good” level. These ratings tend to be
higher than the accuracy ratings.
CONCLUSIONS
There has been considerable attention paid to how well the UML serves
as a requirement specifications language, which is its primary
intended purpose. However, at some point specifications need to be
turned into code and the results of this survey suggest that the UML
is not working as well as it could or should in this task.
We are continuing to analyze the data and welcome your comments, ideas
or questions. Please send them to Professor Brian Dobing at the
University of Lethbridge at (e-mail address removed).
And the survey remains open, so feel free to encourage colleagues and
other IT professionals you know to participate and share their views.
The survey can be found at: http://www.surveymonkey.com/s/uml