The main quality aspect is that the frameworks provide the required functionality. This is discussed in the previous chapter. However, this chapter deals with further quality issues of the framework. Table 3 shows a summary of this evaluation part. Afterwards the evaluation is discussed in detail.
| Requirement | CAB/SCSF May 2007 | Spring .NET Version 1.1 | SharpDevelop Version 2.1 |
|---|---|---|---|
| Framework composition | (o) | (+) | (+) |
| Framework dependencies | (o) | (+) | (-) |
| Evolution | (+) | (+) | (-) |
| Documentation | |||
| Purpose of the framework | (+) | (+) | (+) |
| How to use the framework | (+) | (+) | (o) |
| Detailed design of the framework | (+) | (o) | (o) |
| Examples | (+) | (+) | (o) |
Table 3: Shows the evaluation of further quality issues.
It is common that frameworks must be composed with other frameworks to
get all the functionality which is required for the application. Frameworks
are often designed that they are in full control. Problems can occur if two
composed frameworks require both full control over the application [BMMB97,
p. 11]. In case of the Test Suite all investigated frameworks are designed
for working with the .NET Framework. Thus, no issues occur if the frameworks
are composed together with the .NET Framework. The situation changes if
other frameworks, like a persistence framework, are added.
In CAB and Spring .NET the framework takes over the control during
Dependency Injection. All the necessary objects for the Dependency Injection
are created by the framework. This can be an issue if the frameworks are
composed with other frameworks. For example, a persistence framework is
usually responsible for retrieving the persisted data and creating the
necessary objects which are filled with this data. In this case, both
frameworks want to create the same objects. A solution is that these objects
are not created by Dependency Injection. Instead, the objects retrieve the
necessary services via the Service Locator implementation. In Spring .NET
the situation looks a bit better as the framework already supports some
additional frameworks (e.g. NHibernate) out of the box.
SharpDevelop is not affected by this requirement because it does not
provide Dependency Injection.
A characteristic of frameworks is that beside code reuse they also define the application design. The application developer can concentrate on the implementation without worrying too much about creating a good object-oriented design. However, the drawback is that a framework is closely coupled to the application.
As a framework evolves, applications have to evolve with it. That makes loose coupling all the more important; otherwise even a minor change to the framework will have major repercussions [GHJV95, p. 27].
Dependency Injection helps to minimize or completely remove the
dependencies between the framework and the application. The investigated
frameworks, which support decoupling, use this pattern. An introduction in
Dependency Injection can be found in
chapter 4.2.
The Composite UI Application Block uses the Object Builder for Dependency
Injection. Nevertheless, it does not completely decouple the modules from
the framework since the modules use Attributes to control the injection. The
Attributes are part of the framework. Therefore a reference to the framework
is necessary. Other dependencies arise through sub classing of the ModuleInit class and the use of own sub
WorkItems. Additionally, the
Dependency Injection implementation is not able to inject primitive types.
If a service has to be configured with primitive types, it has to be
manually instantiated and passed to the Service Locator. The use of the
Service Locator also requires a reference to the framework assembly. With
all these dependencies to the framework they are still lower as without
using Dependency Injection at all.
Spring .NET provides a powerful Dependency Injection mechanism that
allows the complete decoupling of the modules from the framework. One
exception is the use of loosely coupled events. The modules which are using
them require access to the IApplicationContext and consequently, a reference
to a Spring .NET assembly is necessary. Another drawback is the high amount
of configuration which is necessary for the complete decoupling of the
components.
SharpDevelop does not support Dependency Injection and thus, the add-ins
are closely coupled to the SharpDevelop core.
The number of breaking changes during the framework evolution is an
important quality metric. Braking changes are all modifications in a
framework that require the application to be modified too. If many breaking
changes are introduced in a framework the effort for updating the
applications to a new framework version is high. This metric is very
difficult to define. One solution is to have a look at the past years of the
framework and use this data to predict the future.
Since CAB was released in December 2005 no chances were made until
September 2007. Microsoft has introduced new features with the Smart Client
Software Factory, but the CAB framework itself was not updated by SCSF. The
fact, that the framework did not need to be changed since its release,
approves the high quality of the framework.
In Spring .NET the breaking changes of the version 1.0 release in
September 2005 until the version 1.1 RC1 release in August 2007 are minimal10.
It shows also here that Spring .NET is a well designed framework. The
conclusion is further supported as version 1.1 introduces lots of new
functionality without the need for changing the API.
SharpDevelop is not intended to be a framework. That is why the stability
of the interfaces is not that important than in the other solutions. An
example for a breaking change with high impact is the elimination of the
ServiceManager. The book "Dissecting a C# Application - Inside SharpDevelop"
that was first printed in February 2003 describes the ServiceManager [HKS03,
p. 109]. In the version 2.1 the ServiceManager does not exist anymore.
Johnson discusses the amount of documentation that is necessary for a user to understand a framework [Johnson92]. He states that the documentation should contain:
The purpose of all investigated solutions is well documented. The information on using the framework is useful in CAB and Spring .NET but lacks with SharpDevelop. This results from the fact that SharpDevelop is not intended to be used as an application framework. The detailed design of the framework is only sufficiently explained for CAB. The documentation of Spring .NET shows every possible usage scenarios of the framework but does not explain the internal design in detail. The SharpDevelop team has devoted a book to the design decisions they made in the application [HKS03]. However, the book is outdated today and does not reflect the current state of SharpDevelop. CAB and Spring .NET ships useful examples which are also documented. In the case of SharpDevelop just a few examples exist regarding the usage of the core to build own applications. These examples are not in the official documentation of SharpDevelop.
10 Breaking changes of Spring .NET 1.1: http://www.springframework.net/BreakingChanges-1.1.txt