SWSV home assignment

Throughout the semester students shall form teams of four and solve a home assignment, in which the course's verification and validation techniques (review, source code analysis, testing...) should be applied to a given software module.

The home assignment consists of four phases, which will be published on this page. In each phase new parts of the module will be shared, and several tasks have to be solved.

 

Supporting infrastructure (repository)

To solve the assignments GitHub's infrastructure shall be used. The home assignment is available under the FTSRG-VIMIMA01 organization.

After the teams have been formed, a new private repository will be created for each team and each team member will get access to this repository. Team members shall work in this repository. It is forbidden to (1) change the collaborators and teams settings of the repository, (2) to change the name or make the repository public, (3) to delete or transfer the repository's ownership, or (4) to turn on GiHub Pages for the repository.

The instructors will update the ris-2017 repository for each phase of the home assignment. Initially this only contains a folder structure that was copied into each team's repository (note: these are not forks!). When the instructors' repository will be updated, each team has to manually update its repository.

Therefore each team member will work with three repositories (see the figure, 20YY is the actual year). GitHub remotely stores an instance of the instructors' and the teams' repositories (Git is a distributed version control system). When a team member wants to work on some files, she has to clone the team's repository first to her own computer. During the clone operation Git will create a remote called origin that points to the repository instance stored on GitHub (see Working with Remotes). After that another remote should be configured that points to the instructors' repository in order to fetch the updates of the new phases. This configuration will be part of the HA0 Preparations phase, and the team members will have the opportunity to practice the handling of the repositories.

Notes:

  1. The remote can have any name, it is not mandatory to have the same name as the repository.
  2. The instructors' repository cannot be written by students, do not push to that repository.
  3. There are more advanced options (e.g. remote-tracking branch), but use them only if all team members are confident with using Git.

Requirements and grading

  • Roles:
    • For every phase the team shall designate a lead. For teams with 4 students every member shall lead exactly one phase, for teams with 3 students one member shall take on the lead role twice. The lead is responsible for coordinating the team members and the lead is the team's contact to the instructors for the given phase. The lead shall make sure that the issues for the given phase are closed and every team member finished their job.
    • For every phase other roles will be defined. Assigning them to team members and balancing the required work is the responsibility of the current lead. These decisions shall be documented in the team's repository.
  • Subtasks: The subtasks of each phase will be defined by issues opened by the instructors in the team's GitHub repository.
    • The progress of the subtask shall be documented in these issues, and all issues shall be finished and closed in order to complete the phase.
    • Working on the solution of each subtask should preferably go into a separate Git branch. However, the final solution shall be always merged to the master branch, as only content of the master branch will be evaluated.
  • Grading criteria
    • As this is a course about quality, the quality of the artefacts produced substantially determines the grades. For example, the review documentation should be clear and consistent, the test code written should be properly formatted and easy to understand, and commits should be structured.
    • As grading is based on quality and not quantity, every subtasks is graded on a qualitative scale (e.g. "Poor"/"Fair"/"Good"/"Excellent" or "Not OK"/"OK"), and allocation is done based on the following general guidelines:
      • 0-3 points: some subtasks have not been completed at all or major issues with the solution (several "Poor" grades)
      • 4 points: minimal-effort solution or several issues with the submitted artefacts ("Poor" grades)
      • 5-6 points: all subtasks have been solved with average quality, the team completed what was necessary to do, but not much effort/thought was put into the solution (mostly "Fair" grades)
      • 7-8 points: all subtasks have been completed, most of them with good quality (mostly "Fair" and "Good" grades) 
      • 9-10 points: all subtasks have been completed with good quality, and at least one of the tasks received "Excellent", meaning that the team has put some extra effort to come up with an outstanding solution. 
    • Every artefact submitted for evaluation is required to be written in English, however the command of English language will not be evaluated.
    • For every phase the specific grading criteria will be listed on this page.
    • Each team member will get the same points for each phase, unless the team reports different work distribution.
      • For some advices about how to manage problematic team members see: "Coping with Hitchhikers and Couch Potatoes on Teams" (in the appendix of this article)
  • Results
    • The team's points can be viewed on this page: Link 
    • Each team will receive detailed feedback on the home assignments via the team's GitHub wiki.

HA0 Preparations

To be published:  1st week
Deadline: 2017-09-17 23:59
Goal: Get familiar with the GitHub repository and tools used in the home assignments, try out teamwork, plan for future phases.
Tasks: See the issues in the team's repository.
Grading: This phase won't be graded, but we will check whether the team have completed all the issues, because these tasks are essential for the completion of the subsequent phases.

 

HA1 Specification review

To be published:  3rd week
Deadline: 2017-10-02 18:00
Goal: Practice review techniques by reviewing the textual requirement specification of a module of a sample system (this module will be used in the subsequent assignment phases).
Tasks: See the issues in the team's repository.
Grading:
  • How thorough was the definition of the review process and criteria used by the team (teams have to adapt the examples seen in the lecture).
  • How thorough was the review of the specification (what types of issues were found, quality of review comments...).
  • How "professional" is the inspection report (the report is a formal document sent to the customer: is it specific, does it summarize the performed work, does it evaluate the specification).
  • How complete is the requirement table, are requirements identifiable and traceable.
  • Teamwork (were the tasks balanced, have members verified each other's work, was progress monitored, tasks finished on time...).

 

HA2 Development testing

To be published:  6th week
Deadline: 2017-10-23 18:00
Goal: Practice development testing by performing code reviews, writing isolated unit tests and measuring code coverage.
Tasks: See the issues in the team's repository.
Grading:
  • How thorough was the manual code review of the existing class.
  • Quality of the unit tests designed and implemented for the existing method (test code is well-structured, readable, maintainable, conforms to the existing coding conventions, low technical dept, unit is isolated...).
  • Quality of the newly implemented method and its tests (see above) plus whether TDD was used and it is evident from the commits.
  • Teamwork (were the tasks balanced, have members verified each other's work, was progress monitored, tasks finished on time...).

 

HA3 Integration testing

To be published:  8th week
Deadline: 2017-11-12 23:59
Goal: Practice integration testing by creating test scenarios and test execution frameworks for module and platform integration.
Tasks: See the issues in the team's repository.
Grading:
  • Quality of test scenario design (are the selected scenarios important, do they offer a broad coverage of situations, is the selection justified, quality of definition).
  • Quality of the implementation of the test framework (code quality, elegant design, functionality provided to the tests).
  • Execution and analysis of the test results (thoroughness, identifying problems, reproducibility).
  • Teamwork (were the tasks balanced, have members verified each other's work, was progress monitored, tasks finished on time...).

HA4 Test generation

To be published:  11th week
Deadline: 2017-11-27 18:00
Goal: Get to know tools generating tests from models or source code.
Tasks: See the issues in the team's repository.
Grading:
  • Model-based: quality of the test model (is it valid w.r.t to the specification, does it contain enough details), quality of the test adaptation code, analysis of the test results (thoroughness, identifying problems...).
  • Code-based: how well the tool was used (e.g. was it just run with default settings or the team tried to improve the test generation with different configurations), how thorough was the evaluation of the generated tests.
  • Teamwork (were the tasks balanced, have members verified each other's work, was progress monitored, tasks finished on time...).