Test-driven Fault Navigation for Debugging Reproducible Failures

Keywords: Debugging, Testing, Spectrum-based Fault Localization, Developer Ranking, Back-in-time Debugging, Likely Invariants, Squeak/Smalltalk

The correction of software failures tends to be very cost-intensive because their debugging is an often time-consuming development activity. During this activity, developers largely attempt to understand what causes failures: Starting with a test case that reproduces the observable failure they have to follow failure causes on the infection chain back to the root cause (defect). This idealized procedure requires deep knowledge of the system and its behavior because failures and defects can be far apart from each other. Unfortunately, common debugging tools are inadequate for systematically investigating such infection chains in detail. Thus, developers have to rely primarily on their intuition and the localization of failure causes is not time-efficient. To prevent debugging by disorganized trial and error, experienced developers apply the scientific method and its systematic hypothesis-testing. However, even when using the scientific method, the search for failure causes can still be a laborious task. First, lacking expertise about the system makes it hard to understand incorrect behavior and to create reasonable hypotheses. Second, contemporary debugging approaches provide no or only partial support for the scientific method.

We present test-driven fault navigation as an interconnected guide for debugging reproducible failures with the scientific method. Based on the analysis of passing and failing test cases, we reveal anomalies and integrate them into a breadth-first search that leads developers to defects. This systematic search consists of four specific navigation techniques that together support the creation, evaluation, and refinement of failure cause hypotheses for the scientific method. First, structure navigation localizes suspicious system parts and restricts the initial search space. Second, team navigation recommends experienced developers for helping with failures. Third, behavior navigation allows developers to follow emphasized infection chains back to root causes. Fourth, state navigation identifies corrupted state and reveals parts of the infection chain automatically. We implement test-driven fault navigation in our Path Tools framework for the Squeak/Smalltalk development environment and limit its computation cost with the help of our incremental dynamic analysis. This lightweight dynamic analysis ensures a feeling of immediacy when debugging with our tools by splitting the run-time overhead over multiple test runs depending on developers’ needs. Hence, our test-driven fault navigation in combination with our incremental dynamic analysis answers important questions in a short time: where to start debugging, who understands failure causes best, what happened before failures, and which program entities are in question.

Test Quality Feedback - Improving Effectivity and Efficiency of Unit Testing

Keywords: Unit Tests, Dynamic Analysis, Test Quality Feedback, Squeak/Smalltalk

Writing unit tests for a software system enhances the confidence that a system works as expected. Since time pressure often prevents a complete testing of all application details developers need to know which new tests the system requires. Developers also need to know which existing tests take the most time and slow down the whole development process. Missing feedback about less tested functionality and reasons for long running test cases make it, however, harder to create a test suite that covers all important parts of a software system in a minimum of time. As a result a software system may be inadequately tested and developers may test less frequently.

Our approach provides test quality feedback to guide developers in identifying missing tests and correcting low-quality tests. We provide developers with a tool that analyzes test suites with respect to their effectivity (e.g., missing tests) and efficiency (e.g., time and memory consumption). We implement our approach, named PathMap, as an extended test runner within the Squeak Smalltalk IDE and demonstrate its benefits by improving the test quality of representative software systems.

Immediacy through Interactivity: Online Analysis of Run-time Behavior

Keywords: Program Comprehension, Dynamic Analysis, Object Collaboration, Testing, Back-in-time Debugging, Squeak/Smalltalk

Visualizations of actual run-time data support the comprehension of programs, like examples support the explanation of abstract concepts and principles. Unfortunately, the required run-time analysis is often associated with an inconvenient overhead that renders current tools impractical for frequent use.

We propose an interactive approach to collect and present run-time data. An initial shallow analysis provides for immediate access to visualizations of run-time information. As users explore this information, it is incrementally refined on-demand. We present an implementation that realizes our proposed approach and enables developers to instantly explore run-time behavior of selected code entities. We evaluate our interactive approach by measuring time and memory overhead in the context of ten different-sized projects. Our empirical results show that run-time data for an initial overview can be collected in less than 300 milliseconds for 95% of cases.

Model-based Source Code Editing

Keywords: Program Comprehension, Model-driven Development, Programming, Class Diagrams, UML, Squeak/Smalltalk

How much different is a class in source code and a UML diagram?

Often modeling and programming worlds are unnecessarily separated from each other. This leads to several synchronization issues that make software development harder than necessary. For example, a change in the overall class design (UML diagram) needs manual adaption in source code. Also an edit in source code is not reflected in diagrams so that they tend to be out of date and in worst case useless for programmers.

We propose model-based source code editing that makes no difference between a model and the underlying source code. Our single-source technique relies on source code and test cases as executable program representations and allows developers to derive higher-level abstractions. This baseline enables a comfortable synchronization mechanism that reflects changes in the model directly in source code and vice versa. Based on such mid-level abstractions, developers can edit these models and the related source code is changed automatically in background. PathView is our model-based source code editor that semi-automatically creates class diagrams for a specific concern. Depending on the development tasks, developers can now choose whether they prefer a standard browser or a model and they can switch whenever desired.

Type Harvesting: A Practical Approach to Obtaining Typing Information in Dynamic Programming Languages

Keywords: Dynamically Typed Programming Languages, Type Inference, Dynamic Analysis, Unit Tests, Program Comprehension, Squeak/Smalltalk

Dynamically typed programming languages are powerful tools for rapid software development. However, there are scenarios that would benefit from actual type information being available—e. g., code generation and optimisation as well as program comprehension. Since code written in such languages usually makes little or no explicit assumptions about types, static type inference is not particularly well suited to obtain the desired information.

Our approach introduces type harvesting, a practical approach to obtaining type information. For a specified system under observation, we wrap all corresponding methods and closely observe their types at run-time. Type harvesting allows for exploiting unit tests to automatically obtain type information for a code base. Its evaluation, using several complex applications, shows that type harvesting yields excellent results with high precision.

Explicit Use-case Representation in Object-oriented Programming Languages

Keywords: Use-cases, Separation of Concerns, Traceability, Dynamic Analysis, Python

Use-cases are considered an integral part of most contemporary development processes since they describe a software system’s expected behavior from the perspective of its prospective users. However, the presence of and traceability to use-cases is increasingly lost in later more code-centric development activities. Usecases, being well-encapsulated at the level of requirements descriptions, eventually lead to crosscutting concerns in system design and source code. Tracing which parts of the system contribute to which use-cases is therefore hard and so limits understandability.

We propose an approach to making use-cases firstclass entities in both the programming language and the runtime environment. Having use-cases present in the code and the running system will allow developers, maintainers, and operators to easily associate their units of work with what matters to the users. We suggest the combination of use-cases, acceptance tests, and dynamic analysis to automatically associate source code with usecases. We present UseCasePy, an implementation of our approach to use-case-centered development in Python, and its application to the Django Web framework.

Please Note: The following screencast presents a similar implementation as part of our Path tools framework for Squeak/Smalltalk:

PhidgetLab: Crossing the Border from Virtual to Real-World Objects

Keywords: Etoys, Phidgets, PhidgetLab, Squeak/Smalltalk, Design Thinking

Teaching pupils the ideas behind objects in programming languages can be difficult since these concepts are mostly abstract and not comprehensible at first sight. Etoys as a visual programming environment counters such issues by introducing visible objects and simple tiles for programming them. However, all of these objects can only be experienced virtually on the screen. This paper presents PhidgetLab, a programming environment for electronic components (Phidgets) realised on top of the Etoys environment. PhidgetLab helps crossing the border from virtual to real-world objects. Pupils interact with tangible objects that are seamlessly connected to the digital world. PhidgetLab was evaluated in a case study with 22 pupils, following the principles of the De- sign Thinking methodology and comprised the realisation of five prototypes within a short period of time.

Dynamic Contract Layers

Keywords: Design by Contract, Context-oriented Programming, Software Composition, Python

Design by Contract (DBC) is a programming technique to separate contract enforcement from application code. DBC provides information about the applicability of methods and helps to narrow down the search space in case of a software failure. However, most DBC implementations suffer from inflexibility: Contract enforcement can only be activated or deactivated at compile-time or start-up, contracts are checked globally and cannot be restricted in their scope such as to the current thread of execution, and contracts cannot be grouped according to the concerns they relate to.

In this paper, we present dynamic contract layers (DCL) for fine-grained and exible contract management. Based on ideas from context-oriented programming, we extend DBC by a grouping mechanism for contracts, thread-local activation and deactivation of such groups, and selective contract enforcement at run-time. PyDCL, our proof-of-concept implementation of DCL, is built onto ContextPy, our COP extension for the Python programming language. We evaluate our approach by applying PyDCL contracts to the Moin-Moin Wiki framework.