Overview‎ > ‎

Motivation

Experimental evaluation of middleware usually requires instances of a stub application deployed in multiple hosts to accurately reproduce the impact of distribution and avoid mutual interference. The amount of required hardware resources to assess systems aimed at large scale is thus significant. Moreover, a distributed system is inherently difficult to observe due to the lack of a time reference and central control point.

As a consequence, many middleware components have performance bottlenecks and even outright flaws that are exposed only in production in large scale deployments and with real workloads. For instance, in small scale deployments with several instances in the same or in a few physical hosts, concurrency issues might not be exposed, scalability challenges do not arise (e.g. unstable performance or oscillation), and run-time complexity of embedded algorithms is left unchecked.

The goal of this project is to provide a platform that has the following desirable properties:
  • Practical large scale. It should be possible to \uline{improve the scale of testing by 100 times in the same hardware}, both by virtualizing time (i.e. trading hardware resources for longer running tests) and simulating the environment, while at the same time making test outcomes more deterministic and thus repeatable.
  • Global observation without interference. Once the time across the system is virtualized, it is possible to get a global observation of all operation and system variables. Contrary to execution in a real environment, there is no overhead introduced by observation and control, even debugging, so execution time can be considered for analysis. Moreover, distributed execution and state can be observed consistently by any simple debuggers.
  • Deployment of incomplete systems. When in a real execution, still in the development phase, it is necessary to evaluate the system for different environments (e.g. network configurations) and software components (i.e. the application layer on top of the middleware). Our goal is that  environments and software models can be replaced by simulation models, and incorporated in a standard test harness to be run automatically as code evolves.
  • Automated ``What-If'' analysis. By resorting to simulated components and running the system with varying parameters, the impact of extreme environments can be assessed, exploring even conditions that are not yet possible in practice. In fact, this can even be automated by having parameters to be generated automatically in order to seek for a given observable condition, e.g. the number of CPU cores to which some code scales up to.