VMware Weathervane

2020: Weathervane 2.0 utility

On March 2, 2020 the VMware company submitted the updated Weathervane 2.0 utility which allows to make testing of performance of clusters of Kubernetes under loading.

According to the developer, this means can be useful in the following cases:

• When it is necessary to compare two clusters on performance (for example, on the different equipment)
• When it is necessary to understand influence of configuration changes of a cluster on performance
• When it is necessary to check correctness of setup of a new cluster before start it on production Wednesday

For start of Weathervane it is necessary to create images of containers, to prepare a configuration file and to start a benchmark. Further the utility itself will unroll containers in a cluster, will start applications and will collect results of testing.

Weathervane deploit a benchmark-application on nodes and gives there loading which is generated through the Workload driver component. This driver can be located both together with a benchmark-application, and in the external environment, in a separate cluster.

Weathervane can be set on a permanent load for the fixed number of the simulated users, and it is possible to configure on search of the maximum number of users so that the requirements of quality-of-service (QoS) were fulfilled. In the latter case the maximum number of WvUsers which the cluster is capable to sustain will be test result. Actually, this parameter also needs to be used for comparison of clusters on performance, noted in VMware.

Here is how Weathervane solution components look (the Run harness component is responsible for execution of test runs and obtaining results of testing):

Weathervane uses the multilevel web application which includes stateless and stateful services. The user can select one of these types of deployment of applications. Several copies of applications can be started within one run that allows to scale testing in big clusters.

The Weathervane application consists of several tiers. The logic of the application is implemented through Java services, started on the Tomcat server[[Apache Tomcat|]] which communicate through REST API and messages of RabbitMQ, and Zookeeper use for coordination. Bekend-storages are implemented by means of PostgreSQL and Cassandra. A frontend Web servers and a proxy cache servers are implemented on Nginx.

Recently VMware held testing of clusters using Weathervane 2 for two scenarios. In the first case of the colleague compared performance of two generations of servers:

The result for different number of mikroinstans of applications turned out such:

Apparently from the picture if to judge by WvUsers number, then new iron won against old twice (there and cores in processors are twice more, but they work at smaller frequency). And on equivalent number of users cluster performance on the new equipment was 15-29% higher.

The second test was done on different network configurations of clusters Kubernetes which were scaled up to 16 copies of applications. In the first case the mechanics of Flannel/VXLAN, and in the second - Flannel/host-gw which won against the first approximately for 10%^[1] was used^[2]:

Notes