Load Test using Locust

Main Purpose

• This idea has been initiated for preparing the INCORE workshop at Nov 15, 2023. 
• There will be more than 50 users in the workshop
• There will be chances that 50 users access Geoserver and IN-CORE services at the same time 
• Needed to check the performance to see if the Geoserver and IN-CORE services are good enough to provide the prompt response with many user requests
• This documentation might be helpful to others to do the similar load or stress test for there own project

Test Method

• Test was done by Locust
  • An open source load testing tool that allows users to write test scenarios in Python code, to simulate large numbers of users interacting with the given application. 
  • Simulated 70 users hit the server at the same time
• In case of IN-CORE GeoServer
  • Used two types of dataset
    • vector 
      • Joplin building inventory
      • This is the biggest dataset in size for the workshop material, making it proper candidate to measure the maximum stress to the server.

      • https://incore-dev.ncsa.illinois.edu/geoserver/incore/wms?service=WMS&version=1.1.0&request=GetMap&layers=incore%3A60622b01c57ada48e48d7013&bbox=-94.58364094982268%2C37.01554491871878%2C-94.40576902948776%2C37.14723277660234&width=768&height=568&srs=EPSG%3A4326&styles=&format=image/png

    • raster
      • Lumberton flood
      • This is also the biggest raster dataset for the workshop, so should be the best for checking the maximum stress for the workshop purpose 

      • https://incore-dev.ncsa.illinois.edu/geoserver/incore/wms?service=WMS&version=1.1.0&request=GetMap&layers=incore%3A5f4d02e352147a614c71960b&bbox=-79.212607077%2C34.562496808%2C-78.973249171%2C34.705963648&width=768&height=460&srs=EPSG%3A4326&styles=&format=image/png

• In case of IN-CORE Services

  • Various services are tested:
    • Data services
    • DFR3 services
    • Hazard services, including several operations:
      • Fetch hazard
      • Create model-based hazard
      • Create dataset-based hazard
      • Update hazard values
  • The implementation and description of this locust test are in IN-CORE GitHub repository. Please see here for more details.

• Steps for testing

  • Install locust  

    pip install locust

  • Create a code (Please see here for more implementation details)

    import json
    import logging
    import mimetypes
    import time
    from http.client import HTTPConnection
    from locust import HttpUser, task, between
    from codecs import encode
    
    class MyUser(HttpUser):
        wait_time = between(1, 5)  # Add the desired wait time between tasks
    
        headers = {
            "Authorization": ""
        }
    
        ##### Debugging #####
        # HTTPConnection.debuglevel = 1
        # logging.basicConfig()
        # logging.getLogger().setLevel(logging.DEBUG)
        # requests_log = logging.getLogger("requests.packages.urllib3")
        # requests_log.setLevel(logging.DEBUG)
        # requests_log.propagate = True
    
        @task
        def geoserver(self):
            # Joplin building inventory
            self.client.get("/geoserver/incore/wms?service=WMS&version=1.1.0&request=GetMap&layers=incore%3A60622b01c57ada48e48d7013&bbox=-94.58364094982268%2C37.01554491871878%2C-94.40576902948776%2C37.14723277660234&width=768&height=568&srs=EPSG%3A4326&styles=&format=image/png")
    
            # Lumberton flood
            self.client.get("/geoserver/incore/wms?service=WMS&version=1.1.0&request=GetMap&layers=incore%3A5f4d02e352147a614c71960b&bbox=-79.212607077%2C34.562496808%2C-78.973249171%2C34.705963648&width=768&height=460&srs=EPSG%3A4326&styles=&format=image/png")
    
        @task
        def dataset(self):
            # Get tornado dataset (60a44ae77da24b7b5ba0e86f)
            self.client.get("/data/api/datasets/60a44ae77da24b7b5ba0e86f", headers=self.headers)
    
            # Get building dataset (5dbc8478b9219c06dd242c0d)
            self.client.get("/data/api/datasets/5dbc8478b9219c06dd242c0d", headers=self.headers)
    
            # Get socio_demographic_data = "60fb4241544e944c3cedb507"
            self.client.get("/data/api/datasets/60fb4241544e944c3cedb507", headers=self.headers)
    
        @task
        def dfr3(self):
            # GET /fragilities/{id}
            self.client.get("/dfr3/api/fragilities/5b47b2d7337d4a36187c61c9", headers=self.headers)
    
            # GET /mappings/{id}
            self.client.get("/dfr3/api/mappings/5b47b2d9337d4a36187c7564", headers=self.headers)
    
            # GET /repairs/{id}
            self.client.get("/dfr3/api/repairs/6513058862ba036575da5fca", headers=self.headers)
    
        @task
        def hazard_earthquake(self):
            id = "5b902cb273c3371e1236b36b";
            url = "/hazard/api/earthquakes/"
            earthquake_model_json = {...}
            earthquake_dataset_json = {...}
            points = "..."
    
            files = [
                {"name": "eq-dataset-SA.tif",
                 "location": "/data/hazard/earthquake/eq-dataset-SA.tif"},
                {"name": "eq-dataset-PGA.tif",
                 "location": "/data/hazard/earthquake/eq-dataset-PGA.tif"}
            ]
    
            # GET /earthquakes/{id}
            self.client.get(f"{url}{id}", headers=self.headers)
    
            # POST /earthquake for model based
            self.post_hazard_model(url, self.headers, earthquake_model_json, "earthquake")
    
            # POST /earthquake for dataset based
            self.post_hazard_dataset(url, self.headers, earthquake_dataset_json, "earthquake", files)
    
            # POST /earthquakes/{id}/values
            self.post_hazard_values(url, id, self.headers, points)

  • Run locust -f <<filename_of_above_code>> 
  • Go to http://localhost:8089 
  • 
  • The web browser shows the user interface
    • User means total number of the user to test
    • Spawn means how much users are accessing the test for each second

What to test

In case of IN-CORE GeoServer

Memory

• Tested with JAVA memory heap size
• Fortunately there were env variable for setting this.
• Changed the variables in helm chart and deployed multiple times with various setting to incore dev cluster
• Tested with the heap size of minimum of 256m, 512m, 1024m and maximum to 1G
• Confirmed if the variable change actually modified geoserver's memory size by 
  • Exec into geoserver container and do ps -wwaux 
  • In Geoserver GUI, go to About & Status  → Server Status  in the left side menu
• Testing with the various maximum memory changes did not make any effect either
  • Tested with minimum 1024 and maximum 2G and it was almost identical as the other tests

Result

• Changing memory size doesn't improve any performance in the test