Main Purpose
- This idea has been initiated for preparing the INCORE workshop at Nov 15, 2023.
- There will be more than 50 users in the workshop
- There will be chances that 50 users access Geoserver and IN-CORE services at the same time
- Needed to check the performance to see if the Geoserver and IN-CORE services are good enough to provide the prompt response with many user requests
- This documentation might be helpful to others to do the similar load or stress test for there own project
Test Method
- Test was done by Locust
- An open source load testing tool that allows users to write test scenarios in Python code, to simulate large numbers of users interacting with the given application.
- Simulated 70 users hit the server at the same time
- In case of IN-CORE GeoServer
- Used two types of dataset
- vector
- Joplin building inventory
- This is the biggest dataset in size for the workshop material, making it proper candidate to measure the maximum stress to the server.
- raster
- Lumberton flood
- This is also the biggest raster dataset for the workshop, so should be the best for checking the maximum stress for the workshop purpose
- vector
- Used two types of dataset
In case of IN-CORE Services
- Various services are tested:
- Data services
- DFR3 services
- Hazard services, including several operations:
- Fetch hazard
- Create model-based hazard
- Create dataset-based hazard
- Update hazard values
- The implementation and description of this locust test are in IN-CORE GitHub repository. Please see here for more details.
- Various services are tested:
Steps for testing
Install locust
pip install locust
Create a Python script (Please see here for more implementation details)
import json import logging import mimetypes import time from http.client import HTTPConnection from locust import HttpUser, task, between from codecs import encode class MyUser(HttpUser): wait_time = between(1, 5) # Add the desired wait time between tasks headers = { "Authorization": "" } ##### Debugging ##### # HTTPConnection.debuglevel = 1 # logging.basicConfig() # logging.getLogger().setLevel(logging.DEBUG) # requests_log = logging.getLogger("requests.packages.urllib3") # requests_log.setLevel(logging.DEBUG) # requests_log.propagate = True @task def geoserver(self): # Joplin building inventory self.client.get("/geoserver/incore/wms?service=WMS&version=1.1.0&request=GetMap&layers=incore%3A60622b01c57ada48e48d7013&bbox=-94.58364094982268%2C37.01554491871878%2C-94.40576902948776%2C37.14723277660234&width=768&height=568&srs=EPSG%3A4326&styles=&format=image/png") # Lumberton flood self.client.get("/geoserver/incore/wms?service=WMS&version=1.1.0&request=GetMap&layers=incore%3A5f4d02e352147a614c71960b&bbox=-79.212607077%2C34.562496808%2C-78.973249171%2C34.705963648&width=768&height=460&srs=EPSG%3A4326&styles=&format=image/png") @task def dataset(self): # Get tornado dataset (60a44ae77da24b7b5ba0e86f) self.client.get("/data/api/datasets/60a44ae77da24b7b5ba0e86f", headers=self.headers) # Get building dataset (5dbc8478b9219c06dd242c0d) self.client.get("/data/api/datasets/5dbc8478b9219c06dd242c0d", headers=self.headers) # Get socio_demographic_data = "60fb4241544e944c3cedb507" self.client.get("/data/api/datasets/60fb4241544e944c3cedb507", headers=self.headers) @task def dfr3(self): # GET /fragilities/{id} self.client.get("/dfr3/api/fragilities/5b47b2d7337d4a36187c61c9", headers=self.headers) # GET /mappings/{id} self.client.get("/dfr3/api/mappings/5b47b2d9337d4a36187c7564", headers=self.headers) # GET /repairs/{id} self.client.get("/dfr3/api/repairs/6513058862ba036575da5fca", headers=self.headers) @task def hazard_earthquake(self): id = "5b902cb273c3371e1236b36b"; url = "/hazard/api/earthquakes/" earthquake_model_json = {...} earthquake_dataset_json = {...} points = "..." files = [ {"name": "eq-dataset-SA.tif", "location": "/data/hazard/earthquake/eq-dataset-SA.tif"}, {"name": "eq-dataset-PGA.tif", "location": "/data/hazard/earthquake/eq-dataset-PGA.tif"} ] # GET /earthquakes/{id} self.client.get(f"{url}{id}", headers=self.headers) # POST /earthquake for model based self.post_hazard_model(url, self.headers, earthquake_model_json, "earthquake") # POST /earthquake for dataset based self.post_hazard_dataset(url, self.headers, earthquake_dataset_json, "earthquake", files) # POST /earthquakes/{id}/values self.post_hazard_values(url, id, self.headers, points)
Start the Locust Master
locust -f <<filename_of_above_code>>
- Configure and start a test
- Go to
http://localhost:8089
- The web browser shows the user interface
- User means total number of the user to test
- Spawn means how much users are accessing the test for each second
- Go to
- Monitor and analyze the results
What to test
- In case of IN-CORE GeoServer
- Memory
Tested with JAVA memory heap size
Fortunately there were env variable for setting this.
Changed the variables in helm chart and deployed multiple times with various setting to incore dev cluster
Tested with the heap size of minimum of 256m, 512m, 1024m and maximum to 1G
Confirmed if the variable change actually modified geoserver's memory size by
Exec into geoserver container and do
ps -wwaux
In Geoserver GUI, go to
About & Status
→Server Status
in the left side menu
Testing with the various maximum memory changes did not make any effect either
Tested with minimum 1024 and maximum 2G and it was almost identical as the other tests
- Result
Changing memory size doesn't improve any performance in the test
- Memory
instance | spawn | heap size | min average time (ms) | max average time (ms) | average time after 15 sec (ms) | |
joplin buildings | dev | 50 users / 1 new user per sec | 256 | 140 | 10344 | 2239 |
Lumberton flood | dev | 50 users / 1 new user per sec | 256 | 174 | 10468 | 1716 |
Joplin buildings | dev | 50 users / 1 new user per sec | 512 | 171 | 14565 | 2360 |
Lumberton flood | dev | 50 users / 1 new user per sec | 512 | 178 | 10390 | 1674 |
Joplin buildings | dev | 50 users / 1 new user per sec | 1024 | 186 | 10596 | 1753 |
Lumberton flood | dev | 50 users / 1 new user per sec | 1024 | 167 | 9954 | 1665 |
- In case of IN-CORE Services
- Make sure there are no failures and the response time is reasonable.
- Monitor the cluster performance during the load test.