#Interactive Session by Srithanga Aishvarya T, "Machine Learning Model to automate performance test script development using Jmeter" at #ATAGTR2023.

#Interactive Session by Srithanga Aishvarya T, "Machine Learning Model to automate performance test script development using Jmeter" at #ATAGTR2023.

• 1.
  G lobal Testing Retreat #ATAGTR2023 M A C H I N E L E A R N I N G M O D E L T O A U T O M A T E P E R F O R M A N C E T E S T S C R I P T D E V E L O P M E N T U S I N G J M E T E R Srithanga Aishvarya, Thangaraj Global Testing Retreat #ATAGTR2023
• 2.
  G lobal Testing Retreat #ATAGTR2023 G lobal T esting Retreat #ATAGTR2023 PERFORMANCE TESTING IN A NUTSHELL • Testing a system under heavy load conditions to ensure the best customer experience • Scripts are developed to simulate user actions • Below Dynamic values have to be captured in the script o Session id, o Cookies o User ids, passwords etc
• 3.
  G lobal Testing Retreat #ATAGTR2023 G lobal T esting Retreat #ATAGTR2023 SOLUTION • Develop a custom machine learning model to automate the script development process • Combination of decision tree, string matching and rule-based approaches • Leverage JMeter – Industry standard open source tool for the same
• 4.
  G lobal Testing Retreat #ATAGTR2023 G lobal T esting Retreat #ATAGTR2023 WHY BUILD CUSTOM ML MODEL? • Existing models such as BERT and GPT are mostly NLP Models • Lack of domain understanding • Understanding script structure and complexity for interpretation requires extensive customization • Preprocessing and post processing pipelines needed to transform scripts to format GPT can understand
• 5.
  G lobal Testing Retreat #ATAGTR2023 G lobal T esting Retreat #ATAGTR2023 ARCHITECTURE Auto Correlate Run Report Reuse Standardization Debug Developed Jmeter script Reports (.csv) Developed JMeter script Jmeter script and Result File Autocorrelation – Identify and implement correlation Standardization – Standardize script as per best practices (Transaction names, Text checks etc). Run – Execute the scripts Debug - Identify errors and fix the same Alerts & Reporting – Alert the users in case of manual intervention needed , Detailed reports on changes made, Execution report Reuse – Compare existing script of previous release to copy customizations
• 6.
  G lobal Testing Retreat #ATAGTR2023 G lobal T esting Retreat #ATAGTR2023 D E E P D I V E I N TO T H E M O D E L Inputs: Case 1 : No pre existing script Recording of the script Results file of recorded script Case 2: Baseline script available Baseline Script Excel input for standardization Interim Outputs: Intermediate versions of the script Excel report detailing the below information: • Newly added requests • Removed requests • Existing requests with body changes Outputs: Case 1: Developed JMeter script Excel report detailing the below information: • Requests which require manual intervention • Requests where changes have been made Case 2: Baseline Script Excel report detailing the below information: • Requests which require manual intervention • Requests where changes have been made
• 7.
  G lobal Testing Retreat #ATAGTR2023 G lobal T esting Retreat #ATAGTR2023 DEEP DIVE INTO THE MODEL Libraries which can be used • NLP Libraries - SpaCy,NTLK • Parsing Script - BeautifulSoup, Regex • ML libraries - TensorFlow,Keras,PyTorch • Data processing - Pandas,NumPy • Model evaluation -Scikit-Learn Step 1: Data Collection and Annotation Collect and annotate a dataset of JMeter scripts Step 2: Data Preprocessing - Preprocess JMeter scripts to convert them into a suitable format for model input Step 3: Feature Engineering - Extract relevant features from the preprocessed scripts Step 4: Model Selection - Choose a machine learning or deep learning model for classification Step 5: Training - Split the dataset into training and validation sets and train the model on the training data Step 6: Evaluation- Evaluate the model's performance on the validation dataset Step 7: Post-Processing - Develop post-processing logic to refine the identification results Step 8: Real-World Testing - Test the model in real-world scenarios Step 9: Monitoring and Maintenance - Continuously monitor and update the model as needed
• 8.
  G lobal Testing Retreat #ATAGTR2023 G lobal T esting Retreat #ATAGTR2023 TRAI NI NG THE MODEL The model is to be trained on realworld scripts across below parameters • Protocol (Web, SAP Web etc) • Technology (SAP, Salesforce, Pega etc) • Domain level knowledge (Retail, Finance etc) • Error Codes and debugging error messages • Performance Testing Best practices
• 9.
  G lobal Testing Retreat #ATAGTR2023 G lobal T esting Retreat #ATAGTR2023 BENEFITS & FURTHER ENHANCEMENTS • Cost Savings and Effort savings • Reduced turnaround time Further Enhancements: Feasibility of integration of this model with Language Models (LLMs) to enhance web UI automation as an additional feature.