Re: pgAdmin4 Feature Tests intermittent failures fix
Shubham Agarwal <[email protected]>
| Newsgroups | gmane.comp.db.postgresql.pgadmin.devel |
|---|---|
| Message-ID | <CAKbCA9RO5rZo8n+Z6hdzn0PMqTMKin4yx-Htf+pf09fUqYFTvQ@mail.gmail.com> |
Hi Akshay, PFA the patch with fixed review comments. On Thu, Aug 22, 2019 at 4:26 PM Shubham Agarwal < [email protected]> wrote: > Yes, I agreed too. > The only thinking was if the test case objective is to check the debugger, > then it should fail instead of skipping it to come into direct notice. > If the only test case for debugger gets skip, then the functionality will > never be tested. > > On Thu, Aug 22, 2019 at 4:06 PM Ashesh Vashi < > [email protected]> wrote: > >> On Thu, Aug 22, 2019 at 4:04 PM Dave Page <[email protected]> wrote: >> >>> >>> >>> On Thu, Aug 22, 2019 at 11:26 AM Shubham Agarwal < >>> [email protected]> wrote: >>> >>>> Hi Akshay, >>>> Thanks for the overall review and comments. >>>> >>>> From the logs and screenshot shared by you, it is cleared that the >>>> debugger plugin is not installed for PG12 server, hence resulting in the >>>> failures of "xss_checks_pgadmin_debugger_test.py" test case. >>>> Please install and re-execute the test cases. >>>> >>> >>> It should skip the test if that's the case. >>> >> +1 >> >> -- Thanks, Ashesh >> >>> >>> >>>> >>>> The rest of the issues need some debugging since I have never >>>> encountered these failures while executing on Jenkins. >>>> >>>> Thanks and regards, >>>> Shubham Agarwal >>>> >>>> On Thu, Aug 22, 2019 at 2:45 PM Ashesh Vashi < >>>> [email protected]> wrote: >>>> >>>>> On Thu, Aug 22, 2019 at 2:43 PM Akshay Joshi < >>>>> [email protected]> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Thu, Aug 22, 2019 at 2:41 PM Ashesh Vashi < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Akshay, >>>>>>> >>>>>>> On Thu, Aug 22, 2019 at 2:29 PM Akshay Joshi < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Nice work! >>>>>>>> >>>>>>> It's big patch, and also improvement over last status. >>>>>>> Can we commit the patch, as to avoid any merging issues later? >>>>>>> >>>>>> >>>>>> Sure will do that. >>>>>> >>>>> Thanks >>>>> >>>>> -- Ashesh >>>>> >>>>>> >>>>>>> -- Thanks, Ashesh >>>>>>> >>>>>>>> >>>>>>>> I have run the feature test couple of times on EPAS 9.6, 11, 12 and >>>>>>>> PG 9.4, 10, 11, 12. Following are the review comments: >>>>>>>> >>>>>>>> - Following test cases are failing every time on *EPAS 12:* >>>>>>>> - Explain query with JIT stats... fails with error "*no such >>>>>>>> element: Unable to locate element: >>>>>>>> {"method":"xpath","selector":"//*[contains(string(), 'JIT')]"}* >>>>>>>> " >>>>>>>> - Following test cases are failing every time on *PG 12*: >>>>>>>> - History query sources and generated queries toggle...ERROR >>>>>>>> - Tests to check if Debugger is vulnerable to XSS ... ERROR >>>>>>>> - We have *retry_count* in "browser_tool_bar_test.py" and >>>>>>>> "file_manager_test.py" file. Can we make a generic function and pass the >>>>>>>> locator and retry_count it would be good. >>>>>>>> - We have also import WebDriverWait in so many files can we >>>>>>>> have one generic class or function for all the wait events. We will pass >>>>>>>> the required timeout and the locator if possible. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Aug 21, 2019 at 4:18 PM Shubham Agarwal < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Hackers, >>>>>>>>> >>>>>>>>> Attached is the patch for fixing the intermittent failures >>>>>>>>> encountered while executing the test cases multiple times. >>>>>>>>> >>>>>>>>> Please review the patch. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Thanks & Regards, >>>>>>>>> Shubham Agarwal >>>>>>>>> EnterpriseDB Corporation >>>>>>>>> >>>>>>>>> The Postgres Database Company >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Thanks & Regards* >>>>>>>> *Akshay Joshi* >>>>>>>> >>>>>>>> *Sr. Software Architect* >>>>>>>> *EnterpriseDB Software India Private Limited* >>>>>>>> *Mobile: +91 976-788-8246* >>>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> *Thanks & Regards* >>>>>> *Akshay Joshi* >>>>>> >>>>>> *Sr. Software Architect* >>>>>> *EnterpriseDB Software India Private Limited* >>>>>> *Mobile: +91 976-788-8246* >>>>>> >>>>> >>>> >>>> -- >>>> Thanks & Regards, >>>> Shubham Agarwal >>>> EnterpriseDB Corporation >>>> >>>> The Postgres Database Company >>>> >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >> > > -- > Thanks & Regards, > Shubham Agarwal > EnterpriseDB Corporation > > The Postgres Database Company > -- Thanks & Regards, Shubham Agarwal EnterpriseDB Corporation The Postgres Database Company
feature_test_fix_v1.patch
(application/octet-stream, 17.3 KB)
diff --git a/web/pgadmin/feature_tests/browser_tool_bar_test.py b/web/pgadmin/feature_tests/browser_tool_bar_test.py
index 2d504d8ae..63e432e66 100644
--- a/web/pgadmin/feature_tests/browser_tool_bar_test.py
+++ b/web/pgadmin/feature_tests/browser_tool_bar_test.py
@@ -15,6 +15,7 @@ from regression.feature_utils.locators import BrowserToolBarLocators
from regression.feature_utils.base_feature_test import BaseFeatureTest
from selenium.common.exceptions import TimeoutException, \
StaleElementReferenceException
+from selenium.webdriver.common.by import By
class BrowserToolBarFeatureTest(BaseFeatureTest):
@@ -63,18 +64,10 @@ class BrowserToolBarFeatureTest(BaseFeatureTest):
self.page.toggle_open_tree_item(self.server['name'])
self.page.toggle_open_tree_item('Databases')
self.page.toggle_open_tree_item(self.test_db)
- retry_count = 0
- while retry_count < 5:
- try:
- self.page.find_by_css_selector(
- BrowserToolBarLocators.open_query_tool_button_css)\
- .click()
- break
- except (StaleElementReferenceException, TimeoutException):
- retry_count += 1
-
- self.page.find_by_css_selector(
- BrowserToolBarLocators.query_tool_panel_css)
+ self.page.retry_click(
+ (By.CSS_SELECTOR,
+ BrowserToolBarLocators.open_query_tool_button_css),
+ (By.CSS_SELECTOR, BrowserToolBarLocators.query_tool_panel_css))
def test_view_data_tool_button(self):
self.page.select_tree_item(self.test_db)
@@ -83,27 +76,14 @@ class BrowserToolBarFeatureTest(BaseFeatureTest):
self.page.toggle_open_tables_node()
self.page.select_tree_item(self.test_table_name)
- retry_count = 0
- while retry_count < 5:
- try:
- self.page.find_by_css_selector(
- BrowserToolBarLocators.view_table_data_button_css).click()
- break
- except (StaleElementReferenceException, TimeoutException):
- retry_count += 1
- self.page.find_by_css_selector(
- BrowserToolBarLocators.view_data_panel_css)
+ self.page.retry_click(
+ (By.CSS_SELECTOR,
+ BrowserToolBarLocators.view_table_data_button_css),
+ (By.CSS_SELECTOR, BrowserToolBarLocators.view_data_panel_css))
def test_filtered_rows_tool_button(self):
- retry_count = 0
- while retry_count < 5:
- try:
- self.page.find_by_css_selector(
- BrowserToolBarLocators.filter_data_button_css)\
- .click()
- break
- except (StaleElementReferenceException, TimeoutException):
- retry_count += 1
- self.page.find_by_css_selector(
- BrowserToolBarLocators.filter_alertify_box_css)
+ self.page.retry_click(
+ (By.CSS_SELECTOR,
+ BrowserToolBarLocators.filter_data_button_css),
+ (By.CSS_SELECTOR, BrowserToolBarLocators.filter_alertify_box_css))
self.page.click_modal('Cancel')
diff --git a/web/pgadmin/feature_tests/file_manager_test.py b/web/pgadmin/feature_tests/file_manager_test.py
index 3df37af84..424d7b282 100644
--- a/web/pgadmin/feature_tests/file_manager_test.py
+++ b/web/pgadmin/feature_tests/file_manager_test.py
@@ -126,21 +126,11 @@ class CheckFileManagerFeatureTest(BaseFeatureTest):
# Intermittently facing issue on first click it is not successful
# so tried couple of times.
- iteration = 0
- success = False
- while not success and iteration < 4:
- # Check for sort Ascending
- try:
- self.page.find_by_xpath("//th[@data-column='0']"
- "/div/span[text()='Name']").click()
- self.wait.until(
- EC.presence_of_element_located((
- By.CSS_SELECTOR,
- "#contents th[data-column='0'].tablesorter-headerAsc")
- ))
- success = True
- except Exception as e:
- iteration += 1
+ success = self.page.retry_click(
+ (By.XPATH,
+ "//th[@data-column='0']/div/span[text()='Name']"),
+ (By.CSS_SELECTOR,
+ "#contents th[data-column='0'].tablesorter-headerAsc"))
if not success:
raise Exception("Unable to sort in ascending order while clicked "
@@ -151,21 +141,11 @@ class CheckFileManagerFeatureTest(BaseFeatureTest):
# Click and Check for sort Descending
# Intermittently facing issue on first click it is not successful
# so tried couple of times.
- iteration = 0
- success = False
- while not success and iteration < 4:
-
- try:
- self.page.find_by_xpath("//th[@data-column='0']"
- "/div/span[text()='Name']").click()
- self.wait.until(
- EC.presence_of_element_located((
- By.CSS_SELECTOR,
- "#contents th[data-column='0'].tablesorter-headerDesc")
- ))
- success = True
- except Exception as e:
- iteration += 1
+ success = self.page.retry_click(
+ (By.XPATH,
+ "//th[@data-column='0']/div/span[text()='Name']"),
+ (By.CSS_SELECTOR,
+ "#contents th[data-column='0'].tablesorter-headerDesc"))
if not success:
raise Exception("Unable to sort in descending order while clicked "
diff --git a/web/pgadmin/feature_tests/pg_utilities_backup_restore_test.py b/web/pgadmin/feature_tests/pg_utilities_backup_restore_test.py
index 0d4d8e9f6..0e57d46b8 100644
--- a/web/pgadmin/feature_tests/pg_utilities_backup_restore_test.py
+++ b/web/pgadmin/feature_tests/pg_utilities_backup_restore_test.py
@@ -11,7 +11,6 @@ import os
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
-from selenium.common.exceptions import ElementClickInterceptedException
from regression.feature_utils.base_feature_test import BaseFeatureTest
from regression.python_test_utils import test_utils
from regression.python_test_utils import test_gui_helper
@@ -39,6 +38,11 @@ class PGUtilitiesBackupFeatureTest(BaseFeatureTest):
self.server['name']
)
)
+ if '<' in self.database_name and os.name == 'nt':
+ self.skipTest(
+ "HTML tags '<' and '>' in object name does not "
+ "work for windows so skipping the test case"
+ )
connection = test_utils.get_db_connection(
self.server['db'],
@@ -63,14 +67,11 @@ class PGUtilitiesBackupFeatureTest(BaseFeatureTest):
self.page.toggle_open_tree_item(self.database_name)
# Backup
- retry = 3
- while retry > 0:
- try:
- self.driver.find_element_by_link_text(
- NavMenuLocators.tools_menu_link_text).click()
- break
- except ElementClickInterceptedException:
- retry -= 1
+ self.page.retry_click(
+ (By.LINK_TEXT,
+ NavMenuLocators.tools_menu_link_text),
+ (By.CSS_SELECTOR,
+ NavMenuLocators.backup_obj_css))
backup_object = self.wait.until(EC.visibility_of_element_located(
(By.CSS_SELECTOR, NavMenuLocators.backup_obj_css)))
diff --git a/web/pgadmin/feature_tests/pg_utilities_maintenance_test.py b/web/pgadmin/feature_tests/pg_utilities_maintenance_test.py
index 7524f141d..5d9a4f741 100644
--- a/web/pgadmin/feature_tests/pg_utilities_maintenance_test.py
+++ b/web/pgadmin/feature_tests/pg_utilities_maintenance_test.py
@@ -48,6 +48,11 @@ class PGUtilitiesMaintenanceFeatureTest(BaseFeatureTest):
self.server['name']
)
)
+ if '<' in self.table_name and os.name == 'nt':
+ self.skipTest(
+ "HTML tags '<' and '>' in object name does not "
+ "work for windows so skipping the test case"
+ )
connection = test_utils.get_db_connection(
self.server['db'],
@@ -90,15 +95,12 @@ class PGUtilitiesMaintenanceFeatureTest(BaseFeatureTest):
self.page.toggle_open_tree_item('public')
self.page.toggle_open_tables_node()
self.page.select_tree_item(self.table_name)
- retry = 3
- while retry > 0:
- try:
- tools_menu = self.driver.find_element_by_link_text(
- NavMenuLocators.tools_menu_link_text)
- tools_menu.click()
- break
- except ElementClickInterceptedException:
- retry -= 1
+
+ self.page.retry_click(
+ (By.LINK_TEXT,
+ NavMenuLocators.tools_menu_link_text),
+ (By.CSS_SELECTOR, NavMenuLocators.maintenance_obj_css))
+
maintenance_obj = self.wait.until(EC.visibility_of_element_located(
(By.CSS_SELECTOR, NavMenuLocators.maintenance_obj_css)))
maintenance_obj.click()
diff --git a/web/pgadmin/feature_tests/query_tool_journey_test.py b/web/pgadmin/feature_tests/query_tool_journey_test.py
index fbd0d28e3..937cd97a8 100644
--- a/web/pgadmin/feature_tests/query_tool_journey_test.py
+++ b/web/pgadmin/feature_tests/query_tool_journey_test.py
@@ -308,9 +308,7 @@ class QueryToolJourneyTest(BaseFeatureTest):
query_options = self.page.find_by_css_selector(
QueryToolLocators.btn_query_dropdown)
query_options.click()
- self.page.find_by_css_selector(
- QueryToolLocators.btn_auto_commit).click()
- query_options.click() # Click again to close dropdown
+ self.page.uncheck_execute_option("auto_commit")
self._update_numeric_cell(2, 10)
@@ -320,9 +318,7 @@ class QueryToolJourneyTest(BaseFeatureTest):
query_options = self.page.find_by_css_selector(
QueryToolLocators.btn_query_dropdown)
query_options.click()
- self.page.find_by_css_selector(
- QueryToolLocators.btn_auto_commit).click()
- query_options.click() # Click again to close dropdown
+ self.page.check_execute_option("auto_commit")
def _check_history_queries_and_icons(self, history_queries, history_icons):
# Select first query history entry
@@ -422,9 +418,6 @@ class QueryToolJourneyTest(BaseFeatureTest):
def _check_cell_editable(self, cell_index):
"""Checks if a cell in the first row of the resultset is editable"""
- # self.page.check_if_element_exist_by_xpath(
- # "//div[contains(@style, 'top:0px')]//div[contains(@class, "
- # "'l{0} r{1}')]".format(cell_index, cell_index))
cell_el = self.page.find_by_xpath(
"//div[contains(@style, 'top:0px')]//div[contains(@class, "
"'l{0} r{1}')]".format(cell_index, cell_index))
diff --git a/web/pgadmin/feature_tests/query_tool_tests.py b/web/pgadmin/feature_tests/query_tool_tests.py
index 9252b93ca..868cb5d54 100644
--- a/web/pgadmin/feature_tests/query_tool_tests.py
+++ b/web/pgadmin/feature_tests/query_tool_tests.py
@@ -9,7 +9,7 @@
from __future__ import print_function
import sys
-
+import time
from selenium.common.exceptions import StaleElementReferenceException, \
ElementClickInterceptedException
from selenium.webdriver.support.ui import WebDriverWait
@@ -242,7 +242,6 @@ SELECT generate_series(1, {}) as id1, 'dummy' as id2""".format(
def _check_ondemand_result(self, row_id_to_find):
# scroll to bottom to bring last row of next chunk in viewport.
- # canvas_ele = self.page.find_by_css_selector()
scroll = 10
while scroll:
canvas_ele = self.page.find_by_css_selector('.grid-canvas')
@@ -252,8 +251,9 @@ SELECT generate_series(1, {}) as id1, 'dummy' as id2""".format(
".scrollTop(pgAdmin.SqlEditor.jquery('.grid-canvas')"
".height());"
)
- import time
- time.sleep(0.5)
+ # Table height takes some time to update, for which their is no
+ # particular way
+ time.sleep(1)
if canvas_ele.size['height'] == scrolling_height:
break
else:
diff --git a/web/pgadmin/feature_tests/xss_checks_roles_control_test.py b/web/pgadmin/feature_tests/xss_checks_roles_control_test.py
index 6673390a2..6dc7ef7da 100644
--- a/web/pgadmin/feature_tests/xss_checks_roles_control_test.py
+++ b/web/pgadmin/feature_tests/xss_checks_roles_control_test.py
@@ -89,16 +89,8 @@ class CheckRoleMembershipControlFeatureTest(BaseFeatureTest):
def click_membership_tab(self):
"""This will click and open membership tab of role"""
- success = False
- attempts = 3
- while not success and attempts > 0:
- membership_tab_link = self.page.find_by_xpath(
- "//a[normalize-space(text())='Membership']")
- membership_tab_link.click()
- try:
- self.page.find_by_xpath("//input[@placeholder="
- "'Select members']")
- break
- except Exception as e:
- attempts -= 1
- pass
+
+ self.page.retry_click(
+ (By.XPATH,
+ "//a[normalize-space(text())='Membership']"),
+ (By.XPATH, "//input[@placeholder='Select members']"))
diff --git a/web/regression/feature_utils/pgadmin_page.py b/web/regression/feature_utils/pgadmin_page.py
index b5b8fb5d7..7bc0b0b8a 100644
--- a/web/regression/feature_utils/pgadmin_page.py
+++ b/web/regression/feature_utils/pgadmin_page.py
@@ -175,6 +175,38 @@ class PgadminPage:
execute_button.click()
self.wait_for_query_tool_loading_indicator_to_disappear()
+ def check_execute_option(self, option):
+ """"This function will check auto commit or auto roll back based on
+ user input. If button is already checked, no action will be taken"""
+ if option == 'auto_commit':
+ check_status = self.driver.find_element_by_css_selector(
+ QueryToolLocators.btn_auto_commit_check_status)
+ if 'visibility-hidden' in check_status.get_attribute('class'):
+ self.find_by_css_selector(QueryToolLocators.
+ btn_auto_commit).click()
+ if option == 'auto_rollback':
+ check_status = self.driver.find_element_by_css_selector(
+ QueryToolLocators.btn_auto_rollback_check_status)
+ if 'visibility-hidden' in check_status.get_attribute('class'):
+ self.find_by_css_selector(QueryToolLocators.
+ btn_auto_rollback).click()
+
+ def uncheck_execute_option(self, option):
+ """"This function will uncheck auto commit or auto roll back based on
+ user input. If button is already unchecked, no action will be taken"""
+ if option == 'auto_commit':
+ check_status = self.driver.find_element_by_css_selector(
+ QueryToolLocators.btn_auto_commit_check_status)
+ if 'visibility-hidden' not in check_status.get_attribute('class'):
+ self.find_by_css_selector(QueryToolLocators.
+ btn_auto_commit).click()
+ if option == 'auto_rollback':
+ check_status = self.driver.find_element_by_css_selector(
+ QueryToolLocators.btn_auto_rollback_check_status)
+ if 'visibility-hidden' not in check_status.get_attribute('class'):
+ self.find_by_css_selector(QueryToolLocators.
+ btn_auto_rollback).click()
+
def close_data_grid(self):
self.driver.switch_to_default_content()
xpath = "//*[@id='dockerContainer']/div/div[3]/div/div[2]/div[1]"
@@ -712,24 +744,17 @@ class PgadminPage:
status_changed_successfully = True
return status_changed_successfully
- def retry_click_operation(self, element_to_click,
- element_to_verify_after_click):
- """This will attempt to click add button multiple time,
- some different exception encountered while clicking, so handled
- through this"""
-
+ def retry_click(self, click_locator, verify_locator):
click_status = False
attempt = 0
while click_status is not True and attempt < 5:
try:
- if element_to_verify_after_click.is_displayed():
- click_status = True
- element_to_click.click()
- if element_to_verify_after_click.is_displayed():
- click_status = True
+ element = self.driver.find_element(*click_locator)
+ element.click()
+ WebDriverWait(self.driver, 5).until(
+ EC.visibility_of_element_located(verify_locator))
+ click_status = True
except Exception:
- print("The click operation is not performed for "
- "attempt %s, will try 5 attempts" % attempt)
- attempt = +1
+ attempt = +1
return click_status