Update tags for rhods 1.24 (#729)

* In Model Serving, wait until button "Create data science project" is available This is to try to avoid errors when cluster is slow, like the one in ODS-1921 at https://opendatascience-jenkins-csb-rhods.apps.ocp-c1.prod.psi.redhat.com/job/rhods-sanity/1155/TestReport/ Signed-off-by: Jorge Garcia Oncins <[email protected]> * Add AutomationBug and ProductBug tags based on recent test runs results Signed-off-by: Jorge Garcia Oncins <[email protected]> * Delete JupyterHub test "Verify Telemetry Data Is Accessible" Test was skipped since RHODS 1.16 but RHOSi Setup and Teardown was being executed and failing Signed-off-by: Jorge Garcia Oncins <[email protected]> * Wait until metrics are available to avoid test flakiness Signed-off-by: Jorge Garcia Oncins <[email protected]> --------- Signed-off-by: Jorge Garcia Oncins <[email protected]>
red-hat-data-services · Mar 27, 2023 · c8f829a · c8f829a
1 parent 381247a
commit c8f829a
Show file tree

Hide file tree

Showing 7 changed files with 12 additions and 90 deletions.
diff --git a/ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDataScienceProject/Projects.resource b/ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDataScienceProject/Projects.resource
@@ -84,6 +84,7 @@ Project's Owner Should Be
 Create Data Science Project
     [Documentation]    Creates a DS Project with the given ${title}, ${description} and ${resource_name}
     [Arguments]    ${title}    ${description}    ${resource_name}=${NONE}
+    Wait Until Page Contains Element    ${PROJECT_CREATE_BTN_XP}
     Click Button    ${PROJECT_CREATE_BTN_XP}
     Wait Until Page Contains Element    ${TITLE_INPUT_XP}
     Run Keyword And Warn On Failure     Element Should Be Disabled    ${GENERIC_CREATE_BTN_XP}
@@ -258,4 +259,4 @@ Launch Data Science Project Main Page
     ...             ${ocp_user_auth_type}=${TEST_USER_3.AUTH_TYPE}
     Launch Dashboard    ocp_user_name=${username}  ocp_user_pw=${password}  ocp_user_auth_type=${ocp_user_auth_type}
     ...                 dashboard_url=${ODH_DASHBOARD_URL}    browser=${BROWSER.NAME}   browser_options=${BROWSER.OPTIONS}
-    Open Data Science Projects Home Page
+    Open Data Science Projects Home Page
diff --git a/...ts/Tests/100__deploy/100__installation/103__post_install_monitoring_port_forwarding.robot b/...ts/Tests/100__deploy/100__installation/103__post_install_monitoring_port_forwarding.robot
@@ -31,6 +31,7 @@ Verify MT-SRE Can Connect To Prometheus Using Port-Forwarding
     [Documentation]    Verifies that MT-SRE can connect to Prometheus using Port-forwarding
     [Tags]    Tier2
     ...       ODS-752
+    ...       AutomationBug
     ${prometheus_port_forwarding_process} =  Enable Access To Prometheus Using OpenShift Port Forwarding
     Wait Until HTTP Status Code Is  url=http://localhost:9090  retry=10x  expected_status_code=200
     Go To  http://localhost:9090
@@ -41,6 +42,7 @@ Verify MT-SRE Can Connect To Alert Manager Using Port-forwarding
     [Documentation]    Verifies that MT-SRE can connect to Alert Manager using Port Forwarding
     [Tags]    Tier2
     ...       ODS-753
+    ...       AutomationBug
     ${alertmanager_port_forwarding_process} =  Enable Access To Alert Manager Using OpenShift Port Forwarding
     Wait Until HTTP Status Code Is  url=http://localhost:9093  retry=10x  expected_status_code=200
     Go To  http://localhost:9093

diff --git a/ods_ci/tests/Tests/200__monitor_and_manage/200__metrics/201__billing_metrics.robot b/ods_ci/tests/Tests/200__monitor_and_manage/200__metrics/201__billing_metrics.robot
@@ -77,6 +77,8 @@ Test Metric "Active_Users" On OpenShift Monitoring On Cluster Monitoring Prometh
     Log In N Users To JupyterLab And Launch A Notebook For Each Of Them
     ...    list_of_usernames=${list_of_usernames}
 
+    Sleep    60s    reason=Wait until metrics are available
+
     ${active_users_after} =    Run OpenShift Metrics Query
     ...    query=cluster:usage:consumption:rhods:active_users
     ...    retry_attempts=1    return_zero_if_result_empty=True
@@ -96,6 +98,7 @@ Test Metric "Active Notebook Pod Time" On OpenShift Monitoring - Cluster Monitor
     @{list_of_usernames} =    Create List    ${TEST_USER_3.USERNAME}    ${TEST_USER_4.USERNAME}
     Log In N Users To JupyterLab And Launch A Notebook For Each Of Them
     ...    list_of_usernames=${list_of_usernames}
+    Sleep    60s    reason=Wait until metrics are available
     ${value} =    Run OpenShift Metrics Query    query=cluster:usage:consumption:rhods:pod:up
     Should Not Be Empty    ${value}    msg=Metrics does not contains value for pod:up query
     [Teardown]    CleanUp JupyterHub For N Users    list_of_usernames=${list_of_usernames}

diff --git a/ods_ci/tests/Tests/400__ods_dashboard/400__ods_dashboard.robot b/ods_ci/tests/Tests/400__ods_dashboard/400__ods_dashboard.robot
@@ -56,7 +56,7 @@ Verify Content In RHODS Explore Section
     ...    - Card's details (text, badges, images)
     ...    - Sidebar (titles, links text, links status)
     [Tags]    Sanity
-    ...       ODS-488    ODS-993    ODS-749    ODS-352    ODS-282
+    ...       ODS-488    ODS-993    ODS-749    ODS-352    ODS-282    ProductBug
     ${EXP_DATA_DICT}=    Load Expected Data Of RHODS Explore Section
     Click Link    Explore
     Wait For RHODS Dashboard To Load    expected_page=Explore

diff --git a/ods_ci/tests/Tests/400__ods_dashboard/412__ods_dashboard_resources.robot b/ods_ci/tests/Tests/400__ods_dashboard/412__ods_dashboard_resources.robot
@@ -41,7 +41,7 @@ Verify Resource Link HTTP Status Code
     [Documentation]    Verifies the how-to, documentation and tutorial cards in Resource page
     ...                redirects users to working URLs (i.e., http status must be 200)
     [Tags]    Sanity
-    ...       ODS-531    ODS-507
+    ...       ODS-531    ODS-507    ProductBug
     Click Link    Resources
     Sleep    5
     ${link_elements}=     Get Link Web Elements From Resource Page

diff --git a/ods_ci/tests/Tests/500__jupyterhub/test-jupyterhub-api-telemetry.robot b/ods_ci/tests/Tests/500__jupyterhub/test-jupyterhub-api-telemetry.robot
diff --git a/ods_ci/tests/Tests/900__high_availability/900__openshift/900__openshift.robot b/ods_ci/tests/Tests/900__high_availability/900__openshift/900__openshift.robot
@@ -17,6 +17,7 @@ Verify ODS Availability After OpenShift Node Failure
     ...       Tier3
     ...       ODS-568
     ...       Execution-Time-Over-5min
+    ...       AutomationBug
     @{cluster_nodes_info}=    Fetch Cluster Worker Nodes Info
     &{cluster_node_info_dict}=    Set Variable    ${cluster_nodes_info}[0]
     Force Reboot OpenShift Cluster Node    ${cluster_node_info_dict.metadata.name}
@@ -26,10 +27,12 @@ Verify rhods_aggregated_availability Detects Downtime In Jupyterhub And Dashboar
     [Documentation]    Verifies if rhods_aggregated_availability detects downtime in Jupyterhub and Dashboard.
     [Tags]    ODS-1608
     ...       Tier3
+    ...       AutomationBug
     Verify rhods_aggregated_availability Detects Downtime In Component    jupyterhub
     Verify rhods_aggregated_availability Detects Downtime In Component    rhods-dashboard
     Verify rhods_aggregated_availability Detects Downtime In Component    combined
 
+
 *** Keywords ***
 Verify rhods_aggregated_availability Detects Downtime In Component
     [Documentation]  Verifies if rhods_aggregated_availability detects downtime