prevent duplicate ip table rules in SSVM #8530

DaanHoogland · 2024-01-18T12:03:40Z

Description

This PR...

Fixes: #8061

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

might become more than minor on long running ssvms

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

codecov · 2024-01-18T14:00:13Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 13.17%. Comparing base (0b080cf) to head (0d50cba).

Additional details and impacted files

@@            Coverage Diff            @@
##               4.18    #8530   +/-   ##
=========================================
  Coverage     13.17%   13.17%           
  Complexity     9204     9204           
=========================================
  Files          2724     2724           
  Lines        258137   258137           
  Branches      40235    40235           
=========================================
  Hits          33998    33998           
  Misses       219830   219830           
  Partials       4309     4309

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

...server/src/main/java/org/apache/cloudstack/storage/resource/NfsSecondaryStorageResource.java

weizhouapache · 2024-02-21T17:59:44Z

@DaanHoogland
this looks not work

...server/src/main/java/org/apache/cloudstack/storage/resource/NfsSecondaryStorageResource.java

blueorangutan · 2024-06-11T08:08:38Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9860

...dary-storage/server/src/main/java/org/apache/cloudstack/storage/resource/IpTablesHelper.java

...storage/server/src/main/java/org/apache/cloudstack/storage/template/DownloadManagerImpl.java

utils/src/main/java/com/cloud/utils/script/Script.java

weizhouapache · 2024-06-12T07:27:45Z

@blueorangutan package

blueorangutan · 2024-06-12T07:30:03Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-06-12T08:32:40Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9881

weizhouapache · 2024-06-12T09:08:48Z

@blueorangutan test rocky8 kvm-rocky8

blueorangutan · 2024-06-12T09:10:04Z

@weizhouapache a [SL] Trillian-Jenkins test job (rocky8 mgmt + kvm-rocky8) has been kicked to run smoke tests

weizhouapache

overall lgtm

left minor comments

...storage/server/src/main/java/org/apache/cloudstack/storage/template/DownloadManagerImpl.java

weizhouapache · 2024-06-12T09:11:17Z

utils/src/main/java/com/cloud/utils/script/Script.java

@@ -243,12 +243,19 @@ public String execute(OutputInterpreter interpreter) {
                        //process completed successfully
                        if (_process.exitValue() == 0) {
                            _logger.debug("Execution is successful.");
+                            String result;


I guess these are for debugging, right ? @DaanHoogland

yes, should I remove those? I think they could serve purpose for other work as well.

...storage/server/src/main/java/org/apache/cloudstack/storage/template/DownloadManagerImpl.java

DaanHoogland · 2024-06-12T12:31:58Z

utils/src/main/java/com/cloud/utils/script/Script.java

@@ -243,12 +243,19 @@ public String execute(OutputInterpreter interpreter) {
                        //process completed successfully
                        if (_process.exitValue() == 0) {
                            _logger.debug("Execution is successful.");
+                            String result;


yes, should I remove those? I think they could serve purpose for other work as well.

...storage/server/src/main/java/org/apache/cloudstack/storage/template/DownloadManagerImpl.java

...dary-storage/server/src/main/java/org/apache/cloudstack/storage/resource/IpTablesHelper.java

…udstack/storage/template/DownloadManagerImpl.java Co-authored-by: Wei Zhou <[email protected]>

vladimirpetrov · 2024-06-12T14:37:00Z

@blueorangutan package

blueorangutan · 2024-06-12T14:38:04Z

@vladimirpetrov a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-06-12T15:44:56Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9902

blueorangutan · 2024-06-12T22:14:50Z

[SF] Trillian test result (tid-10417)
Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8
Total time taken: 44990 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8530-t10417-kvm-rocky8.zip
Smoke tests completed. 109 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_migrate_VM_and_root_volume	`Error`	88.05	test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks	`Error`	55.20	test_vm_life_cycle.py
test_08_migrate_vm	`Error`	0.07	test_vm_life_cycle.py

nvazquez

Code LGTM

DaanHoogland · 2024-06-13T05:57:50Z

[SF] Trillian test result (tid-10417) Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8 Total time taken: 44990 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8530-t10417-kvm-rocky8.zip Smoke tests completed. 109 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:
Test Result Time (s) Test File
test_01_migrate_VM_and_root_volume Error 88.05 test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks Error 55.20 test_vm_life_cycle.py
test_08_migrate_vm Error 0.07 test_vm_life_cycle.py

The failures are due to a host being in connecting state, which has not to do with the SSVM, so I consider them unrelated.

weizhouapache · 2024-06-13T07:01:58Z

The failures are due to a host being in connecting state, which has not to do with the SSVM, so I consider them unrelated.

a side-note, I have faced the issue frequently, a random node (among 3) is stuck at Connecting state. It comes normal after 3600 seconds.
the error log

2024-06-10T09:30:54,046 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentConnectTaskPool-35:[ctx-d47a6aa2]) (logid:8d018ce9) Failed to handle host connection: com.cloud.exception.ConnectionException: Unable to acquire lock on host 9c8f74c5-17aa-46f3-9b65-b905520b2879
        at com.cloud.agent.manager.AgentManagerImpl.sendReadyAndGetAttache(AgentManagerImpl.java:1127)
        at com.cloud.agent.manager.AgentManagerImpl.handleConnectedAgent(AgentManagerImpl.java:1140)

I think it deserves an investigation. cc @vishesh92 @DaanHoogland

vladimirpetrov

LGTM based on manual testing, using steps as described in the issue.

Co-authored-by: Wei Zhou <[email protected]>

boring-cyborg bot added component:networking component:secondary-storage component:ssvm component:virtual-router Python Warning... Python code Ahead! labels Jan 18, 2024

DaanHoogland changed the title ~~Ghi8061 duplicate ip table rules~~ prevent duplicate ip table rules in SSVM Jan 18, 2024

DaanHoogland force-pushed the ghi8061-duplicateIpTableRules branch from 057ab32 to ca2f543 Compare January 18, 2024 12:47

DaanHoogland linked an issue Jan 18, 2024 that may be closed by this pull request

Duplicated iptables rules in Secondary Storage VM #8061

Closed

weizhouapache reviewed Jan 18, 2024

View reviewed changes

...server/src/main/java/org/apache/cloudstack/storage/resource/NfsSecondaryStorageResource.java Outdated Show resolved Hide resolved

apache deleted a comment from blueorangutan Jan 19, 2024

weizhouapache reviewed Jan 19, 2024

View reviewed changes

...server/src/main/java/org/apache/cloudstack/storage/resource/NfsSecondaryStorageResource.java Outdated Show resolved Hide resolved

apache deleted a comment from blueorangutan Feb 16, 2024

DaanHoogland marked this pull request as ready for review February 16, 2024 14:34

apache deleted a comment from ustcweizhou Feb 19, 2024

DaanHoogland marked this pull request as draft February 23, 2024 08:24

DaanHoogland commented Mar 4, 2024

View reviewed changes

...server/src/main/java/org/apache/cloudstack/storage/resource/NfsSecondaryStorageResource.java Outdated Show resolved Hide resolved

DaanHoogland force-pushed the ghi8061-duplicateIpTableRules branch from 5dd121c to 0d50cba Compare March 5, 2024 11:59

apache deleted a comment from blueorangutan Mar 5, 2024

DaanHoogland added the status:needs-testing label Jun 11, 2024

weizhouapache reviewed Jun 11, 2024

View reviewed changes

review comments addressed

f22a0a8

weizhouapache approved these changes Jun 12, 2024

View reviewed changes

DaanHoogland commented Jun 12, 2024

View reviewed changes

Apply suggestions from code review

1f3ef3b

weizhouapache reviewed Jun 12, 2024

View reviewed changes

...storage/server/src/main/java/org/apache/cloudstack/storage/template/DownloadManagerImpl.java Outdated Show resolved Hide resolved

nvazquez reviewed Jun 12, 2024

View reviewed changes

...dary-storage/server/src/main/java/org/apache/cloudstack/storage/resource/IpTablesHelper.java Outdated Show resolved Hide resolved

Update services/secondary-storage/server/src/main/java/org/apache/clo…

183fe99

…udstack/storage/template/DownloadManagerImpl.java Co-authored-by: Wei Zhou <[email protected]>

vladimirpetrov self-assigned this Jun 12, 2024

more generic errMsg passing

24cb34c

nvazquez approved these changes Jun 13, 2024

View reviewed changes

weizhouapache mentioned this pull request Jun 13, 2024

Host is stuck in Connecting state after update to 4.18.2.0 #9250

Open

vladimirpetrov approved these changes Jun 13, 2024

View reviewed changes

vladimirpetrov removed the status:needs-testing label Jun 13, 2024

DaanHoogland merged commit 56b69b1 into apache:4.18 Jun 14, 2024
27 checks passed

DaanHoogland deleted the ghi8061-duplicateIpTableRules branch June 14, 2024 07:01

dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Jun 20, 2024

prevent duplicate ip table rules in SSVM (apache#8530)

514d1ec

Co-authored-by: Wei Zhou <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

prevent duplicate ip table rules in SSVM #8530

prevent duplicate ip table rules in SSVM #8530

DaanHoogland commented Jan 18, 2024

codecov bot commented Jan 18, 2024 •

edited

Loading

weizhouapache commented Feb 21, 2024

blueorangutan commented Jun 11, 2024

weizhouapache commented Jun 12, 2024

blueorangutan commented Jun 12, 2024

blueorangutan commented Jun 12, 2024

weizhouapache commented Jun 12, 2024

blueorangutan commented Jun 12, 2024

weizhouapache left a comment

weizhouapache Jun 12, 2024

DaanHoogland Jun 12, 2024

DaanHoogland Jun 12, 2024

vladimirpetrov commented Jun 12, 2024

blueorangutan commented Jun 12, 2024

blueorangutan commented Jun 12, 2024

blueorangutan commented Jun 12, 2024

nvazquez left a comment

DaanHoogland commented Jun 13, 2024

weizhouapache commented Jun 13, 2024

vladimirpetrov left a comment

prevent duplicate ip table rules in SSVM #8530

prevent duplicate ip table rules in SSVM #8530

Conversation

DaanHoogland commented Jan 18, 2024

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

codecov bot commented Jan 18, 2024 • edited Loading

Codecov Report

weizhouapache commented Feb 21, 2024

blueorangutan commented Jun 11, 2024

weizhouapache commented Jun 12, 2024

blueorangutan commented Jun 12, 2024

blueorangutan commented Jun 12, 2024

weizhouapache commented Jun 12, 2024

blueorangutan commented Jun 12, 2024

weizhouapache left a comment

Choose a reason for hiding this comment

weizhouapache Jun 12, 2024

Choose a reason for hiding this comment

DaanHoogland Jun 12, 2024

Choose a reason for hiding this comment

DaanHoogland Jun 12, 2024

Choose a reason for hiding this comment

vladimirpetrov commented Jun 12, 2024

blueorangutan commented Jun 12, 2024

blueorangutan commented Jun 12, 2024

blueorangutan commented Jun 12, 2024

nvazquez left a comment

Choose a reason for hiding this comment

DaanHoogland commented Jun 13, 2024

weizhouapache commented Jun 13, 2024

vladimirpetrov left a comment

Choose a reason for hiding this comment

codecov bot commented Jan 18, 2024 •

edited

Loading