Commit Graph

806 Commits (63fe9f1f1f47c944d800df9f20c0693dc51c08a8)

Author SHA1 Message Date
qiuming ea04a86eb2
Merge pull request #6771 from qiuming-best/bsl-fix
Fix default BSL setting not work
2023-12-05 19:09:50 +08:00
Xun Jiang/Bruce Jiang 6093e651cb
Merge pull request #7161 from Lyndon-Li/node-agent-config-doc
Add node-agent concurrency doc
2023-12-05 16:52:29 +08:00
Lyndon-Li ac5d030ab4 Merge branch 'main' into issue-fix-6695 2023-12-05 16:46:31 +08:00
Lyndon-Li 434e073c67 csi data mover backup describe, support legacy backups
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-12-05 15:49:35 +08:00
Xun Jiang c8e76f4602 Fix the DownloadRequest context error.
Clean the DownloadRequest Stream function.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-12-05 13:29:23 +08:00
Xun Jiang 28df14d9d5 Modify restore logic.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-12-05 10:01:16 +08:00
Xun Jiang c77bec73bb Move VolumesInformation to an independant package.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-12-04 08:33:37 +08:00
Ming Qiu c6cba300fb Fix default BSL setting not work
Signed-off-by: Ming Qiu <mqiu@vmware.com>
2023-12-01 02:06:35 +00:00
Ming Qiu 0afaa70e9b Merge branch 'main' of https://github.com/qiuming-best/velero into support-restore-sparse 2023-11-30 10:55:55 +00:00
Lyndon-Li 81183f683e Merge branch 'main' into issue-fix-6695 2023-11-29 15:12:21 +08:00
Lyndon-Li 33b570d5cd Merge branch 'main' into node-agent-config-doc 2023-11-29 14:45:20 +08:00
Lyndon-Li 8968ae5ec4 add node-agent concurrency doc
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-11-29 14:33:51 +08:00
Xun Jiang f5c159ce56 Resolve linter issues.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-11-29 11:15:43 +08:00
Xun Jiang ec03d1ebce Add noctx linter.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-11-29 11:13:46 +08:00
Xun Jiang cddc11e000 Enable linter errchkjson.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-11-29 11:13:46 +08:00
Ming Qiu b57dde1572 Allow sparse option for Kopia & Restic restore
Signed-off-by: Ming Qiu <mqiu@vmware.com>
2023-11-28 13:48:09 +00:00
Daniel Jiang 85482aefaf
Merge pull request #7117 from allenxu404/issue6567
Add hook status to backup/restore CR
2023-11-28 16:54:11 +08:00
allenxu404 5d1a632be4 Add hook status to backup/restore CR
Signed-off-by: allenxu404 <qix2@vmware.com>
2023-11-28 14:47:31 +08:00
Wenkai Yin(尹文开) 6ac7ff1230
Merge pull request #7130 from qiuming-best/data-mover-recoverbility
Node agent restart enhancement
2023-11-28 14:25:47 +08:00
Ming Qiu 98a56eb5c7 Node agent restart enhancement
Signed-off-by: Ming Qiu <mqiu@vmware.com>
2023-11-28 05:50:46 +00:00
Xun Jiang/Bruce Jiang d336e2812e
Merge pull request #6958 from blackpiglet/5156_list_option_fix
Change controller-runtime List option from MatchingFields to ListOpti…
2023-11-27 17:38:12 +08:00
Lyndon-Li 8ab0c017a9 issue 6695: add backup description for data mover
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-11-27 16:19:34 +08:00
qiuming ccd3f220ad
Merge pull request #7090 from qiuming-best/perf-test-0
Enhance perf test
2023-11-27 16:10:26 +08:00
Ming 507157f812 Add perf test namespace mapping when restore
Signed-off-by: Ming <mqiu@vmware.com>
2023-11-27 02:11:13 +00:00
Lyndon-Li 1815c1691f Merge branch 'main' into issue-fix-6695 2023-11-27 09:46:22 +08:00
Ming Qiu c2d4495efe Merge branch 'main' of https://github.com/qiuming-best/velero into kopia-parallelism 2023-11-22 03:52:20 +00:00
Wenkai Yin(尹文开) 5c958d820d
Merge pull request #7100 from blackpiglet/6595_volumeinfo_generate
6595 volumeinfo generate
2023-11-22 11:14:36 +08:00
Ming Qiu fea22bbbc9 Merge branch 'main' of https://github.com/qiuming-best/velero into kopia-parallelism 2023-11-22 01:42:39 +00:00
Xun Jiang 7f52321772 Generate VolumeInfo.
Remove CSI VolumeSnapshot listter and the informer.
Add download the VolumeInfos metadata for backup.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-11-22 09:40:38 +08:00
qiuming b8a5859fe7
Merge pull request #7091 from anshulahuja98/recoverplugin
Don't fail backup/restore on velero server restart in PhaseWaitingFor…
2023-11-20 14:49:15 +08:00
Lyndon-Li 4e4f0aa1da issue 6695: add backup describe for CSI snapshot data movement 02
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-11-20 12:11:21 +08:00
Lyndon-Li b99ac448ae issue 6695: add backup describe for CSI snapshot data movement
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-11-18 00:11:29 +08:00
yanggang c78e8980d8
Use camel case for API objects: configmaps and secrets.
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-11-16 22:17:35 +00:00
Wenkai Yin(尹文开) 2a533d01bf
Merge pull request #7046 from kaovilai/backup-patch-status-unittest
Update Backup.Status.CSIVolumeSnapshotsCompleted during finalize
2023-11-15 15:32:51 +08:00
Anshul Ahuja dd6ab8c32a Don't fail backup/restore on velero server restart in PhaseWaitingForPluginOperation
Signed-off-by: Anshul Ahuja <anshulahuja@microsoft.com>
2023-11-13 11:13:32 +05:30
Lyndon-Li efc5319c1c Issue 6693: partially fail restore if CSI snapshot is involved but CSI feature is not ready
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-11-10 12:40:41 +08:00
Lyndon-Li 2841be7681 Merge branch 'main' into issue-fix-6693 2023-11-10 10:04:27 +08:00
Lyndon-Li 873197ff50 issue 6693: partially fail restore if CSI snapshot is involved but CSI feature is not ready
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-11-09 17:37:23 +08:00
Lyndon-Li db43200cc8 configurable data path concurrency: all in one json
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-11-08 12:02:02 +08:00
Lyndon-Li 68579448d6 configurable data path concurrency: UT
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-11-06 20:29:33 +08:00
Lyndon-Li 262f10ff49 Merge branch 'main' into issue-fix-6663 2023-11-06 16:52:41 +08:00
Lyndon-Li 04a9851ee9 configurable data path concurrency: all in cm
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-11-06 16:46:13 +08:00
Xun Jiang 56b5e982d9 Remove dependency of generated client part 3
Replace generated discovery client with client-go client.
Remove generated client from PVR action.
Remove generated client from pkg/cmd directory.
Delete velero generate client from client factory.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-11-06 11:34:39 +08:00
Xun Jiang a221a88945 Remove dependency of generated client part 2
Remove dependecy of generate client from pkg/cmd/cli/snapshotLocation.
Remove the Velero generated informer from PVB and PVR. 
Remove dependency of generated client from pkg/podvolume directory.
Replace generated codec with runtime codec. 

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-11-03 17:11:36 +08:00
Tiger Kaovilai 9311a4269b
refactor backup snapshot status updates into UpdateBackupSnapshotsStatus() and run in backup_finalizer_controller
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-11-02 15:30:35 -04:00
Lyndon-Li 0f765ceef2 Merge branch 'main' into issue-fix-6663 2023-10-27 17:44:17 +08:00
Lyndon-Li c44a9b8956 issue 6663: changes for configurable data path concurrency
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-10-27 17:37:29 +08:00
Ming 481cb60493 Make Kopia file parallelism configurable
Signed-off-by: Ming <mqiu@vmware.com>
2023-10-26 02:28:36 +00:00
Daniel Jiang 941dd0039f
Merge pull request #6968 from blackpiglet/6585_fix
Check whether the action is a CSI action and whether CSI feature is
2023-10-25 00:39:58 +08:00
Xun Jiang 908e2c63ba Check whether the action is a CSI action and whether CSI feature is
enabled, before executing the action.

The DeleteItemAction is not checked, because the DIA doesn't have a
method to get the action's plugin name.
This should be OK, because the CSI will check whether the VS and VSC
have a backup name annotation. If the VS and VSC is not handled by
the CSI plugin, then they don't have the annotation.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-10-24 16:54:38 +08:00
yanggang 1efd533d0d
Fix wrong logs in markDataDownloadsCancel() and add missiong license file.
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-10-19 14:04:41 +01:00
Xun Jiang 79c75718ca Change controller-runtime List option from MatchingFields to ListOptions.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-10-19 17:09:12 +08:00
yanggang 069c280f03
Fix node-agent missing metrics-addr parms to define the server start.
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-10-13 03:33:18 +01:00
Shubham Pampattiwar ad114f8f65
Merge pull request #6723 from sseago/restore-get-perf 2023-10-12 07:57:40 -07:00
Scott Seago 7750e12151 Perf improvements for existing resource restore
Use informer cache with dynamic client for Get calls on restore
When enabled, also make the Get call before create.

Add server and install parameter to allow disabling this feature,
but enable by default

Signed-off-by: Scott Seago <sseago@redhat.com>
2023-10-11 10:51:39 -04:00
Scott Seago 09be1f7995 issue #6807: Retry failed create when using generateName
When creating resources with generateName, apimachinery
does not guarantee uniqueness when it appends the random
suffix to the generateName stub, so if it fails with
already exists error, we need to retry.

Signed-off-by: Scott Seago <sseago@redhat.com>
2023-10-09 17:38:37 -04:00
Yang Gang fd67ecb688
Code clean for backup cmd client. (#6750)
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-09-29 12:23:12 -04:00
David Zaninovic 8e01d1b9be
Add support for block volumes (#6680)
Signed-off-by: David Zaninovic <dzaninovic@catalogicsoftware.com>
2023-09-28 09:44:46 -04:00
Wenkai Yin(尹文开) 563f1ccee1
Merge pull request #6475 from nilesh-akhade/main
Add `--or-selector` for backup and restore command
2023-09-27 20:09:07 +08:00
lyndon b3df028e83
Merge pull request #6815 from AgustinRamiroDiaz/main
Typo: remove double space
2023-09-18 12:06:27 +08:00
Wenkai Yin(尹文开) 185a95585a
Set data mover related properties for schedule (#6824)
Set data mover related properties for schedule

Fixes #6820

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-09-14 18:14:06 +08:00
Agustín Díaz ff45be6fdd Typo: remove double space
Signed-off-by: Agustín Díaz <agustin.ramiro.diaz@gmail.com>
2023-09-13 10:46:28 -03:00
qiuming 8a366c6924
Merge pull request #6798 from yanggangtony/clean-some-code
Fix issue #6781,  and some code clean.
2023-09-12 14:56:27 +08:00
yanggang c97b31363d
Fix some wrong logs and code clean.
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-09-11 13:38:32 +08:00
yanggang 76b6077683
Keep the logs info ns/name is the same with other modules.
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-09-06 18:40:10 +08:00
lyndon 8cb04d4f69
Merge pull request #6751 from Lyndon-Li/issue-fix-6647
Fix issue 6647
2023-09-06 09:03:00 +08:00
Lyndon-Li 778feba3ae fix issue 6647
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-09-04 16:55:36 +08:00
Yang Gang (成都) 25898305ef
delete unused shcema package and parms. (#6716)
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-09-04 14:50:10 +08:00
qiuming 0e5022254f
[Cherry-pick Main] Fix velero uninstall bug (#6729)
Signed-off-by: Ming <mqiu@vmware.com>
2023-08-31 16:15:24 +08:00
Ming 85d5785d68 [Cherry-Pick main] make velero uninstall backward compatible
Signed-off-by: Ming <mqiu@vmware.com>
2023-08-29 01:07:41 +00:00
Ming 7f3b7fe853 Fix delete dataupload datadownload failure when Velero uninstall
Signed-off-by: Ming <mqiu@vmware.com>
2023-08-24 03:30:28 +00:00
Lyndon-Li dac28084a3 issue fix 6662
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-08-17 11:35:18 +08:00
Anshul Ahuja e753a08f85
E2E test for Resource Modifier & Bug fixes (#6483)
* Add resource modifier e2e test and bug fixes

Signed-off-by: Anshul Ahuja <anshulahuja@microsoft.com>
2023-08-16 11:09:59 +08:00
qiuming 411bd54920
Merge pull request #6616 from qiuming-best/add-accept-label
Fix data mover controller bugs
2023-08-15 12:04:31 +08:00
Ming Qiu 5485616abf Fix data mover bugs #6550 #6563 #6600
Signed-off-by: Ming Qiu <mqiu@vmware.com>
2023-08-14 08:37:19 +00:00
allenxu404 7a8a68d9e9 modify output for command of `restore delete`
Signed-off-by: allenxu404 <qix2@vmware.com>
2023-08-09 15:58:31 +08:00
Qi Xu 5f463c59ec
add "Deleting" phase for restore display (#6604)
Signed-off-by: allenxu404 <qix2@vmware.com>
2023-08-08 17:16:53 -04:00
Xun Jiang/Bruce Jiang d027a1641d
Merge pull request #6591 from allenxu404/finalizer-enhance
Enhance uninstall and backup deletion for restore finalizer
2023-08-03 17:23:07 +08:00
allenxu404 846f0de178 enhance uninstall and backup deletion for restore finalizer
Signed-off-by: allenxu404 <qix2@vmware.com>
2023-08-03 16:43:33 +08:00
lyndon bb74c352fb
Merge pull request #6578 from Lyndon-Li/issue-fix-6560
Fix issue 6560
2023-08-03 14:55:09 +08:00
Lyndon-Li 7cf3559fab fix issue 6560
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-08-01 21:37:51 +08:00
Yang Gang 16613f5fe1
Fix the velero crd define locations. (#6554)
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-07-31 15:58:07 +08:00
yanggang 2f20fac78b
Delete redundance checkNodeAgent code
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-07-28 16:16:26 +08:00
Xun Jiang/Bruce Jiang f666667e5b
Merge pull request #6547 from reasonerjt/del-moved-snapshot
Delete moved snapshots when the backup is deleted
2023-07-27 15:10:33 +08:00
Daniel Jiang b0a343cd48 Delete moved snapshots when the backup is deleted
This commit introduces a deleteItemAction which writes a temporary configmap to
record the snapshot info so that the controller can trigger repo manager
to remove the snapshot

This process is a bit chatty and we should consider to refactor the code
so it's easier to connect to the repo directly in the DIA

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-07-27 13:27:33 +08:00
allenxu404 6dbdc54dc7 check if restore crd exist before operating restore
Signed-off-by: allenxu404 <qix2@vmware.com>
2023-07-25 18:54:42 +08:00
lyndon 9c8275eda7
Merge pull request #6533 from Lyndon-Li/data-mover-wrong-bsl-after-sync
Data mover wrong bsl after sync
2023-07-25 14:20:10 +08:00
Lyndon-Li 74bf03b272 data mover wrong bsl after sync
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-07-25 12:38:09 +08:00
Daniel Jiang 4379b9a025
Merge pull request #6532 from yanggangtony/fix-wrong-log-controller
Fix wrong log for server controller logs.
2023-07-24 14:32:48 +08:00
Mateus Oliveira 913b610196
fix: Typos and add more spell checking rules to CI (#6415)
* fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

---------

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
2023-07-21 17:25:24 -04:00
yanggang 970938c89a
Fix wrong log for server controller logs.
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-07-21 16:35:26 +08:00
qiuming 32262babc4
Merge pull request #6479 from allenxu404/i2697
add restore finalizer to clean up external resources
2023-07-20 19:07:02 +08:00
lyndon b4181ef803
restict namespace to node-agent cache (#6523)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-07-19 16:23:28 -04:00
allenxu404 0bb509ccdd add restore finalizer to clean up external resources
Signed-off-by: allenxu404 <qix2@vmware.com>
2023-07-19 18:56:35 +08:00
Nilesh Akhade d9a7e2b6ca Add 'orLabelSelector' for backup, restore command
Signed-off-by: Nilesh Akhade <nakhade@catalogicsoftware.com>
2023-07-19 16:16:35 +05:30
Anshul Ahuja c8f970a4f1
Merge branch 'main' into resourcemodifier
Signed-off-by: Anshul Ahuja <anshul.ahu@gmail.com>
2023-07-19 11:11:40 +05:30
Xun Jiang 4a222b76c6 Improve backup log command UT coverage.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-07-18 14:41:55 +08:00
Xun Jiang 9e515ac397 Remove dependency of the legacy client code from pkg/cmd directory part 2.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-07-17 17:04:53 +08:00
lyndon dfd7970219
Merge pull request #6469 from blackpiglet/6190_fix
Remove dependency of the legacy client code from pkg/cmd directory part 1
2023-07-14 17:15:56 +08:00
Xun Jiang 89d3ad4864 Remove dependency of the legacy client code from pkg/cmd directory.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-07-14 15:55:09 +08:00