Skip to content

Commit 46ad428

Browse files
committed
Adding migration-tools directory for tts-backup-python
Adding migration-tools directory for tts-backup-python
1 parent 796a1e6 commit 46ad428

File tree

4 files changed

+2959
-0
lines changed

4 files changed

+2959
-0
lines changed
Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
# Transportable Tablespaces Using Backup
2+
3+
Transportable Tablespaces can be used to move tablespaces from customer
4+
on-premise or another database cloud service into ADB-S. Tablespaces can
5+
be transported when creating a new database in ADB-S or as modify
6+
operation on an existing database.
7+
8+
## Step-by-step guide
9+
10+
Transporting Tablespaces involves the following steps.
11+
12+
1. Create Object Storage Buckets
13+
14+
2. Create Dynamic Group and Policy
15+
16+
3. Backup Tablespaces on Source Database
17+
18+
4. Create or Modify database in ADB-S by specifying intent to transport
19+
tablespaces using a tag
20+
21+
### Create Object Storage Buckets
22+
23+
Transportable Tablespaces needs two object storage buckets - one for
24+
backups and another for metadata. Create the buckets in your Oracle
25+
Storage Cloud Service account. Note the URLs of the buckets as they are
26+
needed as inputs for the operation. Use [Oracle Cloud Infrastructure Object Storage Native URI Format](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/file-uri-formats.html) for the URL.
27+
28+
Example:
29+
[https://objectstorage.region.oraclecloud.com/n/\<namespace-string\>/b/\<bucket-name\>](https://objectstorage.region.oraclecloud.com/n/namespace-string/b/bucket/o/filename)
30+
31+
To make Object Storage URI work, please [generate an API signing key](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/apisigningkey.htm)].
32+
Download the private key .pem file and the API signing key config file with below contents to source database host.
33+
34+
[DEFAULT]
35+
36+
user=ocid1.user.oc1..xxxxx
37+
38+
fingerprint=f6:d6:e5:xxxxx
39+
40+
tenancy=ocid1.tenancy.oc1..xxxxx
41+
42+
region=us-ashburn-1
43+
44+
key_file=<path to the downloaded private key file>
45+
46+
Note - User should have read and write access to the object storage buckets.
47+
48+
## Create Dynamic Group and Policy
49+
50+
Transportable Tablespaces functionality will download metadata from metadata bucket using [OCI Resource Principal](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/resource-principal.html)].
51+
52+
### Create Dynamic Group and Policy to allow access to the metadata bucket using the resource principal.
53+
54+
1. Create a Dynamic Group **TTSDynamicGroup** with matching rule:\
55+
ALL {resource.type = \'autonomousdatabase\'[resource.compartment.id](http://resource.compartment.id) = \'your_Compartment_OCID\'}
56+
57+
2. Create a Policy using the dynamic group with Policy Statement:\
58+
Allow dynamic-group **TTSDynamicGroup** to manage buckets in tenancy\
59+
Allow dynamic-group **TTSDynamicGroup** to manage objects in tenancy\
60+
\
61+
Prepend domain name to the dynamic group name if needed as below.\
62+
Allow dynamic-group \<*identity_domain_name\>*/**TTSDynamicGroup** to manage buckets in tenancy\
63+
Allow dynamic-group \<*identity_domain_name\>*/**TTSDynamicGroup** to manage objects in tenancy
64+
65+
### Backup Tablespaces on Source Database
66+
67+
#### Pre-requisites
68+
69+
- Create a Project Directory that will used as staging location on the host running the source database.
70+
- Download [Oracle Database Backup Cloud Module](https://www.oracle.com/database/technologies/oracle-cloud-backup-downloads.html) to the Project Directory. Unzip the downloaded opc_installer.zip in the project directory.
71+
- Download [tts-backup-python.zip](file:////confluence/download/attachments/11465427748/tts-backup-python.zip%3fversion=33&modificationDate=1764740426000&api=v2) to the project directory. Unzip the tts-backup-python.zip in the project directory.
72+
- Provide inputs for backup in the **tts-backup-env.txt** file.
73+
- [tts-backup-python.zip for VodaFone](file:////confluence/download/attachments/11465427748/tts-backup-python.zip%3fversion=33&modificationDate=1764740426000&api=v2)
74+
75+
#### TTS Backup Tool inputs
76+
77+
Open tts-backup-env.txt file downloaded to the project directory and provide the following inputs in the file.
78+
79+
##### Project and Tablespace inputs
80+
81+
***PROJECT_NAME** ***: Name for transport tablespace project. (REQUIRED INPUT) \
82+
***DATABASE_NAME*** : Database Name containing the tablespaces. (REQUIRED INPUT) \
83+
***TABLESPACES*** : List of comma separated transportable tablespaces. (OPTIONAL INPUT) if empty all user tablespaces are added.\
84+
***SCHEMAS*** : List of comma separated transportable schemas. (OPTIONAL INPUT) if empty all required users are added. None of the schemas should be a common user.
85+
86+
##### Database connection inputs
87+
88+
***HOSTNAME*** : (REQUIRED INPUT) Host where database is running, used for connecting to the database.\
89+
***LSNR_PORT*** : (REQUIRED INPUT) Listener port, used for connecting to the database.  \
90+
***DB_SVC_NAME*** : (REQUIRED INPUT) Database service name, used for connecting to the database.  \
91+
***ORAHOME*** : (REQUIRED INPUT) Database home, \$ORACLE_HOME.\
92+
***DBUSER*** : (REQUIRED INPUT) Username for connecting to the database. User should have sysdba privileges. \
93+
***DBPASSWORD*** : (REQUIRED INPUT) Password for connection to the database. (Provide as CLI argument or Runtime input)\
94+
***DB_VERSION*** : (REQUIRED INPUT) DB Version, supported values are 11g, 12c, 19c, 23ai.
95+
96+
##### Object Storage Service (OSS) inputs ( Required if using OSS for transport. Leave them empty if using FSS. )
97+
98+
***TTS_BACKUP_URL*** : (REQUIRED INPUT) Object storage bucket uri for backup files.      \
99+
***TTS_BUNDLE_URL*** : (REQUIRED INPUT) Object storage bucket uri for transportable tablespace bundle.\
100+
***OCI_INSTALLER_PATH :*** (REQUIRED INPUT) Path to oci_install.jar\
101+
***CONFIG_FILE :*** **(REQUIRED INPUT) Path to dowloaded API keys config file. Make sure to update the key_file parameter with the file path to your private key in the config file.\
102+
***COMPARTMENT_OCID :*** **(REQUIRED INPUT) Compartment OCID of bucket  (TTS_BACKUP_URL) that stores backup files. \
103+
***OCI_PROXY_HOST :** ***(OPTIONAL INPUT) HTTP proxy server.\
104+
***OCI_PROXY_HOST :** ***(OPTIONAL INPUT)  HTTP proxy server connection port.
105+
106+
##### File Storage Service (FSS) inputs ( Required if using FSS for transport. Leave them empty if using OSS. )
107+
108+
***TTS_FSS_CONFIG*** : (REQUIRED INPUT) FSS configuration should given in the format FSS:\<FIle System Name\>:\<FQDN of Mount Target\>:\<Export Path\>\
109+
***TTS_FSS_MOUNT_DIR*** : (REQUIRED INPUT) Absolute path where file system was mounted on source database host\
110+
\
111+
Refer to [<https://blogs.oracle.com/datawarehousing/post/attach-file-system-autonomous-database>] for details on how File System should be configured for use by ADB-S.
112+
113+
##### TDE keys inputs
114+
115+
***TDE_WALLET_STORE_PASSWORD*** : (REQUIRED only if any of the tablespaces are TDE Encrypted). Required to export TDE KEYS. (Provide as CLI argument or Runtime input)
116+
117+
##### Final backup inputs 
118+
119+
***FINAL_BACKUP*** : (REQUIRED INPUT) Final backup will be done only if FINAL_BACKUP=yes or YES. Accepted values YES, yes, NO, no. Used to indicate incremental operation. Specify YES for non-incremental operation. Specify NO for incremental backups. Last operation should be run with YES for schema to be exported.
120+
121+
##### Performance inputs     
122+
123+
***PARALLELISM*** : (OPTIONAL INPUT) Number of channels to be used for backup, parallelism = cpu_count \* instances.\
124+
***CPU_COUNT*** : (OPTIONAL INPUT) Number of cpus to be used from an instance (used if parallelism is not given).
125+
126+
Leave these a blank unless really needed.
127+
128+
### Create or Modify ADB-S database with TTS tag
129+
130+
#### Create ADB-S database to transport tablespaces
131+
132+
Use the below steps to transport tablespaces while creating an ADB-S database.
133+
134+
1. Go to OCI Console → Oracle Database → Autonomous Database
135+
2. Click on ***Create Autonomous Database***
136+
3. Provide all the necessary inputs.
137+
4. Select database version that is equal to or greater than the source database.
138+
5. Specify ***Storage (TB)*** in ***Configure the database*** section to match size of tablespace(s) being transported.
139+
6. Click on ***Show advanced options*** at the bottom of the page. Click on ***Tags*** tab in the section.
140+
7. Select ***Tag namespace*** **as ***None (add a free-form tag)***, ***Tag key*** as ***ADB\$TTS_BUNDLE_URL***,
141+
**Tag value** as metadata bundle url given by the TTS backup tool.
142+
8. Click on ***Add Tag.***
143+
9. Submit ***Create Autonomous Database.***
144+
145+
The operation will first create the database and then trigger transport tablespaces job.
146+
147+
#### Modify ADB-S database to transport tablespaces
148+
149+
Use the below steps to transport tablespaces to an existing database.
150+
151+
1. Go to OCI Console → Oracle Database → Autonomous Database
152+
2. Select and click on the database for transporting tablespaces
153+
3. Verify ***Storage*** in ***Resource allocation*** section. Use ***Manage resource allocation*** tab to increase storage if needed.
154+
4. If this is the first time specifying **ADB\$TTS_BUNDLE_URL** tag on the database:
155+
a. Go to ***More Actions* *→* *Tags** ****menu item on the Autonomous Database Details page.*
156+
b. Select ***Tag namespace*** **as ***None (add a free-form tag)***, ***Tag key*** as ***ADB\$TTS_BUNDLE_URL***, \
157+
**Tag value** as metadata bundle url given by the TTS backup tool.
158+
c. Click on ***Add Tag.***
159+
d. Submit ***Add Tags***.
160+
5. If **ADB\$TTS_BUNDLE_URL** tag was already specified during create or a previous update of the database:
161+
a. Click on the **Tags** tab on the Autonomous Database Details page.
162+
b. Click on **Free-Form tags** tab and edit the **ADB\$TTS_BUNDLE_URL** tag. 
163+
c. Specify the new URL and submit **Save** action.
164+
165+
The operation will trigger the transport tablespaces job on the database.
166+
167+
#### To transport tablespaces using incremental database backups
168+
169+
Create a database or update an existing database by specifying **ADB\$TTS_BUNDLE_URL** of level 0 backup.\
170+
For each incremental and final backup, edit the tag using the URL corresponding to that backup as mentioned in **Step 5** above. \
171+
Before taking the final backup, alter all tablespaces being transported as read-only. Specify FINAL_BACKUP as YES in tts-backup-env.txt.\
172+
Datafiles with incremental changes are restored to ADB-S from level 0 to final step. Metadata is imported on final step.
173+
174+
#### To transport tablespaces using non-incremental database backups
175+
176+
Non-incremental is a one time operation where datafiles are restored and metadata is imported to ADB-S.\
177+
Alter all tabalespaces being transported as read-only. Specify FINAL_BACKUP as YES in tts-backup-env.txt.\
178+
Create a database or update an existing database by specifying **ADB\$TTS_BUNDLE_URL** of one time backup.
179+
Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
#
2+
# $Header: pdbcs/no_ship_src/service/scripts/tts-backup-env.txt /main/15 2025/08/13 07:58:35 hkarniya Exp $
3+
#
4+
# tts-backup-env.txt
5+
#
6+
# Copyright (c) 2024, 2025, Oracle and/or its affiliates.
7+
#
8+
# NAME
9+
# tts-backup-env.txt - <one-line expansion of the name>
10+
#
11+
# DESCRIPTION
12+
# <short description of component this file declares/defines>
13+
#
14+
# NOTES
15+
# <other useful comments, qualifications, etc.>
16+
#
17+
# MODIFIED (MM/DD/YY)
18+
# hkarniya 08/04/25 - Bug 38274082: Option to exclude statistics
19+
# hkarniya 07/11/25 - Bug 38179045: Option to exclude tables
20+
# hkarniya 07/01/25 - Bug 37848678: Provide dry-run option
21+
# hkarniya 06/15/25 - Bug 38076817: support section size backup for tts
22+
# hkarniya 05/20/25 - Bug 37973559: Code refactor
23+
# hkarniya 05/15/25 - Bug 37819695: Take passwords as cli arguments or runtime inputs
24+
# hkarniya 04/23/25 - Bug 37866112: Add to option USE_ALL_RAC_INSTANCES
25+
# cchowdar 04/07/25 - Bug 37803647: Added FSS support
26+
# hkarniya 04/05/25 - Bug 37793929: Migrate to OCI installer
27+
# hkarniya 03/24/25 - Bug 37746561: Validate plsql objects and
28+
# add opc proxy host and port
29+
# hkarniya 03/24/25 - Bug 37746539: Support 11g source database
30+
# hkarniya 02/13/25 - Bug 37588601: support full transport of DB,
31+
# TABLESPACES is optional input
32+
# hkarniya 01/30/25 - Bug 37515587: Update schemas as optional input
33+
# hkarniya 10/03/24 - Creation
34+
#
35+
[DEFAULT]
36+
### Project and Tablespace inputs
37+
PROJECT_NAME=
38+
DATABASE_NAME=
39+
TABLESPACES=
40+
SCHEMAS=
41+
42+
43+
#### Database connection inputs
44+
HOSTNAME=
45+
LSNR_PORT=
46+
DB_SVC_NAME=
47+
ORAHOME=
48+
DBUSER=
49+
DB_VERSION=
50+
51+
52+
#### Object Storage Service (OSS) inputs
53+
#### OCI Installer inputs
54+
TTS_BACKUP_URL=
55+
TTS_BUNDLE_URL=
56+
OCI_INSTALLER_PATH=
57+
CONFIG_FILE=
58+
COMPARTMENT_OCID=
59+
OCI_PROXY_HOST=
60+
OCI_PROXY_PORT=
61+
62+
63+
### File Storage Service (FSS) inputs
64+
TTS_FSS_CONFIG=
65+
TTS_FSS_MOUNT_DIR=
66+
67+
68+
### Final backup inputs
69+
FINAL_BACKUP=
70+
71+
72+
### Performance inputs
73+
PARALLELISM=
74+
USE_ALL_RAC_INSTANCES=
75+
76+
77+
### Dry run inputs
78+
DRY_RUN=
79+
80+
81+
### Exclude tables input (case-sensitive)
82+
EXCLUDE_TABLES=
83+
84+
85+
## Exclude statistics input (TRUE/FALSE)
86+
## Default to FALSE
87+
EXCLUDE_STATISTICS=

0 commit comments

Comments
 (0)