Note:
These notes are intended for RH AAP upgrades from AAP 2.1 or higher to AAP 2.4. Upgrades previous to 2.1 require a migration of Ansible content for compatibility with AAP 2.4.
Overview:
A. Set up the existing AAP inventory to match the existing environment.
B. Download and run the 2.4 installation program over top of the existing AAP installation.
Automation Controller Upgrade
Warning:
- Credentials or the subscription manifest is required after performing the upgrade on the license page
- Pre-AAP 2.1, RH Enterprise Linux and the AAP Controller upgrades require back-up of the AAP controller data, and then restoration of it.
Automation Hub Upgrade
Warning:
- If you generate a new token instead of adding an existing token, this will invalidate any existing tokens with the hub.
Prerequisites:
1. Backup the current environment. In this case, the location is: /myadminid/aapinstall/aap-setup-bundle-2.3-11-x86_64/
a. Navigate to the last install location still containing the extracted installation folder with the inventory file and the apitoken.
b. Run the backup:
$ sudo su -
# cd /myadminid/aapinstall/aap-setup-bundle-2.3-11-x86_64/
# mkdir /myadmin/aapinstall/aap2-3-11backup/
# ./setup.sh -e 'backup_dest=/myadmin/aapinstall/aap-2-3-11backup/' -b
# exit
2. Ensure you have all your LDAP set-up information.
3. Ensure you AAP is using execution environments (EEs) rather than the older virtual environments.
Convert them, if not.
4. If using EDA, Event Driven Ansible, you'll need to install a separate VM for it.
a. Deploy a RHEL 9 VM, 16 GB RAM, 4 CPU, 40 GB (/ drive)
b. Add an extra disk, via Edit Settings --> Add New Device --> Disk (e.g. 40 GB)
- $ ssh myadminid@aapeda@mindwatering.net
<enter pwd>
- - Confirm the device id:
- - - $ sudo lsblk
- - Format the new disk
- - - $ sudo fdisk /dev/sdb
- - - - Command: n -> p 1 -> nnnn - nnnnnnnnnn <enter>
- - - - Command: w
- - Format the OS:
- - - $ sudo mkfs.xfs -f /dev/sdb
- - Map the new disk via fstab as/var/awx
- - - $ sudo vi /etc/fstab
We have other tasks but we'll do them together below.
5. Verify the RAM on the controllers
Ansible recommends 1GB memory/10 forks, and a 2GB reservation for the AAP controller.
6. AAP Controller requires umask be set to 0022. If is not set, the setup.sh will fail.
This is the default for most Linux environments for normal users.
- To change for all users:
$ sudo vi /etc/profile.d/set-umask-for-all-users.sh
- To adjust just the "current" user
$ vi ~/.bashrc
<a to append>
umask022
<esc>:wq to save and close
7. Verify access to the AAP Controller PostgreSQL 13 db:
$ sudo awx-manage check_db
8. Ensure your ansible_user account set-up in the inventory file is not expired and that the ansible_private_key_file private key is still working.
e.g. if expired:
$ sudo chage -M <max_days> ansible_user
9. Set-up the ansible_user account on the new Event Driven Ansible VM created above. Add the public key to the VM so that this user from the AAP Controller appliance/VM can remotely log-in/ssh. Test a SSH session from the main AAP Controller.
a. Create the ansible_user on the AAP EDA controller:
$ ssh myadminid@aapeda.mindwatering.net
<enter pwd>
$ sudo useradd -d /users/ansible_user -m ansible_user
$ sudo usermod -a -G wheel ansible_user
$ sudo passwd ansible_user
<enter new pwd>
<enter new pwd confirmation>
$ exit
b. Copy the key to the AAP EDA controller:
$ ssh ansible_user@aapdev.mindwatering.net
<enter pwd>
$ ssh-copy-id ansible_user@aapeda.mindwatering.net
<enter pwd>
c. Test the ssh using the key copied:
$ ssh aapeda.mindwatering.net
<verify auto login worked>
$ exit
(to exit appeda)
$ exit
(to exit appdev)
10. Ensure that /tmp and /var/tmp are mounted as exec
$ ssh myadminid@aapdev.mindwatering.net
$ sudo vi /etc/fstab
...
mount -o remount,exec /tmp
mount -o remount,exec /var/tmp
...
<esc>:wq to save and close
Repeat above on the aaphub.mindwatering.net and the aapeda.mindwatering.net
11. Ensure that /var/tmp is 755
$ ssh myadminid@aapdev.mindwatering.net
$ sudo chmod 755 /var/log
12. If you have your system with yum updates disabled, then you need to enable yum to update RH packages. Update yum.conf before the upgrade, remove the following (if you have it) and then re-add it back afterwards
a. Update yum.conf:
$ ssh myadminid@aapdev.mindwatering.net
$ sudo vi /etc/yum.conf
Remove the line: exclude=kernel* redhat-release*
...
<esc>:wq to save and close
b. Update dnf.conf:
$ sudo vi /etc/dnf/dnf.conf
Remove the line: exclude=kernel* redhat-release*
...
<esc>:wq to save and close
Repeat above on the aaphub.mindwatering.net and the aapeda.mindwatering.net
13. The ansible user and the setup.sh program need sudo to be enabled on the AAP Controller, the AAP Hub, and the AAP EDA Controller:
Uncomment the line below, noting to uncomment the one that does not require a password:
$ ssh myadminid@aapdev.mindwatering.net
$ sudo visudo
...
%wheel ALL=(ALL) NOPASSWD: ALL
...
<esc>:wq to save and close
Repeat above on the aaphub.mindwatering.net and the aapeda.mindwatering.net
14. The upgrade will likely override custom cert/keys. If applicable, perform backups of those files:
a. Backup the main AAP controller tower.key and tower.cert files:
$ ssh myadminid@aapdev.mindwatering.net
$ <enter pwd>
$ sudo vi /etc/nginx/conf.d/automation-controller.nginx.conf
<read location e.g. /etc/tower/tower.cert and /etc/tower/tower.key>
$ sudo cp /etc/tower/tower.cert /etc/tower/tower_backup_yyyymmdd.cert
$ sudo cp /etc/tower/tower.key /etc/tower/tower_backup_yyyymmdd.key
$ exit
b. Backup the AAP Hub key and cert files:
Repeat above on the aaphub.mindwatering.net but also backup the custom cert in the pulp config folder
$ ssh myadminid@aapdev.mindwatering.net
<enter pwd>
$ sudo cp /etc/pulp/certs/pulp_webserver.crt /etc/pulp/certs/pulp_webserver.crt_backup_yyyymmdd.crt
$ sudo cp /etc/pulp/certs/pulp_webserver.key /etc/pulp/certs/pulp_webserver.key_backup_yyyymmdd.key
$ exit
Note:
aapeda.mindwatering.net cert and key do not need to be backed up since the EDA does not exist before this upgrade
Upgrade:
1. Download the RH AAP latest 2.4 version:
Browser --> access.redhat.com/downloads/content/480/ver=2.4/rhel---9/2.4/x86_64/product-software
--> Login
Look for the offline bundle version. It includes dependencies and can make installation a bit easier.
- naming format of the "offline" bundle version:
ansible-automation-platform-setup-bundle-<latest-version>.tar.gz
If not retrieved on the AAP server, transfer the downloaded file to the AAP controller VMs via SSH/SCP (e.g. Filezilla)
e.g. to /myadminid/aapinstall/
Uncompress the tar file:
$ ssh myadminid@aapdev.mindwatering.net
<enter pwd>
$ cd /users/myadminid/aapinstall/aap-setup-bundle-2.3-11-x86_64/
$ tar xvzf ansible-automation-platform-setup-bundle-<latest-version>.tar.gz
2. Update the AAP inventory file in the archive, or re-use the current one if nothing has changed. The inventory file is part of the expanded tar extract. The file path differs based on the tar downloaded ("internet" or "bundle") For the offline bundle version...
$ cd ansible-automation-platform-setup-bundle-2.4-1-x86_64
To reuse the existing inventory file and token from the previous installation:
Move the default file to a backup file.
$ mv inventory inventory_backup
$ cp /users/myadminid/aapinstall/inventory ./
$ cp /users/myadminid/aapinstall/apitoken ./
Inventory file notes:
- The use of localhost for any [automationhub] or [automationcontroller] sections of the file is not allowed.
- Add node_state-deprovision to the end of the node line to deprovision only isolated nodes, not execution nodes.
e.g. hostname.domain ansible_host=192.168.111.115 node_type=hybrid node_state=deprovision
- pg_password can only contains the special characters: !, #, 0, and @. The admin username cannot be changed.
- Reuse the existing token like below:
$ vi inventory
automationhub_api_token=<api_token>
...
<esc>:wq, to save
Example simple inventory file from RH AAP documentation:
The following uses an embedded (non external database). With an embedded database, the host under the [database] heading is left out, the pg_host is an empty string (e.g. pg_host='') and the port 5432 is still populated (e.g. pg_port=5432).
$ sudo vi inventory
[automationcontroller]
aapdev.mindwatering.net
[automationhub]
aaphub.mindwatering.net
[automationedacontroller]
aapeda.mindwatering.net
[database]
[all:vars]
admin_password='guiadminpassword'
pg_host=''
pg_port=5432
pg_database='awx'
pg_username='awx'
pg_password='<awxpassword>'
registry_url='registry.readhat.io'
registry_username='reguserid'
registry_password='regpassword'
automationcontroller_main_url= 'https://appdev.mindwatering.net'
automationhub_pg_host='hubdev.mindwatering.net'
automationhub_pg_port=5432
automationhub_pg_database="automationhub'
automationhub_pg_username='automationhub'
automationhub_pg_password='HubReallyAmazingPassword'
automationhub_pg_sslmode = 'prefer'
automationhub_authentication_backend = "ldap"
automationhub_ldap_server_uri = "ldaps://ldapserverint.mindwatering.net"
automationhub_ldap_bind_dn = "cn=myldapadmin, dc=mindwatering, dc=net"
automationhub_ldap_bind_password = "MyBetterThanGoodPassword"
automationhub_ldap_user_search_base_dn = "ou=people, dc=mindwatering, dc=net"
automationhub_ldap_group_search_base_dn = "ou=aapgroup, dc=mindwatering, dc=net"
automationedacontroller_admin_password='EDAUIReallyAmazingPassword'
automationedacontroller_pg_host = 'edadev.mindwatering.net'
automationedacontroller_pg_port=5432
automationedacontroller_pg_database ='automationedacontroller'
automationedacontroller_pg_username='automationedacontroller'
automationedacontroller_pg_password='EDAReallyAmazingPassword'
automationedacontroller_pg_sslmode='prefer'
...
<esc>:wq to save and close
3. After the modifications of the inventory are complete, run the installer:
$ ./setup.sh
Post upgrade.
1. If you had/have custom certs, ssh back into the main AAP Controller, and the AAP Hub, and restore the certificates back on the main AAP controller and hub.
a. Update certificate back on the main AAP controller:
$ ssh myadminid@aapdev.mindwatering.net
<enter pwd>
$ sudo mv /etc/tower/tower.cert /etc/tower/tower_backup_post2-4upgrade.cert
$ sudo mv /etc/tower/tower.key /etc/tower/tower_backup_post2-4upgrade.key
$ sudo mv /etc/tower/tower_backup_yyyymmdd.cert /etc/tower/tower.cert
$ sudo cp /etc/tower/tower_backup_yyyymmdd.key /etc/tower/tower.key
$ sudo systemctl restart nginx.service
Verify the GUI works in the web browser.
$ exit
b. Update the certificate back on the main AAP Hub
$ ssh myadminid@aaphub.mindwatering.net
<enter pwd>
$ sudo mv /etc/pulp/certs/pulp_webserver.crt /etc/pulp/certs/pulp_webserver_backup_post2-4upgrade.crt
$ sudo mv /etc/pulp/certs/pulp_webserver.key /etc/pulp/certs/pulp_webserver_backup_post2-4upgrade.key
$ sudo mv /etc/pulp/certs/pulp_webserver.crt_backup_yyyymmdd.crt /etc/pulp/certs/pulp_webserver.crt
$ sudo mv /etc/pulp/certs/pulp_webserver.crt_backup_yyyymmdd.key /etc/pulp/certs/pulp_webserver.key
$ sudo systemctl restart nginx.service
Verify the GUI works in the web browser.
$ exit
______________
Additional Notes:
Ansible Application (Private) Hub Upgrade Variables for Inventory File:
LDAP Information:
Folder: ldap_extra_settings
Required settings:
automationhub_authentication_backend = "ldap"
automationhub_ldap_server_uri = "ldaps://ldapserverint.mindwatering.net"
automationhub_ldap_bind_dn = "cn=myldapadmin, dc=mindwatering, dc=net"
automationhub_ldap_bind_password = "MyBetterThanGoodPassword"
automationhub_ldap_user_search_base_dn = "ou=people, dc=mindwatering, dc=net"
automationhub_ldap_group_search_base_dn = "ou=aapgroup, dc=mindwatering, dc=net"
Additional settings:
automationhub_ldap_user_search_scope = "SUBTREE"
automationhub_ldap_user_search_filter = "(uid=%(user)s)"
automationhub_ldap_group_search_scope = "SUBTREE"
automationhub_ldap_group_search_filter = "(objectClass = Group)"
automationhub_ldap_group_type_class = "django_auth_ldap.config:GroupOfNamesType"
Note:
For AD LDAP, the user_search_filter and group_type_class will likely be like:
automationhub_ldap_user_search_filter = "(sAMAccountName=%(user)s)"
automationhub_ldap_group_type_class = "django_auth_ldap.config:NestedActiveDirectoryGroupType"
Example Passing Additional LDAP settings (e.g. a field map) with the setup.sh run:
# vi ldapextras.yml
#ldapextras.yml
---
ldap.extra_settings:
AUTH_LDAP_USER_ATTR_MAP: '{"first_name": "FirstName", "last_name": "LastName", "email": "MailAddress"}'
AUTH_LDAP_REQUIRE_GROUP: 'cn=aapadmins,ou=groups,dc=mindwatering,dc=net'
<esc>:wq (to save)
Note:
For AD LDAP, the user field mapping will be more like:
AUTH_LDAP_USER_ATTR_MAP: '{"first_name": "givenName", "last_name": "sn", "email": "upn"}'
To run the setup with the custom map:
# ./setup.sh -e @ldapextras.yml
RH SSO to LDAP for AAP Controller:
Scan for sso lines in the inventory file to comment out or remove:
[sso]
ssoserverint.mindwatering.net
...
# sso_host=''
# sso_custom_keystore_file='/path/to/sso.jks'
# sso_keystore_password='********'
# sso_console_admin_password='********'
Add the LDAP configuration above.
previous page
|