When troubleshooting PostgreSQL issues, we are currently identifying the generic trace, but we missed identifying the absolute cause of that issue and following the workaround, provided. Most of those issues have their own workaround in our KBs with the exact cause.
Trace what we find - 1
- com.zoho.postgres.exception.MigrationFailureException: Migration failed for ALL
- com.zoho.postgres.exception.MigrationFailureException: Migration failed for DATA
- Problem during upgrade.com.zoho.postgres.exception.MigrationFailureException: Restoration failed for PRE_DATA
- com.zoho.postgres.exception.MigrationFailureException: Restoration failed for POST_DATA
How do I find the exact cause?
- Search for the keyword 'ERROR:' which will be located above the failure message.
- Check all the files that were generated in that upgrade attempt, such as updatemgrlog0.txt , updatemgrlog0.txt.1 & updatemgrlog0.txt.2 if they exist.
What are the cases? (4)
Case 1 : Presence of CtrlZ characters
- ERROR: missing data for column "columnoldvalue"
- ERROR: extra data after last expected column
- \n Invalid Command
Case 2 : Privilege Changes of Removed Functions
- ERROR: function <function_name> does not exist
Case 3 : Corruption
- ERROR: could not create unique index "requestnotificationtodesc_pk"
- DETALLE: Key (notificationid)=(293933) is duplicated
- (OR)
- could not locate a valid checkpoint record
- (OR)
- invalid checkpoint record
These corruption tickets shall be moved to G3-Corruption Team.
Case 4 : Database Inconsistency
- Key (<column>)=(<value>) is not present in table "<table>".
These Database inconsistency ticket will be handled by CART-Hariharan P M
Case 5 : Temporary Connectivity Issues
- Sep 17, 2024 3:24:30 PM [com.adventnet.db.adapter.postgres.DefaultPostgresDBInitializer] [INFO] : is DB registered as a service ::: false
- Sep 17, 2024 3:24:30 PM [com.zoho.postgres.migrate.BundledPGMigration] [SEVERE] : nulljava.lang.NullPointerException
- at com.adventnet.db.adapter.postgres.DefaultPostgresDBInitializer.isPgReady(DefaultPostgresDBInitializer.java:1194)
- at com.adventnet.db.adapter.postgres.DefaultPostgresDBInitializer.getServerStatus(DefaultPostgresDBInitializer.java:973)
- at com.adventnet.db.adapter.postgres.DefaultPostgresDBInitializer.isServerStarted(DefaultPostgresDBInitializer.java:913)
Refer this
KB for Troubleshooting Connectivity Issues
Trace what we find-2
Postgresql Upgrade Sanity Failure
- PostgreSQL upgrade sanity failed.
Resolution :
- We can resolve this by executing this below query as postgres user (https://pitstop.manageengine.com/portal/en/kb/articles/how-to-login-into-postgres-database) and invoke upgrade.
- REINDEX DATABASE SERVICEDESK
Trace what we find -3
- Please login to the system as <machine_name>\Administrator and execute initPgsql.bat script with administrator privileges. If issue is not yet solved, please contact support
How do I find the exact cause?
Check the "[pglog]" above this error message in the same file. Match the exact cause with the below cases.
Case 1 : Permission Restriction
- waiting for server to start....postgres: could not access directory "C:/ManageEngine/ServiceDesk/pgsql/data": Permission denied
- (OR)
- waiting for server to start....postgres: could not access directory "C:/Program Files/ManageEngine/ServiceDesk/pgsql/data": No such file or directory
- Run initdb or pg_basebackup to initialize a PostgreSQL data directory
Resolution :
- Open a command prompt as the user, with whom you had installed the application. Or, with the user who you would like to start the application service. Or, with the user who you would like to invoke upgrade. Execute <product_installation_location> \ bin \ initPgsql script.
Refer : "Permission restriction" in
KB
Case 2 : corruption
- FATAL: could not create lock file "postmaster.pid": Permission denied
This ticket shall be moved to G3-Corruption Team
Case 3 : Environmental Delay
- waiting for server to start............................................................... stopped waiting
- pg_ctl: server did not start in time
Resolution :
- We can resolve this issue by excluding <product_installation_folder> from the Antivirus/Windows Defender Intervention and For a safer side, ensuring no other process is running inside the <product_installation_folder> directory.
- If possible, try disabling those defender processes temporarily. Refer this KB further for troubleshooting connectivity failures.
Case 4 : Unstable application instance
- program "postgres" was found by "pg_ctl.exe" but was not the same version as pg_ctl
Resolution :
- This issue is caused by an unstable application instance. Hence, make a fresh setup and restore the data there.
Case 5 : Native Library Conflicts (Linux Specific cases)
- su: cannot open session: Module is unknown
Resolution :
- LD_LIBRARY conflict issue caused in Linux. Refer to this KB.
ToDo If the issue persist
If the issue persists even after following all these instructions, kindly report it to our support team with the below files.
- <product_installation_location>\logs
- <product_installation_location>\pgsql\data\postgresql.conf
- <product_installation_location>\pgsql\data\pg_hba.conf
- <product_installation_location>\pgsql\ext_conf
- output of <product_installation_location> \ bin \ initPgsql script