Java System Startup Issue: Application Server Unable to Connect to Message Server

Estimated read time 6 min read

Overview

This blog post describes a production issue where a SAP Java application server failed to start because it could not connect to the Message Server. The root cause was an SSL/PSE certificate misconfiguration on the SCS instance, which prevented the HTTPS port from activating. Combined with a security hardening change that had also disabled the HTTP port, the application server had no available port to reach the Message Server on.

 

Symptom

After starting the SCS instance, the Java Application Server (AS Java) failed to come up. Reviewing the AS Java startup logs (std_server*.log or dev_server*.log) revealed the following error:

This error indicates the AS Java was attempting to discover the Message Server but found neither the HTTP nor HTTPS port to be reachable.

 

Investigation

Step 1 — Checking dev_ms logs

The dev_ms log (located at /usr/sap/<SID>/SCS<NR>/work/dev_ms) showed that the Message Server HTTPS port had failed to activate due to an SSL/PSE error

The key error is SSSLERR_PSE_MISSING_PIN — the SSL PSE file (SAPSSLS.pse) used by the SCS instance for server-side HTTPS was either corrupt, had a mismatched PIN, or its certificate had not been properly signed and imported.

Step 2 — Confirming HTTP port was also disabled

Checking the Message Server profile (DEFAULT.PFL or instance profile of SCS) confirmed the HTTP port had been intentionally disabled as part of a security hardening measure recommended by an Onapsis vulnerability assessment

With #ms/server_port_0 , the plain HTTP Message Server port was deactivated. The only available port for external AS Java connectivity was HTTPS. Since HTTPS also failed to activate (due to the PSE issue), the Message Server was effectively unreachable from the application server perspective.

Root Cause

The root cause was a broken SSL/PSE configuration on the SCS instance. The SAPSSLS.pse file — which holds the server certificate used by the Message Server for HTTPS — was either:

Not properly signed by a trusted CA (or self-signed certificate not re-signed after a system change),Referencing a PIN/credential that no longer matched, orCorrupted / not updated after a certificate renewal or system copy.

As a result, the HTTPS port on the Message Server could not initialize. Since the HTTP port was simultaneously disabled (Onapsis hardening), there was no port available for the AS Java to communicate with the SCS Message Server, causing the startup to fail entirely.

Resolution:

The fix involved re-creating the PSE file since the Password we had was not working and then re-performing the SCS HTTPS SSL certificate signing process by Re-sign the certificate using the appropriate CA then Import the signed certificate back into the PSE (SAPSSLS.pse) for the SCS instance. Restart the SCS instance and monitor the dev_ms log to confirm the HTTPS port activates successfully. Restarted the applcation server and now it was able to connect to message server.

Key Takeaways

Always verify PSE/SSL certificates on SCS after any certificate renewal, system copy, or credential rotation.

When disabling the HTTP port on the Message Server (e.g., per Onapsis or security hardening guidelines), ensure the HTTPS port is fully functional before the change goes live — otherwise you lose all AS Java connectivity. Port activation can be verified either from dev_ms logs or we can even use https://<sapscs/ascs servername>:<httpsport>/msgserver/commands and if this url opening with HTTPS that mean SSL is successfully activated. 

 

​ OverviewThis blog post describes a production issue where a SAP Java application server failed to start because it could not connect to the Message Server. The root cause was an SSL/PSE certificate misconfiguration on the SCS instance, which prevented the HTTPS port from activating. Combined with a security hardening change that had also disabled the HTTP port, the application server had no available port to reach the Message Server on. SymptomAfter starting the SCS instance, the Java Application Server (AS Java) failed to come up. Reviewing the AS Java startup logs (std_server*.log or dev_server*.log) revealed the following error:This error indicates the AS Java was attempting to discover the Message Server but found neither the HTTP nor HTTPS port to be reachable. InvestigationStep 1 — Checking dev_ms logsThe dev_ms log (located at /usr/sap/<SID>/SCS<NR>/work/dev_ms) showed that the Message Server HTTPS port had failed to activate due to an SSL/PSE errorThe key error is SSSLERR_PSE_MISSING_PIN — the SSL PSE file (SAPSSLS.pse) used by the SCS instance for server-side HTTPS was either corrupt, had a mismatched PIN, or its certificate had not been properly signed and imported.Step 2 — Confirming HTTP port was also disabledChecking the Message Server profile (DEFAULT.PFL or instance profile of SCS) confirmed the HTTP port had been intentionally disabled as part of a security hardening measure recommended by an Onapsis vulnerability assessmentWith #ms/server_port_0 , the plain HTTP Message Server port was deactivated. The only available port for external AS Java connectivity was HTTPS. Since HTTPS also failed to activate (due to the PSE issue), the Message Server was effectively unreachable from the application server perspective.Root CauseThe root cause was a broken SSL/PSE configuration on the SCS instance. The SAPSSLS.pse file — which holds the server certificate used by the Message Server for HTTPS — was either:Not properly signed by a trusted CA (or self-signed certificate not re-signed after a system change),Referencing a PIN/credential that no longer matched, orCorrupted / not updated after a certificate renewal or system copy.As a result, the HTTPS port on the Message Server could not initialize. Since the HTTP port was simultaneously disabled (Onapsis hardening), there was no port available for the AS Java to communicate with the SCS Message Server, causing the startup to fail entirely.Resolution:The fix involved re-creating the PSE file since the Password we had was not working and then re-performing the SCS HTTPS SSL certificate signing process by Re-sign the certificate using the appropriate CA then Import the signed certificate back into the PSE (SAPSSLS.pse) for the SCS instance. Restart the SCS instance and monitor the dev_ms log to confirm the HTTPS port activates successfully. Restarted the applcation server and now it was able to connect to message server.Key TakeawaysAlways verify PSE/SSL certificates on SCS after any certificate renewal, system copy, or credential rotation.When disabling the HTTP port on the Message Server (e.g., per Onapsis or security hardening guidelines), ensure the HTTPS port is fully functional before the change goes live — otherwise you lose all AS Java connectivity. Port activation can be verified either from dev_ms logs or we can even use https://<sapscs/ascs servername>:<httpsport>/msgserver/commands and if this url opening with HTTPS that mean SSL is successfully activated.    Read More Technology Blog Posts by Members articles 

#SAP

#SAPTechnologyblog

You May Also Like

More From Author