How to fix 'Problem parsing XML' / '500 server error' in the Zimbra frontend

04.12.2019 18:57 sysadmin zimbra zcs en

Today I came across an error that damn near made me rip my hair out. I couldn’t find an english article on how to solve it, and having to translate something while you’re frantically troubleshooting is not a great experience, so I decided to write it down here.

TL;DR

Check if you have a malformed zimbraHttpThrottleSafeIPs entry in your config, and delete it directly from LDAP.

What happened?

Not much! I just installed a new TLS certificate and restarted Zimbra, nothing special - until I realized that both the normal user interface and the admin interface had stopped working with a message like “500 Internal server error”. On each reload of the page, the following was written to /opt/zimbra/log/mailbox.log:

2019-12-04 17:47:35,364 WARN  [qtp509886383-145:https://10.0.42.6:443/zimbra/] [] misc - Problem parsing XML - Fatal Error: Problem on line 10 of document : The element type "hr" must be terminated by the matching end-tag "</hr>".
2019-12-04 17:47:35,484 WARN  [qtp509886383-145:https://10.0.42.6:443/zimbra/css/common,login,zhtml,skin.css?skin=&v=] [] misc - Problem parsing XML - Fatal Error: Problem on line 10 of document : The element type "hr" must be terminated by the matching end-tag "</hr>".
2019-12-04 17:47:35,557 WARN  [qtp509886383-145:https://10.0.42.6:443/zimbra/img/dwt/ImgWarning_32.png?v=] [] misc - Problem parsing XML - Fatal Error: Problem on line 10 of document : The element type "hr" must be terminated by the matching end-tag "</hr>".

And, oh horror, using zmprov resulted in the following output:

[] WARN: Problem parsing XML - Fatal Error: Problem on line 10 of document : The element type "hr" must be terminated by the matching end-tag "</hr>".
ERROR: zclient.IO_ERROR (invoke unable to parse response: <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 500 Server Error</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /service/admin/soap/AuthRequest. Reason:
<pre>    Server Error</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>

</body>
</html>
, server: localhost) (cause: com.zimbra.common.soap.SoapParseException unable to parse response: <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 500 Server Error</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /service/admin/soap/AuthRequest. Reason:
<pre>    Server Error</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>

</body>
</html>
)

Drop the SOAP

The reason for that was that SOAP had become completely unusable. Using zmprov to talk directly to LDAP (as in, zmprov --ldap gacf) still worked. The problem was a faulty entry that I had made over a month ago: zmprov mcf +zimbraHttpThrottleSafeIPs 192.168.42.163/32, which works in newer Zimbra versions, but makes all hell break loose in 8.6.0. That release only supports putting in singular IP addresses - no CIDR! Today was the first restart of zmmailbox after making that change, which made Zimbra crash.

The solution

As instructed by this article I simply deleted that configuration entry directly from LDAP. I used JXplorer, but ldapmodify should do the trick just fine if you’re comfortable with it. Then, restart zmmailbox, and everything will be fine. In hindsight, I wonder if zmprov --ldap mcf -zimbraHttpThrottleSafeIPs "192.168.42.163/32" would’ve also worked, but I’m not exactly eager to go back and try it out ;)