Hello,
First of don't forget to check the release notes. For example HTTP Class profile is no longer available in BIG-IP 11.4.0 and later, so you have some operation to perform before Upgrade:
https://support.f5.com/kb/en-us/solutions/public/14000/400/sol14409.html
Additional, Yes it is possible to perform an Upgrade without downtime (minmimum side effet).
First of Upgrade Standby appliance then be watchful that Standby peer does not become active...
If it is the case check that there is no side effect. But if you face any problems switch (force standby/offline) in order to switch in old version. but in general everything goes well :-)
So when you Upgrade The standby, check logs, /var/log/ltm, /var/log/asm (if you use this module)...
If you use ASM you have to wait a little time until the asm service goes up (so do not worry if you see your equipment offline)... if there is nothing special in the logs, you can switch on you 11.5.3 equipement , validate your service and upgrade the primary...
be careful, in some case you have to redo clusturing configuration. And for this you have to reset device trust (so both equipement become active). That's why you have to perform this upgrade during a maintenance period (there could be disturbances on your services)...
For example due to certificate (1024 in old version...)
https://support.f5.com/kb/en-us/solutions/public/15000/600/sol15664.html
Let me know if you need additional information.
Just don't forget to follow advice provide by F5:
https://support.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/bigip-11x-software-upgrade-11-5-0.html