Sorry, @Anu Momin, I was commenting on the original question, not your comment, so my answer may not have made sense for your scenario.
A couple things to consider for your scenario...
I would use caution proceeding without enforcing a signed assertion (the "Want Signed Assertion" value). Without that someone may be able to forge an assertion and pretend to be another user.
One possible thing to consider for your situation with the assertion validation issue is that when importing the XML for the external IdP connector it may not have properly imported or attached the ADFS signing certificate. You could go into the external IdP connector and check the "Security Settings" section and see if the IdP Assertion Verification Certificate is selected there.
Regarding the backend auth, it sounds like you're on the right track and moving towards getting Splunk to use SAML auth, since you won't be able to just form fill a logon page that does LDAP on the backend since you don't get the password from ADFS.