Enabling custom attack signature only on one security policy

Simply don't assign the newly created user-defined signature to any systems/technology!

Once you have the attack signature in its own own custom signature set you can easily assign/unassign your App1/App2 signature sets from individual policies