Is there a size limit or recommended maximum size for external datagroup files.

If you have that many redirects you may want to look for patterns. That might save a lot of work/maintenance.

Recently our content group sent me a list of around 15,000 redirects. I was able to bring that down to maybe half a dozen pattern-based redirects and perhaps a few hundred unique redirects.