I would date someone not into sports definitely over someone into them. I don't know if this is elsewhere too, but in the UK we get a "lad" mentality when it comes to sport which often comes with sexist and aggressive behaviour towards men and women, so while I have no issue in men who enjoy playing sports or watching them every now and then, the character traits that come with religious sport watching are not ones I enjoy in a partner. My housemate (female) loves watching football and is a huge supporter and says she has trouble dating guys who like football as a lot of them feel put out or like she's impinging on their hobby, or even treat her like she must be lying. There is definitely a culture there which isn't so much genuine love of the sport, but love of the bro bonding time. I have never seriously gone out with a guy who "watches sport", and there is a reason for that! I don't have anything against the activity, just clearly the types who do so.
Playing sports is actually pretty attractive though, it shows levels of fitness and interest in keeping them up, interest in a hobby, proactiveness, teamwork and often a feeling of community.
Watching sports is completely different, how attractive is it really for a person to watch television religiously in a sense that they will put it above their personal lives? If you're watching it in the pub, then how attractive is a person who spends huge amounts of time in the pub with other people who spend lots of time in a pub? Maybe to people who also enjoy those sorts of activities, but unless you like them, then it's not really a big deal, or could be seen as an unattractive trait.