Announcement

Collapse
No announcement yet.

Rankings App Flaw

Collapse
X
  •  
  • Filter
  • Time
  • Show
Clear All
new posts

    Rankings App Flaw

    So I found a major potential flaw in the youthsoccerrankings app and it revolves around the SOS algorithm and its weight.

    Prime example: Currently DUSC plays in the NAL. Teams SOS against are held low bc of presumed level of play. When DUSC gets GA In Fall and loses every game, every team that they have played in the past will automatically get a boost just bc the SOS will now be top 10 verses top 25.

    This confirm too much weight is given to SOS. DUSC is just an example. I'm sure there are more teams going through similar promotions.

    #2
    Gotta admit I clicked on this link expecting something a bit more substantive ...

    DUSC' SOS should certainly rise when they move to GA and play tougher opponents (note the Rankings app does not explicitly take the league into account). Their previous opponents will only get a boost if DUSC' overall rating rises as a result, which it may not if they get blown out every game.

    Comment


      #3
      I have issues with the league "power" calculation that's added to results to define rankings as well.

      It benefits bad teams in a strong league if the happen to win and it penalizes strong teams in a weak league if they happen lose.

      Over time rankings will band by league if there's never any cross league play.

      Put more simply a really good GA team needs to beat their league opponents by 8+ goals to move up in rankings. While if they happen to lose to GA team by 1-2 goals they drop in rankings quickly.

      Girls ECNL results are the opposite.

      Comment


        #4
        Originally posted by Guest View Post
        So I found a major potential flaw in the youthsoccerrankings app and it revolves around the SOS algorithm and its weight.
        Let's assume you mean the Soccer Ranking app.

        Prime example: Currently DUSC plays in the NAL. Teams SOS against are held low bc of presumed level of play. When DUSC gets GA In Fall and loses every game, every team that they have played in the past will automatically get a boost just bc the SOS will now be top 10 verses top 25.

        This confirm too much weight is given to SOS. DUSC is just an example. I'm sure there are more teams going through similar promotions.
        It doesn't use "presumed level of play." It uses the actual opponents (in your example the NAL teams), and then also their opponents, etc. So there is no "automatic boost" for the league they are in. "We don't make any assumptions about the quality of leagues or even the age of teams. Instead, we look at the results of all teams and calculate how good every team is (this is their rating)." from https://usasportstatistics.net/ see the bottom sections

        As another example, the reason teams like Stars White can have a high rating even without winning games is that the individual opponents they play are strong teams. If they keep the final score close, it indicates that they deserve a rating not too far below those teams. It is not because they are in ECNL with any "presumed level of play."

        By the way, normally the SOS calculation is run multiple times to reach a solution. It is iterative. So you calculate it for the opponents, then recalculate it for updated scores of their opponents, etc. until the numbers stop changing. The MIAA Power Ratings are similar.

        The app uses one year of data. Older games get rotated out.

        Also, "More recent games and games against a similar team are given higher weight."

        The real issue - not flaw - with the app is that you never know what the roster is of the team you are facing on any given day. Especially in showcases. Teams pull together different players under the same team name.

        But the app does the best with the data that is available.

        Originally posted by Guest View Post
        I have issues with the league "power" calculation that's added to results to define rankings as well.

        It benefits bad teams in a strong league if the happen to win and it penalizes strong teams in a weak league if they happen lose.

        Over time rankings will band by league if there's never any cross league play.

        Put more simply a really good GA team needs to beat their league opponents by 8+ goals to move up in rankings. While if they happen to lose to GA team by 1-2 goals they drop in rankings quickly.

        Girls ECNL results are the opposite.
        There is no league power calculation included, as noted above.

        If the best team in a league loses to the weakest team its ranking should drop, and the lower ranked team's rating should go up. That's just as true in ECNL or GA or most any league.

        It does seem true that the fewer number of games between teams in different leagues makes it harder to compare the ratings between (for example ECNL and GA). Though if you look at this weekend's schedule for PDA, there are a number of inter-league games. So there will be a few or several games that could be used to infer the relative strength of teams in ECNL and GA. There are some other showcases over the season where that's the case.

        Comment


          #5
          Originally posted by Guest View Post
          Let's assume you mean the Soccer Ranking app.



          It doesn't use "presumed level of play." It uses the actual opponents (in your example the NAL teams), and then also their opponents, etc. So there is no "automatic boost" for the league they are in. "We don't make any assumptions about the quality of leagues or even the age of teams. Instead, we look at the results of all teams and calculate how good every team is (this is their rating)." from https://usasportstatistics.net/ see the bottom sections

          As another example, the reason teams like Stars White can have a high rating even without winning games is that the individual opponents they play are strong teams. If they keep the final score close, it indicates that they deserve a rating not too far below those teams. It is not because they are in ECNL with any "presumed level of play."

          By the way, normally the SOS calculation is run multiple times to reach a solution. It is iterative. So you calculate it for the opponents, then recalculate it for updated scores of their opponents, etc. until the numbers stop changing. The MIAA Power Ratings are similar.

          The app uses one year of data. Older games get rotated out.

          Also, "More recent games and games against a similar team are given higher weight."

          The real issue - not flaw - with the app is that you never know what the roster is of the team you are facing on any given day. Especially in showcases. Teams pull together different players under the same team name.

          But the app does the best with the data that is available.

          There is no league power calculation included, as noted above.

          If the best team in a league loses to the weakest team its ranking should drop, and the lower ranked team's rating should go up. That's just as true in ECNL or GA or most any league.

          It does seem true that the fewer number of games between teams in different leagues makes it harder to compare the ratings between (for example ECNL and GA). Though if you look at this weekend's schedule for PDA, there are a number of inter-league games. So there will be a few or several games that could be used to infer the relative strength of teams in ECNL and GA. There are some other showcases over the season where that's the case.
          good post. The point re roster is 100% correct. The app is also dependent on the data given to it, I’ve seen teams mislabeled when they enter a tournament and it takes a year for an obvious set of outlier results to rotate out

          Comment


            #6
            With whatever flaws that rankings app has, it's still the best indicator of how teams will fare against each other. I'm so much happier to look at this app for upcoming opponents vs looking at the stupid rankings from Got Soccer in the past.

            Comment


              #7
              All anyone needs to know is that Stars Blue throughout all age groups is the top Mass. team in each division. Pretty simple.

              Comment


                #8
                Originally posted by Guest View Post
                With whatever flaws that rankings app has, it's still the best indicator of how teams will fare against each other. I'm so much happier to look at this app for upcoming opponents vs looking at the stupid rankings from Got Soccer in the past.
                does that make or break your day?

                Comment


                  #9
                  Originally posted by Guest View Post
                  All anyone needs to know is that Stars Blue throughout all age groups is the top Mass. team in each division. Pretty simple.
                  Definitely don't let the data confuse you on that one ;)

                  Comment


                    #10
                    Originally posted by Guest View Post
                    All anyone needs to know is that Stars Blue throughout all age groups is the top Mass. team in each division. Pretty simple.
                    Stars Blue doesn’t care about winning. Only exposure. Pay attention.

                    Comment

                    Previously entered content was automatically saved. Restore or Discard.
                    Auto-Saved
                    x
                    Insert: Thumbnail Small Medium Large Fullsize Remove  
                    x
                    Working...
                    X