[DEV] ISFL Forums
*Improving the New User Experience - Printable Version

+- [DEV] ISFL Forums (http://dev.sim-football.com/forums)
+-- Forum: Community (http://dev.sim-football.com/forums/forumdisplay.php?fid=5)
+--- Forum: Media (http://dev.sim-football.com/forums/forumdisplay.php?fid=37)
+---- Forum: Graded Articles (http://dev.sim-football.com/forums/forumdisplay.php?fid=38)
+---- Thread: *Improving the New User Experience (/showthread.php?tid=21758)

*Improving the New User Experience - Sylphreni - 04-28-2020

For those who may not know, I entered the S23 prospect class as a completely new player to sim leagues. While I do have some experience in forum-based online games, a sim league was a completely foreign idea to me. It’s been an incredible ride for me thus far in the NSFL, and I’ve enjoyed it immensely! Today I hope to shine a light on a few things that I noticed along the way, in an attempt to improve the League's long term retention of new users based on things I’ve noticed as well as my own experience.

It should be said at the beginning of this piece, that my intentions with this are not malicious to the league as a whole, nor do I wish for this to become some sort of witch hunt. Rather, I hope by analyzing the data made available to me we as a league can collectively begin to consider changes in how we recruit and retain new players.

A couple of caveats before we begin:
  • Any data provided to me, was scrubbed of any player names or other identifying features. Users anonymity was maintained throughout the process.
  • The majority of this article will be filled with findings pulled from the dates between March 27th and April 23rd. March 27th was selected as, to my understanding, it was the rough cut off for when new users became Season 23 class candidates.April 23rd was selected, largely because the DSFL draft has concluded, and players have since been allotted to their respective teams. Also…. If I was ever to get this piece out, I needed to select a cutoff date at some point. The 23rd seemed fitting, as it becomes difficult to track new creates post-draft, across the various forums.
  • The actual class size as of the current moment is also a bit challenging to nail down. The DSFL draft boasted 78 players, since that time there have been several new creates, added off of waivers. Having asked around, the rough number I’ve been given for those new creates is 8, bringing the total class size to 86, as of the end of April 23rd. If anyone can officially confirm the actual number of new creates in that time period, I’m happy to adjust the numbers below accordingly, but for our purposes now… 86 is the number.
  • Important to remember too (which I did not on the first draft of this article….) that there are roughly 19 recreates in this class (again a bit variable, but that's the number I got through HO. Bringing our potential number of surveys down to 67.
  • Admittedly, there is a margin of error to be had here, as there is potential that some surveys were submitted by players in the S22 class, therefore inflating the numbers of gathered surveys. Also, as I stated in the prior bullet point, the class size I’m using may be a tad low. Regardless, both offer a degree of variance which would only affect the numbers I’m about to give by a percentage point or two, and to be honest… I’m kind of ok with that. I still think it’s reasonable to be able to use these numbers on the basis of some conclusions I’ll look to draw at the end.
Sooooo, with all that out of the way let's look at some data. My search began with looking into the Rookie Surveys. I know from speaking to HO, that they really value the feedback they receive from the surveys and that seemed like an obvious place to start. The surveys themselves ask for user feedback in a number of ways, but the ones I most concerned myself with were the following three: “Was our site easy to navigate?”, “Did you have trouble making your player?”, and “Did you join our league Discord?” I’m going to examine the first two now, and we’ll address the Discord question after, because it takes a little more explaining. Here were the findings of the first two questions:

[div align=\\\"center\\\"][Image: table1.png][/div]

Important to remember at this point, how vital that first number is. Out of a possible 67 new creates, only 46 had completed the Rookie Survey during the dates I mentioned at the beginning of this article. Now it is possible of course that more have filled it out since writing this, but 69% feels like a number the league should look to improve upon to me, especially if it is one of the primary ways we as a league are gathering feedback from new users. Also important is the lost feedback from those 21 who did not provide a survey. Presumably some of those would be people who did not progress much farther beyond the point of player creation, and yet now we’ll never know for what reason they left. It's very possible those reasons were not league or site specific, but that information would presumably be very valuable to have.

I began to look into why the leagues return rate on Rookie Surveys as it had caught my attention. The Rookie Surveys are part of the Pookie Point Tasks, one of the easiest ways for new players to earn TPE and money quickly, something new players should be overly excited about getting. However, there within lies part of the problem in my opinion. If you take a look at the Rookie Guide the place all new creates go to begin their path in the league, you’ll see there is no mention of the Rookie Point Tasks…… anywhere. Furthermore, upon closer inspection, there is no mention of the Rookie Survey anywhere in the Rookie Guide either. To me this is a huge misstep by the league as a whole, especially when you factor in that 27.78% of Rookies find the site difficult to navigate, and 25.71% are struggling to make their player to begin with. To then expect them to be able to find the Rookie Tasks on their own, is a big ask, especially for someone new to forum based sim leagues.

Speaking from personal experience now, I never would have found the Rookie Point tasks or Survey had it not been for my teammates in Tijuana helping guide me through the process. I was completely lost after character creation, and really had no idea of what to do next. As I write that, I realize most of you are probably saying to yourself, “exactly, we help them through that process” and I really do believe that. Once a new user makes it to Discord, I know that many members of the league are excited to answer their questions. I myself experienced this when Jay_Doctor reached out to me and asked how things were going… I have said in the past and still maintain that had Jay not reached out to me I never would have continued on with the league and just gone IA (thanks Jay.) This brings us to that third question on the survey however: “Did you join our League Discord?”

[div align=\\\"center\\\"][Image: pi-chart1.png][/div]

Important to remember these numbers only include the new creates who have completed the Rookie Survey. 46 Rookie Surveys have been submitted between March 27th and April 24th. Subtracting the number of remakes from this pool, leaves you with a possible 67, meaning 69% (nice) have filled out their survey. Thus, out of the data we have to draw from, the actual percentages look more like this:

[div align=\\\"center\\\"][Image: pi-chart2.png][/div]

There’s a lot going on in these two charts, so I’m going to do my best to analyze the information here as best I can. I’m sure those of you in the comments will have more to say on the matter, but I’ll give my takes here.

The biggest thing I notice when looking at these two charts is how much the second one highlights the need for a much higher return on Rookie Surveys. For 38.1% of my class we have no idea whether or not they joined any of our league discords. This is a huge issue for me, because as I’ve already outlined above, as a league we rely on the Discord for new players to connect with Rookie Mentors to answer any questions for them. Yet here is over a third of the class unaccounted for in regards to that. Further reasoning to me that we need to reconsider both our placement of the Rookie Survey and how we handle it.

The next thing that jumps out to me is 82.6% of users that filled out the Rookie Survey who joined the League Discord. This I think is actually a really strong rate. I’d argue that those who currently fill out the Survey, are those with the most likely chance of keeping with the league over a longer period of time and I think the league should celebrate so many joining the Discord. Good job by us!

The last thing I’ll mention from these graphs, and perhaps the most important, is the numbers involving the Rookie Discord. I think ideally for the league, the Rookie Discord would be a place where new players could join, and be immediately met with help, feedback, suggestions, etc. However we see from the first graph, that only 52.2% of new creates who filled out the Survey opted to join that Discord, compared to the 82.6% who joined the League Discord . This is a problem in my opinion, and illustrates how the Rookie Discord may be being used ineffectively.

As of this writing the Rookie Discord has approximately 235 members; a number that pales in comparison to the roughly 746 members the NSFL Discord boasts. As I mentioned earlier connecting with a Rookie Mentor in Jay, was vital to my staying in the league. That said, how much easier is it for a new create to blend into all of the noise of the NSFL discord, and potentially get lost in the mass of conversation that takes place there. I think it is really important for the league's sustained growth to look into how we are using the Rookie Discord currently, and how we might better utilize it in the future.

On the subject of utilizing the Rookie Discord better, allow me again to direct your attention to the Rookie Guide. Much like the Rookie Tasks and Rookie Survey there is no mention of the Rookie Discord anywhere in the post. For that matter, there is no mention of the League Discord anywhere either. The only place you will find the League Discord mentioned is in the scrolling text bar, towards the top of the screen. A text bar which currently takes about a minute to scroll through, before the Discord link is displayed again (57.59 seconds in my testing…. Your results may vary.) This to me is a huge oversight as most new players are more likely to have their attention snapped up by the large red text at the top of the post, rather than the scrolling bar. I’ll suggest later in this piece, addressing the lack of Discord links, but for now suffice to say I think it an issue.

The final thing I want to look at is the Rookie Tasks as a whole. As mentioned earlier in the piece, Rookie Tasks are designed to be one of the first things a new create does. A way for them to immediately begin boosting their player through both TPE and financial earnings. However, when I looked to see who of my class has done them, I was excited to see that only 40 out of the possible 67 (60%) people in our class have begun them in some capacity. Mind you, that does not mean all 40 have completed the tasks, but have begun working on them and posted one or more in the Rookie Tasks forum. Digging into those 40 posts, the most common task left undone by new players is Task 4 which reads: Make a media entry or Graphics post. For point of reference, 12 of the 40 (30%) Rookie Tasks submitted did not initially include Task 4, with only 5 players returning to the thread to post their Task 4 at a later date.

As a new player to the league, and sim leagues in general, I completely understand why Task 4 is so difficult for new players. It’s scary putting something you’ve created out there in front of the entire league to look at and judge you by. Sitting here writing this now, I fondly look back at the first podcast I recorded and remember having completed it sitting in front of my computer for 20 minutes debating if I was actually going to post it or not. That initial barrier to sharing is a huge challenge for new players, and something that I think we should consider as a league. How better can we serve our new members, to allow them to feel more comfortable sharing? I’m going to suggest a few thoughts I’ve had below, but I’m interested to see what you all might think in the comments.

[div align=\\\"center\\\"]Syl's Suggestions[/div]

I’m calling these “Syl’s Suggestions” intentionally, because that’s all they are… suggestions. I think it important to reiterate here that HO does an amazing job running this league we’ve all come to be a part of, and perhaps at times it’s easy for us to take that for granted. My intentions here are not to point out how poorly things are being done, but rather to notice some potential inefficiencies around the site that I noticed throughout my time here as a completely new player. Inefficiencies, that I would argue, are bound to happen with anything over time as the league grows and reforms. If you are sitting there reading this now, then know this article was vetted by HO, prior to my posting, with an offer to just bin it should they find it not to their liking. My intention is not to potentially cause more harm to the league, but to help move the natural growing process along…. And help patch up any holes I can along the way. So with that disclaimer beat into the ground, let's get on with the suggestions.
  • I think the Rookie Guide needs some consideration, and potentially a complete overhaul. The information there is fantastic, but there’s so much there…. Especially the further you scroll, that it can be a bit much all at once. I’d like to see the league trim out some of the fat, and maybe have links to some of the later posts there…. Rather than telling a user how they retire, regress, or make money when they haven’t even made a player yet.
  • Also, I cannot stress, if you take nothing else from this albatross of an article, pleeeeeease have a Discord link somewhere in there. Preferably right at the top of the whole thing in large, bold lettering. This league’s lifeblood is it’s Discord servers and locker rooms. To me it’s vital we get potential new members on to them as soon as possible. Also, it’s harder for someone to go inactive when they’re getting messaged by people or pinged with announcements of games, drafts, events, etc. Ideally it would be two links one for the NSFL Discord and one for the Rookie Discord… but if that’s too much at least have one in there.
  • I would strongly suggest putting the Rookie Survey at the end of the Character Creation bit. Perhaps that means we need to alter the survey slightly (I think it’s fine as is), but I really think it’s VITAL we get that information from users as soon as possible. As I pointed out earlier only 69% of my class filled it out, which is good feedback, but still a number I think we can improve upon. At the very least, I’d suggest moving the survey to the Rookie Guide, and seeing if we get an uptick in responses. Some may argue that’s too early for feedback, but I’d rather get it early then not at all.
  • Rounding out my suggestions on the Rookie Guide, I’d suggest we also include a link at the end of it to the Rookie Tasks forum, to allow new users to begin working on their player immediately. Player creation is exciting, but when I created mine I had no idea what to do next; it took Jay_Doctor about 8 hours to reach out to me by which point some of the excitement of player creation had begun to wear off. Give those new players something productive to look at and do! Maintain that excitement level, until they get in touch with one of those Rookie Mentors.
  • Alongside the Rookie Guide, the Rookie Discord needs to be better utilized in my opinion. At the very least, have it mentioned in the Rookie Guide somewhere that is obnoxiously in the face of new players. Also, when someone joins for the first time maybe consider having a link there, that will redirect them to a sort of “new player checklist.” Ideally this list would have links to the various things a new player should be doing…. Rookie Tasks, Survey, Free Weekly Training, Activity Checks, etc.
  • Addressing Task 4 of the Rookie Tasks is a challenge. Obviously, we don’t want to provide a way for players to take advantage of the system, but at the same time I think lowering the barrier of entry for new players could be in order. Perhaps, instead of “Make a media entry or Graphics post” it could be “Have a current player interview as a new player to the league or a graphics post.” Doing something like this, would take less of the onus off the new user and allow for some of our more vetran members to come alongside them in more of a “mentoring relationship.” This has the dual benefit of giving them a friendly face in the league, while also tearing down their barrier to entry on the media side of things. I realize the question of payment, may arise…. I think that’s something LO could work out ahead of time: “New Users make X the interviewing veteran makes Y, minimum of Z words or questions.” I could see something like that being really effective, and might encourage some of our non-Rookie Mentors to get involved.
  • The final suggestion I’ll mention, isn’t something I’ve addressed prior to now, but rather an idea I’ve had developing that could work in tandem with my last point. We recently just wrapped up our 1.5x media bonus for the NSFL Rookie draft, sadly however the Bonus ended prior to the DSFL draft. This resulted in very few DSFL draft articles, as most players had already made their money in the weeks before. I appreciate that mocking the DSFL draft may not be the best idea, because if a player “slides” in the draft they may get upset. That said, nothing is more exciting than seeing your name in someone else's media. With that understanding, I’d petition to adjust the 1.5x Media Bonus to have a few less days up front, and a few more at then end. Maybe you don’t encourage mock drafts for the DSFL, but instead apply the bonus to Rookie player profile pieces, Rookie pressers, and who knows…. Rivalry pieces or something? Idk, yall are more creative then me. I trust we’d figure something out that would be a nice middle ground, and encourage people to write about the next generation of NSFL players.
Alright, that’s enough writing from me….. Seriously. I promised myself I wasn’t doing any written media, and now here I am with a 3,300 word monstrosity. I’ll end by stressing again, I’ve really enjoyed my time in the league this far and am super excited to see it continue to grow. I hope this piece might encourage some changes within the league to make it slightly more new user friendly, while also getting more people involved in the discussion. I look forward to any comments, and if you made it this far….. Treat yourself to something nice Wink

~Syl



Notes for the Grader:
  • 3,312 word count per the site
  • Thanks for being such an awesome person Wink


*Improving the New User Experience - Billybolo53 - 04-28-2020

I love the idea of streamlining the rookie guide. You’re completely right, it gives you all the info when you just need enough to make a player and get up and running

*Improving the New User Experience - TeyonSchavari - 04-28-2020

Seeing this a few months after I attempted to overhaul the Rookie Guide hoping to make it better and easier to understand **cries**

Seriously though, thank you for the suggestions. I think a lot of these would be great to include in the rookie guide and I can't believe I didn't think of them before. Sometimes all it takes is a fresh pair of new eyes to see more ways to improve things Smile

EDIT: In fact, I added link to the league discord at the top right now. Not linking the rookie discord since we want to make sure that only rookies can join it

*Improving the New User Experience - ADwyer87 - 04-28-2020

awesome to see this kind of analysis from a new guy Smile awesome work Sylph!

*Improving the New User Experience - Jay_Doctor - 04-28-2020

You've brought up a lot of great ideas here that I would like to see implemented. One thing to note is every player who gets approved does get this message on their player page:

Everything looks good so you are approved, welcome to the NSFL! We invite you to join our main NSFL Discord, if you haven't already.
Because you joined after the DSFL Draft, but before the trade deadline, you will be placed on waivers for the current DSFL season and will clear within the next 48 hours. Next offseason you will be entered into the NSFL draft.
The next thing you should do - if you haven't already - is create an update page in this forum. Once you do that, there are a few things you can do to get started on your player:
1. You can find this week's activity check in this forum. You can post there each week for 2 free TPE.
2. Each new member gets one free training to start off with, worth 5 TPE (recreates are not eligible for this free training). After that, as an unassigned player you can pay for weekly trainings in this thread, and when you are picked up by a team you can post your payments in the team thread found in this forum.
3. You can find the current point tasks in this forum. Most of these will be predictions, short written responses to a prompt, or trivia questions about the league. NOTE: You can only claim TPE for tasks started AND completed AFTER your player has been created. If this is your first player, you can also complete the Rookie Point Task for a nice 12 TPE, $3,000,000 boost to start you off on the right foot.
4. A few days after the draft, you can check this forum to find this season's DSFL training camp thread, which is worth 10 TPE.
5. We encourage you to create a wiki article about your player. Don't be afraid to add background information such as your player's personal history. Have fun with it! You can do that here, and you'll be awarded 10 TPE.
6. Pick a "likeness" or "render" - a person that you want your player to look like. You can post your choice in the given format here.
If you have any questions, feel free to reach out to any member of the rookie mentor team (BaconBrother, FleshBagSoup, IsaStarcrossed, Jay_Doctor, ScorpXCracker, TeyonSchavari, TomHanks, and woelker11), or to your team’s GMs (once you have a team).



It does have a couple things you mentioned, but I don't know how effective it is. It has the rookie task guide listed down at 3, although it should probably be the first sentence or two. It was the only way I figured out what I was doing as a new player at the time.

*Improving the New User Experience - Memento Mori - 04-28-2020

Love that you wrote this. Lots of good ideas in here.

To be honest, I think a 69% response rate to a survey and the retention rates we have are quite good. But there's things we can do to improve, and you've hit on a lot of them here.

*Improving the New User Experience - iseedoug - 04-28-2020

Great analysis. I hope to see you continuing to be active in the league!

*Improving the New User Experience - caltroit_red_flames - 04-28-2020

Hey I'm actually about to start working on a player creator!

*Improving the New User Experience - nunccoepi - 04-28-2020

This is awesome. Great work!