Pages

Monday, June 27, 2016

The Great Survey Experiment, Part One

In which I learn some interesting lessons about leadership online


I have to admit, I spend entirely too much time on Reddit.  If you've never been to Reddit, I highly suggest you simply read my explanation and not click the link, if you ever want to spend time not on the Internet again.

Simply put, Reddit is a website, divided into pages called subreddits, wherein people who have similar interests share information and ideas and have discussions on a specific shared interest.  There is a subreddit for everything - from Toastmasters, to jokes, to cats, to the dark sides of the Internet that most of us don't want to see.

Out of the roughly 883,000 subreddits, one of my favorites is dedicated to the concept of financial independence - reaching the point where you no longer need to work for money, whatever that point is for your lifestyle.  This subreddit is an active Internet community - its 132,000+ members have daily discussions, share their financial plans, and disseminate various strategies for reaching financial independence.  And although they all share a common goal, its members come from varying backgrounds, have a multitude of careers and hobbies, and see the world in different ways.

One day, someone decided it would be neat to survey the members of the financial independence subreddit.  A brief survey was created, but quickly shut down after a wave of feedback indicating that it was missing critical information.  Then, a call for help to create a new and improved survey went out.

Having done quite a few surveys for my day job, I answered the call for help and shortly found myself in discussions with the few others who had volunteered.  Among that group, a call for a leader was issued - which I again answered.  After all, I figured, I know a little about surveys, but I know more about being in charge and getting things done.


I found myself leading a team of four volunteer survey writers.  I quickly developed a plan for the project and posted it for feedback (I'm big on planning).  Then the writing began in earnest.

Throughout the survey development process, I got a good look at what its like to lead a remote team - something I've not done before, and am happy to have done in an environment where there is no meaningful consequence (if things went awry with this project, I wasn't going to lose my job).  The worst that could happen is that some random person on the Internet might be unhappy with me.  In the grand scheme of things, I'm not too worried about that possibility. 

The first challenge was finding a medium to draft the survey questions.  Because Reddit is styled after an Internet forum, with chains of discussions leading off one main post, we began with various posts to discuss questions.  The multiple chains however, were too divided and didn't lend themselves to meaningful discussions.

Instead, we moved to drafting the survey in an online spreadsheet via Google Sheets.  There, each team member was able to draft survey questions, add their comments on previously written questions, and outline options for responses.  The Google Sheet was supplemented by discussions on Reddit.  This worked better than simple discussions on Reddit, but still wasn't quite ideal.  It was too easy to miss a post on Reddit or a change on the Sheet.

Once a majority of the questions were drafted in the Google Sheet, I began to create the actual survey.  Because we were all volunteers and all share the common goal of reaching financial independence which means we don't "waste" money, we were limited to survey tools we could use without cost.  Fortunately, I'm able to use a company SurveyMonkey account.  Unfortunately, I wasn't able to share the login information with anyone else on the team.  This meant I had to create the questions in SurveyMonkey, and then distribute the draft survey to the team for feedback.

Over about six weeks, I fed the survey questions from the Google Sheet into SurveyMonkey and got feedback from the drafting team on the questions.  Once they were in the SurveyMonkey, it seemed much easier for the team to have meaningful discussions on the actual wording and answer options.  Discussions tended to be short, however, with most being feedback being directed to me rather than occurring as conversations between the team members.

Finally, it was time to beta test.  I put out a call for help - and about two dozen volunteers answered.  Each volunteer took a test version of the survey and submitted feedback on the specific questions.  Almost all of the feedback was incorporated into the final survey, although some comments conflicted and thus cancelled each other out.

What I've learned about leading an online team, so far, follows.

First, it's critical to have a tool that enables group communication between all team members.  The information needs to flow in a circle, not up a triangle.  In hindsight, using something like Slack or TeamWork may have been much more fitting for this project.  Our Reddit discussions and Google Sheet were just too limited to really enable a high level of discussion. 

Second, setting milestone dates is crucial.  We had a plan and a path to follow, but because this was a volunteer project with no real deadline, I didn't set milestone dates.  Looking back, milestone dates may have helped facilitate discussion by getting the full team on the same piece of the project at the same time.

Third, when working with a large audience, you can't make everyone happy.  Or even simpler, you can't make it work for everyone.  We are surveying a diverse audience, that wants a lot of information (the survey is a whopping 65 questions).  Logistical questions, like how to get that information, the best way to phrase the questions, and the best way to format the input all have so many possibilities that someone has to make the call and just do it - or else it will never get done.

The great survey experiment has been a learning experience for me, in leading remote teams, using online collaboration tools, and coordinating people who are complete strangers.  The survey results aren't in yet, but I'm sure I'll be posting more lessons when they are. 


No comments:

Post a Comment