The Feedback Loop: evaluating choices, not checklists

Preface: The second round of the Feedback Loop is under way. Stay tuned for future rounds!

Recently, feedback has been a popular (and at times, hotly debated) discussion topic in the data viz community. Ben Jones (@dataRemixed) wrote a post to encourage us to think about our intentions, when we give feedback, and how we give feedback.  Anna Foard (@stats_ninja) retweeted a Harvard Business Review article with similar sentiments that started a healthy threat discussing public feedback. Criticism and feedback are planned discussion points between Steve Wexler (@VizBizWiz), Jeffrey Shaffer (@HighVizAbility), and Andy Cotgreave (@acotgreave) during the next episode of Chart Chat. Lea Pica (@LeaPica) spoke about feedback on a podcast and wrote an article on the 6 steps to powerful criticism. Nathan Yau (@flowingdata) wrote a post responding to the many opinions out there.

In the background, a group of us have been running an experiment. We've decided to call ourselves "The Feedback Loop".

First, let's back up.

Creative arts workshops

In college and grad school, I took a ton of creative arts classes (enough for multiple majors if I'd focused course selection), ranging from poetry to photography to dance. In these courses I was introduced to a common paradigm for criticism and feedback, "workshops". All of us, as artists, would provide peer feedback. Feedback was focused on identifying things like voice, style, tone, and identifying how we could really "turn the volume up" on those features that were making our work succeed.

These groups were helpful because we were receiving aesthetic feedback. It wasn't about checklists and rules. We talked about the rules, like grammar for writing and appropriate focus for photography, but we were encouraged to break the rules if it brought meaning to our piece (I wrote another piece on this idea specifically).

We weren't evaluating checklists, we were evaluating choices. Everything that artist did (whether in line with "best practices" or not) was seen as a choice. What did that choice accomplish? How did that impact voice, tone, style? How did it impact the message? Nothing was wrong, and everything had an impact.

As a result, these workshops were full of interesting questions. People were less likely to say, "This syntax was wrong", but instead were more likely to say "Your syntax here is an interesting choice. It left me confused and I really had to stop and think through what you were trying to say." It was then up to the artist to consider whether that was the impact they really hoped to have -- maybe they wanted to use a bizarre sentence structure for a particular aesthetic goal, like Faulkner often did.

And although the courses were lead by an instructor, feedback came from a room full of equal peers - providing a lot more diversity of thought than expert feedback could offer.

My first "creative vizzing workshop"

A few years ago, as the lead of a data visualization consulting practice, I decided to test the creative arts paradigm in data viz. We got roughly 15 people to sign up (a little high for a workshop but not too many). We'd meet once a month over the course of 3 months, all putting together some visualization, and then we would follow the creative arts workshop format to provide more aesthetic feedback. 

It was a wild success. I saw massive transformations. We went from building dashboards that looked like any out-of-the-box / show-me vizzes to things with personal flair, storytelling styles that were unique -- works of art that had voice, style, and tone.

My second "creative vizzing workshop": The Feedback Loop

With the surge of feedback hitting our community, I thought I might try extending the experiment, so I created an online creative vizzing group. We got 10 people to sign up, a perfect size for a workshop (listed in the order they signed up): 
  • Neil Richards (@theneilrichards) 
  • Robert Janezic (@RobertJanezic)
  • Marian Eerens (@M_Eerens)
  • Christina Gorga (@Poulincogsci)
  • Will Strouse (@dataNOTdoctrine)
  • Vincent Baumel (@quantum_relic)
  • Michelle Gaudette (@learnvizwithme)
  • Katelyn Schaub (@schaubkatelyn)
  • Erik Rettman (@VizAllDay, he also participated in the first group!)
  • yours truly

 Together, we worked through the technology (we decided to use Slack), and we also decided on a name: The Feedback Loop. We each submitted one viz, and then had a month to provide feedback. Feedback was aimed to be constructive, and we followed the format I outlined in my introductory post. We also partnered up to provide feedback on giving feedback.

Did it work?

First, I'll say that some of these folks have released things to the public incorporating the feedback they received -- and these things have been well received. We even created an informal channel, so people could submit other things they were working on for quick feedback -- and many of those things were released, too. So on that front, I'll call it a success. 

But, I'll let the group talk with a few quotes I'll share: 

"Accepting feedback graciously is a skill, and like any other skill it needs to be developed. It's a natural reaction to defend your creation and shield it from criticism. Through this exercise I've learned to be more accepting of different points of view and to look at my own work more objectively. The feedback in this group has helped me to identify specifics about my style that I hadn't noticed before. Knowing what my tendencies are in terms of both strengths and weaknesses provides me a direction to grow." ~ Erik Rettman

"Effective feedback is a two sided equation. Just like a professional sports team needs to develop itself on both offense and defense, we need to be skilled in accepting feedback (what to iterate on, what changes to make, what not to change, etc) as well as giving feedback (visceral, behavioral, ethical, etc). Once we strengthen BOTH skill sets, our ability as analysts and artists can really take off." ~ Vincent Baumel

"For me the most interesting thing so far has been that if you take a group of people with various interests, backgrounds and data viz styles ... the feedback you get from folks will be very diverse and that's what makes it interesting. Getting input on my viz from someone with a similar style would have been easy. Next to getting ‘confirmation’ of what works well, this process has also given me some additional ideas that I would not have thought about before; and which may or may not end up in the final version and that’s ok too. For me thinking through feedback has been the biggest lesson, I may not agree with it all but can honestly say I took it all in and if it didn’t make sense to me now … it has changed the questions I ask myself when I create something new." ~ Marian Eerens

"I haven't really had a way to calibrate my expectations against the expectations of other data viz experts, so I was never sure if I was hitting the mark or if I was way off. There are some amazing vizzes out there, and some I don’t care for. I haven’t really been able to determine where I fit in that scale, and I don’t yet trust my own judgement.

I think one of the best things about this group has been that I am able to use thoughtful feedback from others to recalibrate my expectations and evaluations of my own work. This feedback was focused on what worked and what could be improved, and that gave me not only constructive feedback, but it has also started building confidence in my own judgment.
I can regurgitate the best practices of design and data viz, but applying them in a way that isn’t just rote is a skill I have not yet mastered. You have to _know the rules to know how to break them_, but to know when you are breaking the rules in a good way takes confidence in your own judgment. I am still working on that part, and this group has pushed me forward quite a bit." ~ Michelle Gaudette

"Focusing on feedback brings in a different element to data visualization. It forces other users to interact with the data, to challenge the setup and design, to ask questions about how things could be improved upon. Sure, there are always opinions included as everyone has their own diverse styles, knowledge, and preferences, but feedback allows constructive criticism, and in turn forces the designer to think about different perspectives, rather than staying in their [design] comfort zone. It's a two way street that helps both sides improve upon their work, whether that be through feedback or design." ~ Katelyn Schaub

So...did it work? I think so. We checked all the boxes I'd hoped for the group: applying a creative paradigm for feedback on creative works, getting feedback from diverse perspectives, a positive focus, and positive comments from participants.

But more than that, we've created a small little community. We've had all sorts of little conversations, where this new paradigm guides how we have discussions about a number of topics. I hadn't planned on this happening -- and I'm surely not shutting down the Slack workspace.

In fact, I'm inviting you. If you're interested, fill out this quick survey. You can read about how it works from the post where I introduced the idea, and you can reach out to me with questions at @data_poetry.

A note on expert feedback

One thing I want to make clear -- I do not mean to discourage expert feedback, or even the idea of checklists. In fact, I strongly encourage it. If Jack Gilbert (my favorite poet), were to offer to critique one of my poems, I'd be such a mix of excited, honored, and humbled that I think I'd implode. As part of #IronQuest, Sarah Bartlett (@sarahlovesdata) and Timothy Vermerian (@TimothyVermeire) provided feedback on my Fan-made, data-drive tribute to Daredevil viz. I admire their work and their perspectives, and appreciated their expert feedback. 

If you're doing #IronQuest, or #MakeoverMonday, or #ProjectHealthViz, or #SportsVizSunday, or any of the other community events where a host provides feedback, please, don't stop. This is valuable. You're getting an opportunity to learn from some of the most respected people in our field. 

The idea of the Feedback Loop, and of creative arts workshops in general, is meant to solve a different goal. It's meant to test the subjective and aesthetic elements against peers. It's meant to provide diversity of perspectives, because what works for one person may not work for another. 

And, what I've learned now, it can create an incredible community of people devoted to helping one another grow as artists.

Popular Posts