Dask User Survey 2022

[jrbourbeau]

It'd be great to continue the Dask user survey we've done the past few years (xref #148, #56)

@TomAugspurger this is something you've historically helped with. Do you have bandwidth / interest for this year? Certainly no obligation though. I'm also happy to help out.

[jrbourbeau]

Here's a preview of the survey for this year (all questions are copied from last year's survey) https://docs.google.com/forms/d/e/1FAIpQLSftouQ1fxfPii1nGyUGh6xa1cjxB_SFpVmMLQm_KyqkP2vaEQ/viewform. What feedback to folks have? Are there questions we should edit, add, or remove? Keep in mind that:

1. Changing a question's wording makes it harder to compare to previous years
2. The survey shouldn't grow too long

Some of my own thoughts:

What Dask resources have you used for support in the last six months?

I’d suggest we add “Dask discourse” as an option and remove the gitter chat, Dask Slack, and GitHub discussions options as we’ve consolidated user questions on discourse and stackoverflow.

Do you use Dask's Prometheus Metrics?

@jacobtomlinson IIRC this was something you were interested in asking. Is this something we should ask again this year?

Which Diagnostic plots are most useful?

Add an “Other” option

Style comment: we can probably come up with a better image to use in the survey header

[jacobtomlinson]

Yeah I'm definitely still interested in this. I wonder if we could add another question along the lines of "Do you integrate Dask with your organisation's monitoring/observability infrastructure?" with options like "Yes", "No" and "I'd like to in the future" and maybe a free form text box where folks can share what they'd like to be able to do.

Style comment: we can probably come up with a better image to use in the survey header

I've pinged the designer that did the website/logo refresh to see if they can put something together.

[LilyMelnyk]

@jacobtomlinson - @mrdanjlund77 and I are also happy to help you all create a better image if you'd like 😄

[jrbourbeau]

I wonder if we could add another question along the lines of "Do you integrate Dask with your organisation's monitoring/observability infrastructure?" with options like "Yes", "No" and "I'd like to in the future" and maybe a free form text box where folks can share what they'd like to be able to do.

Seems like a reasonable question to me. I'd be curious to know both what system/service people who answer "yes" are using as well as those who answer "I'd like to in the future". Also cc @fjetter as I know you're interested in this particular topic

@LilyMelnyk do you know where I can find a high-res version of the header image on the Dask twitter account? https://twitter.com/dask_dev/header_photo

[mrdanjlund77]

Hey @jrbourbeau - I have the files for that image. Do you need a specific size or do you want the native .ai file? Happy to make something specific for


.

[mrdanjlund77]

Some options. I think these are the same size as the current header?

[mrdanjlund77]

Actually they are double size for better resolution, but the right proportions

[jrbourbeau]

Thanks @LilyMelnyk @mrdanjlund77, those look great. I've updated the survey to use one of those header images 👍

[GenevieveBuckley]

I compiled the survey results last year in more or less the same way Tom has done in previous years. It was a pretty fun task, so I'm happy to volunteer to do the same again this year.

If someone new wants to give it a go, that's also great! Like I said, it is pretty fun. It's mostly a matter of copying a previous year's analysis notebook and making some changes to add any new questions, and also talk a bit about any trends or changes happening over time. https://github.com/dask/dask-examples/tree/main/surveys

[mrocklin]

I'd like for us to reconsider a lot of the current questions rather than include them by default. I think that we should ask questions for which the answers would actively inform our behavior.

For example "How is Dask's release frequency?" I think is a good question because it informs an active debate among developers (release more often or less often?) while "If you use a cluster, do you have a need for multiple worker / machine types (e.g. GPU / no GPU, low / high memory) in the same cluster?" is maybe a bad question because it isn't an active debate among developers (as far as I know). If someone wanted to work on this then no one would stop them.

Similarly questions like "Do you use Dask's Prometheus Metrics?" or "Do you use Dask's Performance reports?" seem very focused on one particular feature that doesn't seem to be contentious. If we're allowing ourselves only 40 questions or so (or whatever number makes sense to respect survey-takers) this wouldn't be one of them. We may want to have one bulk question about various features.

Thinking more positively, what are some of the questions that we've had as a group recently? Someone should go through the company meeting notes I think and see what recent points of contention are. We should then hopefully craft new questions that would help us to answer those points of contention.

I say all this not really planning to do any of this work, so feel free to ignore all of this

[jacobtomlinson]

I see two benefits from the survey, getting answers to questions we have now and tracking trends over time.

"If you use a cluster, do you have a need for multiple worker / machine types (e.g. GPU / no GPU, low / high memory) in the same cluster?" is maybe a bad question because it isn't an active debate among developers

That particular question isn't necessarily in active debate. But I see value here around trends over time. If the demand for heterogenous clusters is growing over time that helps me prioritise that work.

"Do you use Dask's Prometheus Metrics?"

Again not contentious but useful to know so we can prioritise work in this area. Although as I said I'd be keen to replace this with something more general about the demand for better observability.

If we're allowing ourselves only 40 questions or so (or whatever number makes sense to respect survey-takers) this wouldn't be one of them

I'm very in favour of reducing questions, I think it will increase the number of responses.

[GenevieveBuckley]

If people are going to make changes, is there a way questions can be ranked according to how valuable/actionable the answer is?

I'm very in favour of reducing questions, I think it will increase the number of responses.

Agree on keeping it stable or reducing them. Two years ago there were 25 questions, and last year there were 43 questions. That's a huge jump. It didn't seem to affect the number of responses we got, but I'd guess we're approaching that limit.

One thing to be mindful of is tracking changes over time. If we cut a question this year then add a question that is similar but not quite the same the year after, that's not a good situation for data analysis.

[mrocklin]

I'd like to see us deviate from our historical questions and rethink things here. What are questions that we've actually faced about our users when developing over the last year? Let's try to answer those rather than questions that we've historically asked. I'd love to see folks respond here with some curiosity that they've felt.

…

On Wed, Aug 17, 2022, 1:14 PM James Bourbeau ***@***.***> wrote: It'd be great to continue the Dask user survey we've done the past few years (xref #148 <#148>, #56 <#56>) @TomAugspurger <https://github.com/TomAugspurger> this is something you've historically helped with. Do you have bandwidth / interest for this year? Certainly no obligation though. I'm also happy to help out. — Reply to this email directly, view it on GitHub <#269>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKZTCK6ZBDCLLDW3FOPLLVZUTYLANCNFSM562PBXGQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[InonS]

Was there no survey in the past two years?

[jacobtomlinson]

There wasn't. We've been talking about a 2024 survey though.