Blockchain DevEx Survey Results & Quality of Life Action Items

[will-corcoran]

Introduction

Recently, a 10 question survey was circulated with contributors to the stacks-blockchain repo. The survey was formatted into 10 (intentionally) open-ended questions. Each question was meant to focus on one facet of the developer experience journey. We received 18 responses.

If you are interested you can review the raw, unfiltered feedback here.

A summary by question can be found below.

44 action items resulting from the survey have been identified and added as draft issues to the Stacks Core Eng project board. The Critical Bounty Testing Team ( @sergey-shandar @CAGS295 @bitfari @moodmosaic ) will be diving into these draft issues and determining which ones should be converted into a proper issue and prioritized.

01 Overall Experience

Original Question:

What has been your overall experience as a developer contributing to the Stacks blockchain?
Please share both positive and negative aspects.

Summary Statement | Positive End of the Spectrum:

Developers appreciate the strong community and passionate individuals involved in the Stacks blockchain project. The use of GitHub for collaboration and thorough testing with CI are highlighted as positive aspects. The open-source nature and the potential for innovative technologies contribute to a positive experience.

Summary Statement | Negative End of the Spectrum:

On the negative side, developers often find the codebase complex and difficult to navigate, with challenges in understanding code patterns and organizing contributions. Slow test feedback, unclear organization, lack of high-level overviews, and a lack of up-to-date documentation also hinder the development experience. Challenges related to deployment, configuration, and running a local instance further contribute to a less smooth experience.

02 Onboarding Experience

Original Question:

How would you describe the onboarding process for new contributors to the Stacks blockchain? 
What aspects could be improved to make it more user-friendly?

Summary Statement | Positive End of the Spectrum:

Contributors who had a positive onboarding experience appreciate the helpfulness of mentors and experienced team members, who provide guidance and assign tasks to newcomers. The existence of some resources like documentation and tools has been beneficial, and efforts to establish a more structured onboarding process are recognized as positive steps.

Summary Statement | Negative End of the Spectrum:

Developers who found the onboarding process challenging express frustration due to the lack of clear and comprehensive documentation, as well as the complexity of the codebase. The absence of a centralized source for getting started and understanding the project's architecture hinders effective onboarding. The experience of new contributors is sometimes marred by confusion around organizational structure and a lack of guidance.

03 Friction

Original Question:

What barriers or obstacles have prevented you from contributing to the blockchain codebase?

Summary Statement | Positive End of the Spectrum:

Contributors on the positive end of the spectrum emphasize that their primary barrier to contributing has been related to professional constraints, such as time availability due to their existing contracts or roles. These individuals generally do not see significant obstacles that prevent them from contributing and may be more willing to participate if they have a suitable opportunity.

Summary Statement | Negative End of the Spectrum:

Developers who are less satisfied with their experience or face more significant barriers highlight multiple challenges. These include the complex and intimidating nature of the codebase, lack of clear documentation, unclear priorities and methodologies for contributions, issues with running tests and Cargo, ambiguity in project structure, and concerns about how proposed changes might be received by the community.

04 Complexity & Maintainability

Original Question:

What are your thoughts on the complexity and maintainability of the Stacks blockchain codebase?

Summary Statement | Positive End of the Spectrum:

Contributors on the positive end of the spectrum view the complexity and maintainability of the Stacks blockchain codebase as manageable, with a recognition that while certain areas may present challenges, the overall organization and self-explanatory nature of most parts contribute to a reasonable experience.

Summary Statement | Negative End of the Spectrum:

Developers who express concerns at the negative end of the spectrum find the codebase highly complex and lacking in maintainability. They often cite issues related to code organization, absence of abstraction, interdependencies, insufficient documentation, and difficulties in understanding certain parts of the codebase.

05 Confidence in Testing

Original Question:

When making contributions to consensus-breaking changes, how confident are you in the adequacy of the testing conducted for those changes?

Summary Statement | Positive End of the Spectrum:

Contributors on the positive end of the spectrum generally express confidence in the adequacy of testing for consensus-breaking changes. They believe that the project's testing efforts, although not perfect, have caught many issues and have improved over time.

Summary Statement | Negative End of the Spectrum:

Developers on the negative end of the spectrum often lack confidence in the testing conducted for consensus-breaking changes. They highlight concerns related to testing consistency, integration, and the ability to catch failure modes. Some also mention instances where bugs were not adequately captured by the existing testing mechanisms.

06 Clarity of Testing

Original Question:

In terms of testing, do you find it clear to understand why a test has failed when running it?

Summary Statement | Positive End of the Spectrum:

On the positive end, contributors generally find it clear to understand why a test has failed when running it. They attribute this clarity to their familiarity with the testing environment, debugging tools, and their ability to navigate the codebase effectively.

Summary Statement | Negative End of the Spectrum:

Developers on the negative end of the spectrum often struggle to understand why a test has failed. They highlight challenges such as insufficient error messages, unclear logs, and flaky tests that fail for reasons unrelated to the change being tested. These factors can lead to confusion and make debugging time-consuming.

07 Clarity of Code Patterns

Original Question:

How much time did you spend reviewing code written by others in order to ensure the correctness of your contribution?

Summary Statement | Positive End of the Spectrum:

At the positive end of the spectrum, contributors express that they invest a substantial amount of time reviewing code written by others to ensure the correctness of their contributions. This proactive approach to code review demonstrates their commitment to maintaining high code quality and ensuring that their changes align with the existing codebase.

Summary Statement | Negative End of the Spectrum:

On the negative end, some contributors mention that they haven't had the opportunity to review code written by others or that they spend relatively little time on this aspect. This lack of emphasis on code review could potentially lead to challenges related to code quality and integration.

08 Documentation

Original Question:

How helpful and comprehensive do you find the existing documentation and resources for developers working on the Stacks blockchain?
Are there any specific areas that you believe need improvement or additional details?

Summary Statement | Positive End of the Spectrum:

Contributors at the positive end of the spectrum find the existing documentation and resources for developers working on the Stacks blockchain to be helpful and comprehensive. They appreciate tools like stacks.js and acknowledge that while there may be areas for improvement, overall, the documentation aids them in their development efforts.

Summary Statement | Negative End of the Spectrum:

At the negative end, some contributors express dissatisfaction with the existing documentation and resources. They highlight challenges related to outdated information, lack of comprehensive technical documentation, and difficulties in finding relevant documentation. The lack of documentation for specific modules or components of the codebase is also mentioned as a concern.

09 Deployment

Original Question:

What is your experience with the testing and deployment processes in the Stacks blockchain development workflow?
Are there any pain points or suggestions related to testing and deployment that you'd like to highlight?

Summary Statement | Positive End of the Spectrum:

Contributors at the positive end of the spectrum generally find the testing and deployment processes in the Stacks blockchain development workflow manageable. They appreciate the ongoing efforts to streamline the deployment process and work towards improving integration tests and deployment pipelines.

Summary Statement | Negative End of the Spectrum:

On the negative side, contributors highlight pain points related to deployment complexity, slow tests, crashes, and the lack of clear documentation for certain processes. There's frustration expressed around the difficulty of setting up local test environments, dealing with integration test failures, and the need for clearer guidance on troubleshooting issues.

10 Future Enhancements

Original Question:

From your perspective, what are the most promising aspects of the future development and growth of the Stacks blockchain?
What would you like to see in terms of future updates or enhancements to the Stacks blockchain?

Summary Statement | Positive End of the Spectrum:

Contributors on the positive end of the spectrum are excited about the future development and growth of the Stacks blockchain, particularly with upcoming releases like the Nakamoto release and the integration of sBTC. They appreciate the engagement of developers in the Stacks ecosystem and the focus on robustness, modularity, and code quality. There's a desire to see the ecosystem mature, with improvements in error handling and DeFi offerings.

Summary Statement | Negative End of the Spectrum:

On the negative side, contributors express concerns about certain core changes being shut down, which they believe could hinder progress and improvements in the codebase. Some contributors also mention a lack of trustless peg-in/peg-out mechanisms, making it less competitive compared to other blockchain solutions.

cc: @jcnelson @kantai @obycode @netrome @muneeb-ali @saralab @cylewitruk @CharlieC3 @mhb11 @renashah