Project feedback

[bradmontgomery]

I've recently pulled down this project and attempted to get it (with openPDS) running. A couple of things struck me as surprising and/or unexpected:

• It's unusual for a django project to be distributed (i.e. with a whole settings.py file, ). Typically I'd expect a django app or a collection of apps.
• There's a number of discrepancies between the instructions in the top-level README (which work) vs the information in doc/INSTALL.rst. Also, the additional fabfile.py and bootstrap.sh in the root seem either out of date or out of place (again, these are project-level things I'd not expect to see distributed in a reusable app).
• Your installation instructions tightly couple the project with a virtualenv. These two things should be separated. I should be able to completely remove a virtualenv (or switch to another one) completely separate from my django project.
• There appear to be no unit tests at all (perhaps that should be a separate issue).

A few things I'd typically expect from a reusable django app are:

• Documentation for any app-specific settings (not a settings file)
• A requirements.txt in the root of the project. Yours seems to be buried with some deployment-related config files (again something I wouldn't expect here, since I may choose to deploy my project differently).
• A setuptools-compatible setup.py that allowed me to install with: pip install openPDS-RegistryServer
• Some level of tests, runnable with: python manage.py test registryServer

If the goal of this project is to be something that fits within the django ecosystem, the barrier to usage and contributions would be lower if it followed some of the typical community conventions. Some of this information is now available in the django docs, as well.

I hope this is helpful feedback!

[brian717]

Hi Brad,

Thanks for the feedback. Lots of good points in there. Some that should probably have their own issues filed. There are a few that I can address, and others that we're actively working to resolve:

• The doc folder is out of date. Sorry about that. I'll go ahead and file a separate issue about that, as this issue addresses more than one thing.
• Instructions naming a virtualenv: this is not ideal. How do you think we can update the instructions to remedy this? I'd like to avoid removing the steps to create a virtual environment, as it's not something that everyone has experience with (I don't think removing it, or replacing it with "create a virtual environment" would work).
• Unit tests ARE a separate issue, and a spot where the project could use some love. We'd love the help, if you're interested in contributing to this or know someone who is.
• We can move requirements.txt to the root of the project. I'll file a separate ticket for that.
• setuptools-compatible setup.py is a good idea. As this isn't quite a bug, I'll again solicit for contributions to the project towards this goal.

Finally, a clarification on Django (+ the ecosystem), and its role in openPDS:

Django is a means to meet an end here: no work has gone into fitting openPDS into the larger Django ecosystem. The goal was for someone to clone a repo and have a registry / PDS server up and running in a few instructions. Django was chosen due to its simplicity and prior knowledge on the initial team. In fact, more thought and work has gone into making it easier to use / deploy without prior Django experience than into making it behave and look more like a typical Django app; the target audience for openPDS up until this point has primarily been data scientists, rather than Django developers. We try to use as few Django-isms as possible so that users don't need to learn too much about Django to use the project, and so applications written on the framework can be ported between implementations easily (a bit of work has been done in Node.js, as well as quite a bit of work in Java). We're not attached to Django, and our attachment to Python is only a consequence of its popularity among data scientists.

All that being said, we're definitely interested in making openPDS and the corresponding registry server more "standard" in ways that won't require users to know anything more about Django than they currently do (ideally, they wouldn't need to know anything about Django at all, actually).

Thanks again for the issue / feedback. I'm going to go ahead and file the other issues I mentioned above separately.

[bradmontgomery]

Thanks for the quick response! Your comments on the target audience for this project clarifies a lot of things for me.

Here's a rough idea on how you might go about distributing this project in the future:

• Take the registryServer core (the django parts), and make them as much of a reusable app as possible. You could do the same with openPDS, though it could also remain a separate repo.
• Make that project pip-installable (with a setuptools-compatible setup.py).
• Create a repo for the project-level stuff that you want to distribute (e.g. openpds-installer?), and that could reference both the registry server and openpds in its requirements.txt file.

You might consider taking a look at Pinax, which essentially gives you tools to quickly put together a project based on many django apps. You can see the README in the pinax-project-symposion repo for an example of this.

I realize that's likely a long-term goal and would require further discussion, but it's one idea.

I still think, that it'd be worthwhile to follow as many Django conventions as possible, at lease within the core of the project. There's a huge community of people that consistently contribute to open source. Lowering the barrier for contributions is a Good Thing :) And there's no reason why we couldn't hide some of those django-isms from end-users if there were project installer similar to pinax.

Hope some of that helps!

[RogerTangos]

Hey Brad,

+1 to everything Brian just said. Thank you again for the detailed and helpful feedback.

Al