Revise Django / Wagtail project template approach

[hancush]

Background

Spin off from #318, #278

We have templates for Django and Wagtail projects, but the scaffolding is largely the same, which means we have two places to update code and configs as we refine our stack. We don't always do this, e.g., the Wagtail template is still using Django Compressor instead of WebPack. Moreover, boilerplate code means that our templates must be updated to account for breaking changes when Django / Wagtail have major version releases, and we have not found an easy way to keep other dependencies up to date. Finally, using our cookiecutters isn't very ergonomic, because you have to clone and update this repository, and darn it, I can never find them on first try, even though I'm the one that originally put them in the docker/templates/ directory.

Proposal

I propose three things:

• Relocate DataMade's cookiecutters to their own GitHub repositories so they can be used directly from GitHub, no updates required: https://cookiecutter.readthedocs.io/en/stable/usage.html#works-directly-with-git-and-hg-mercurial-repos-too
• Revise the Django and Wagtail templates to do away with boilerplate Python, relying instead on the startproject command that comes out of the box with Django and is extended by Wagtail
• Extending the work from Create composable Django project template #318 (comment), create a unified Django / Wagtail cookiecutter with a post_gen_project_hook that:
  • Installs Python and JavaScript dependencies, and pins versions
  • Runsstartproject
  • Copies a default settings file from the template to the generated project

Deliverables

I will rehome the vanilla JavaScript cookiecutter to its own repositories. I will then create a combined Django / Wagtail cookiecutter following the proposal above. All other cookiecutters will be retired.

Timeline

I expect this to take a day or two.

[hancush]

Given a list of dependencies, npm can install them and create a package.json and package-lock.json pinning their versions, e.g.,

npm i react react-dom react-leaflet

[hancush]

Started this here: https://github.com/datamade/cookiecutter-django-app

[xmedr]

@hancush let me know whenever this django app is ready to go and i can start automatically opening prs for dependency updates on this instead of the how-to repo! no rush of course

[hancush]

@xmedr That shouldn't be necessary, actually!

The new cookiecutter includes a list of core JavaScript dependencies (negotiable what makes the list).

After the new project is generated, Cookiecutter will npm install those packages, insodoing finding the best combination of package versions ("ideal tree") and pinning everything in package.json.

It also pins the latest version of Django or Wagtail, depending on which framework is installed. If we want a default list of Python dependencies, e.g., for testing, we'll take a similar approach as described above for JavaScript deps.

[fgregg]

looks like a real improvement to what we are doing now!

this approach to dependencies also won't make any guarantees that the code the cookiecutter outputs is in a non-broken state. that would be be mitigated by having a nightly scheduled job to test test if the code still works with the latest dependencies and a commitment to adjusting the code when it fails.