Locally Optimal

Hill climbing in SF

Notes From VanPyDay 2016

| Comments

A one-day conference born from the Vanocuver equivalent of our local SF Python meetup. I had meet one of the organizers (Seb, @elbaschid) during a previous PyconAU and he convinced me to drop by and give a talk.

Really great community (small and friendly!) in a beautiful city. I’ve never had such a friendly conference audience, it was great to have conversations with many people over the course of the day 😊

I’ve tried to write up some useful notes on each of the longer talks, but I very much enjoyed taking part and getting to hear from the diverse backgrounds of all the different speakers and attendees.

There are not yet single videos for each talk, but you can see the entirety of the day at this link for now. If the individual videos become available, I’ll be sure to stick them in the relevant sections below.

Click - Sebastian Vetter

(Video link)

A talk introducing the click library, giving examples of how to use it successfully, and explaining the basic reason it exists.

click: a library for creating beautiful CLIs

Click’s goal is to take a lot of the repetitive/error-prone parts of creating command line interfaces and improve it. Think topics like:

  • Generating help text
  • Input validation
  • Parsing of command line strings into arguments

Comparison of existing options:

  • Optparse is mostly a 2.x option, but deprecated since 3.2
  • Argparse is the replacement, not too shabby.
  • Docopt has some cool features, but imposes some major constraints to get there

Really good docs on “why bother using this library” http://click.pocoo.org/5/why

Seb offered a cookiecutter template for a very basic example of using Click. Nice idea!

Pro-tip: don’t do the uwsgi thing and have your help text take up 500 lines in a terminal. Use subcommands!

Shiny features in Click:

  • Default values can get pulled from environment variables with the envvar kwarg
  • Subcommands can all have distinct sets of arguments
  • Allows passing data across subcommands using @click.pass_context
  • Default help text comes from docstrings on the methods you’re annotating as commands
  • Main command helptext is autogenerated, including all subcommands
  • Bash autocomplete

Resources:

Making the most out of code reviews - Mariatta Wijaya

(Video link)

Some discussion of bad code reviews with lots of ideas for improving them and making it a healthy, valuable interaction.

Code reviews are great in theory, but can be pretty painful and toxic in practice! Code review should be valuable, not a negative, argumentative system.

Things code review is not:

  • A place to enforce your own preferences on other people
  • A platform for grandstanding
  • Somewhere to belittle devs who know less than you

But it can be…

  • A conversation about how to write great code
  • A place to learn new techniques
  • A way to catch mistakes before they hit production
  • Opportunity to build consensus about
  • Ensure your code doesn’t have a bus factor of 1

Mariatta encouraged a lot of work reviewers should do before sitting down to write feedback on a new code review.

  • Empathize wiht whoever you’re reviewing (e.g. are they new to Python? new to this codebase?)
  • Understand the context of the change
  • Give it the proper time it deserves. Good code review is not a fast process.
  • Don’t be afraid to point out where you see good code! Code review can be complimentary too

Remember that this is all important, and you should be explicitly explaining how to go about the process. Don’t fall into the Null Process Trap! Just like python prefers explicit over implicit, we should too 😊

Protecting your users with circuit breakers - Scott Triglia

(Video link)

Learn what the Circuit Breaker pattern is, and how you can customize the default setup for your own use cases.

I was a bit busy giving this one, so I have very few comments about it :)

But some interesting extra reading:

Live Coding in Python - Don Kirkby

Playing with the author’s LightTable-esque plugin for watching python code execute live.

Walkthrough of various small code snippets (Turtle, binary search) with support for introspecting the state of the systme along the way and having live-reruns while code changed.

Very hard to describe in text, but there are some fun demo videos at http://donkirkby.github.io/live-py-plugin/.

The high and low of a Python Data stack - Boris Lau

(Video link)

An exploration of the PyData ecosystem, with lots of discussion of individual tools.

High level overview of the various component parts of the PyData ecosystem.

Jupyter/IPython Notebooks

Interactive python code, executes real code. Reproducible!

Plotting - Bokeh

Visualization inside of Jupyter notebooks! Has some visual similarities with plot.ly

Modeling - Scikit-lrn

Has a ton of the tough ML models you love impelmented for you in a reliable/composable way.

Data access - Pandas/Numpy

Very efficient with multi-dimensional data. Uses a ton of BLAS under the hood. Represents your data on disk in a way that interops efficiently and

Low level optimization - Numba

Compiles python into LLVM…apparently comparable to Fortran’s performance.

Even got a little shout out to @jakevdp for this article on using Numba.

Apache data stack

Very focused on scaling up across a distributed cluster. Kind of the opposite of most python libraries which focus on single-process work.

Python in the NHL Front Office - Josh Weissbock

(Video link)

An extended case study of how you can use Python tools to analyze and predict hockey performance from past statistics.

Overview of how you can use quantitative evidence to answer questions about Hockey!

Tools mentioned:

  • BeautifulSoup for scraping out hockey data from webpages
  • Selenium+PhantomJS for scraping websites like a real user from the command line
  • Various pydata tools for analysis
  • sqlite3 for a simple sql-like database on disk

Deciding between Continuity and Change in Open Source - Christopher Neugebauer

(Video link)

A discussion of how to successfully encourage and execute on large changes within open source communities.

A story about trying to develop an open source software project to help manage conference organization.

How do we migrate an open source community through a complicated transition?

Cost-benefit analysis

In business:

  • What benefit does this change bring?
  • Is it going to make us more money than it costs?

In OSS:

  • How many people will this change bring joy to?
  • How long will it take?
  • How many people will this change burn out in the process?

Continuity

Sometimes it’s right to decide not to change things! This is the result of doing this cost-benefit tradeoff and deciding you should stay put.

Change disguised as continuity

Sometimes you can make a huge change that appears mostly invisible. This isn’t a bad thing! In fact it might be the best of both worlds.

Wholesale replacement

A pretty damaging form of change. This costs a lot, requires work from any of your users that want to change, and might not end up being worth all the effort.

This is the hardest part! Observations about successful wholesale change:

  • Avoid bikeshedding
  • Try to avoid losing major parts of your community via a fork. But keep in mind that attempting to force only one group may also be damaging.
  • Most of the time, you can’t effect wholesale change by fiat. Find consensus!
  • Involve people when there is something they can actively do!

Open Source and You - Josh Simmons

Tips for successfully building and growing your open source project or community.

Make excellent docs

Nobody can start working on your project if they don’t understand what it is or what purpose it serves!

Share your story and knowledge

Never forget that the things you struggled to learn might not be common knowledge. Go give a conference talk!

Expose your own uncertainty to show it’s legitimate and normal to not know everything!

There’s great opportunities out there across communities. Try joining a new one (or helping someone who’s new to yours).

Mentor new contributors

Everyone can use (and will greatly appreciate) a leg up when they’re new on the scene.

Encourage a safe space where the new contributors can get up to speed with someone who knows the ropes.

Use this as an experience to be exposed to someone who’s unlike yourself! As a bonus, you’re making the community more welcoming when you do this.

Inclusion solves your problems for more people

Consider aspects like:

  • accessibility
  • internationalization
  • availablility to non-technical contributors
Don’t tolerate jerks

Our community is only as awesome as the worst behavior we tolerate.

Its up to us all to maintain a healthy community and actively work to make it better.

Ship It: Python Packaging and Distribution - Nathaniel Knight

Intro to the basics of python packaging and the ecosystem of tools surrounding it.

Introduction to python packaging! Discussion of package installation, distribution, and related concerns.

Some additional resources:

Lightning talks

Up first was a Continuum.io employee. Lots of the normal cool tools, including a new one called Navigator that lets you visually explore your conda environments you’ve created. Solves a major pain point I’ve felt from using Conda in the past.

The next talk was about python education (and learning to code in general). Went through a simple “learning to code” exercise about writing instructions to drink a jug of water.

Jas Sohi combined Google’s BigQuery with python’s BeautifulSoup library to interact with web-uis and automatically check they had certain properties.

Talk #4 was a spur of the moment demo of trying to write a SublimeText plugin in the middle of the conference 😸 The speaker attempted to import Don Kirkby’s library (from a previous talk!) and actually got the basic proof-of-concept working! Not half bad on short notice.

Talk #5 was an intro to SQLAlchemy…in particular the Core part of the library. Some talk of the dangers of ORMs and some context on when you might prefer Core over the ORM.

Talk #6 was the story of one particular person’s experience learning Python from Team Treehouse. Sounded like some good teaching from Kenneth Love!

The final talk was on ontologies – dictionaries relating concepts together so we can translate concepts between two distinct systems. Some interesting relationships to work the CDC does with the semantic web. Brings me back to my SPARQL days 😲

Overall

A really great conference! Super friendly and a tremendously excited group of developers. I’d heartily encourage showing up for any future Vancouver meetups…it was a great trip and a very enjoyable conference.

Comments