Data Science?

So I've flopped around for a while, looking for a reasonable way to make a better view of the data gathered by weatherPi. The fist hard part is to decide what a better view might be. The current default display just crams in however many samples will fit in your web view, by dividing the width of the graph view by 5. This moves the points on the graph far enough apart to be able to see them, but even on a 4K display that gives you less than 4 hours visibility, which is likely not what you want.

I'm thinking a 'day' view and a 'week' view are much better guesses. Later I'll probably add a 'month' and 'year' view, because the tools should be nearly identical.

So I went looking for ways to 'downsample' the data. Python has a lot of numerical tools available, but the one I chose is called Pandas. I chose Pandas for several reasons, not the least of which it was the first one I could figure out how to do something useful in under 10 minutes. In actual fact, I found a simple tutorial on how to load your data into a Pandas data frame and how to resample a data frame on the second try. I found another tutorial on how to select data from a database directly into Pandas with a single search. Both of those examples worked. This may be because Pandas is consistent across versions, or because much of the documentation is new, but I often spend days looking for documentation that applies to the current version of something in my day job, and I don't have the patience for it when it comes to weatherPi.

So the new call in wxWeb is '/day'. The code is amazingly simple and I've commented each of the very few steps.

I haven't written a UI page to graph the data yet. I'll get to that, this weekend if I'm lucky, but I wanted to blog about the update to the server code and about Pandas while it's fresh in my mind. Once I get a graph done, it should be trivial to add a day, month, and year view. Then I'll be ready for this fall, when I actually have a years worth data.

And, I'll note, I got all this done with only 8 minutes downtime on the server. Not quite 5-9's level of service, but not bad right?