Lately I've had a lot of motivation to upgrade bittyblog. It started as a simple CGI app to host my personal weblog, however, I have a larger blogging project in mind that I would also like to use bittyblog for. So to get it ready, I've made several nice improvements.

Template Support

Previously, all the HTML had been hard-coded into the bittyblog's C source files. I tried to abstract this away as much as possible but it just became too much. There is a very nice mustache implementation that somebody wrote in C and that I was able to import into my project. Now I can manage HTML completely separate from the C code which greatly speeds up development and layout changes since a recompile is no longer necessary for HTML changes.

Primitive CMS

Old bittyblog had no way to manage posts or images from the browser. Everything was done by manually uploading pictures and editing the database over ssh which was very time consuming and clunky. Now I can add new posts and images easily from the new bbadmin.cgi page.

Setup Script

For a project that aims for simplicity, setting up bittyblog was actually quite a hassle. In an effort to help this I made a small install script that sets up the database and fills in global variables automatically. This has been really useful when setting up bittyblog on different machines. I got used to having a bunch of hard-coded variables that I never had to touch when developing but were nightmare when redeploying.

Of course, I have to eat what I grow, so my personal blog is now running the new version of bittyblog. Even though it looks the same to you in the browser, behind the scenes there have been a lot of nice improvements that will feed nicely into my other blogging project. Next on the list is a round of code cleanup and then improving the CMS system to make it easy working with a large number of posts and media.

Check out the new updates on github.

Please note: this article contains spoilers of the contents. Don't read this review if you plan on buying this product.

Living in Germany, there are a lot of choices when it comes to Advent calendars. Of course, there are lots of styles of chocolate calendars, but also calendars with beer, tea, Lego, and even a 'couples' calendar from the local drug store. One of the more interesting themes was a Raspberry Pi Advent calendar from Conrad.

Conrad is a chain of electronic stores in Germany that sell a lot of classic electronic bits. All the small things you typically associate with electronics: resistors, buttons, etc. They also have a decent selection of Raspberry Pi products, one of them being an Advent calendar. Naturally I couldn't resist and decided to get one for the holidays.

So what do you get for your 29EUR? Well, there isn't much to say about the presentation of the box itself. It looks like a typical calendar with 24 doors, one for each day in December leading up to Christmas. They're all well separated into small compartments so you won't see anything from other doors while opening each one. You also get a large cardboard nativity-scene cutout so you can probably guess what final project will look like. It's also worth noting that it doesn't contain the Raspberry Pi itself so you will need to supply your own Pi as well as a keyboard, mouse, and screen.

Each little door contains a few electronic bits that you will use the day's project. The variety in parts is pretty limited and most days there won't be anything except cables. I think the last 4-5 days opening doors was nothing but cables which was really disappointing. Even the larger doors that allude to something more interesting just contain cables. This was probably the biggest disappointment of the calendar. That said, there were some slightly more interesting pieces like a couple of tri-color LEDs that get used for some neat effects.

The programming is all done using a visual language called Scratch. You can build simple programs using basic visual blocks that represent things like loops and if-else statements. For basic things it works alright but programs can quickly grow very large and they become difficult to work with. By the end of the calendar the programs were so large that my wife and I stopped doing everything ourselves and just used the pre-made ones.

The included instructions are only in German but several other languages are available online. Beware, the English instructions are badly Google translated. My wife and I had to refer to the German instructions several times to clear things up. Annoyingly, all versions of the instructions contained some very obvious mistakes. On one day the project diagram showed all wires going to ground. If this is somebody's first experience with electronics then stupid mistakes like this can be really frustrating.

I would stay far and clear from this calendar in the future. There are just too many negatives between the instructions contained mistakes, the programming language being difficult to work with, and the disappointing amount of components.

Lately, there’s been a lot of interest in Bitcoin, probably sparked by its almost unbelievable growth in December 2017. However, this past week, we saw the price of Bitcoin drop the just above $6000 which was the lowest it has been since November 2017. So I wanted to take a closer look at Bitcoin prices through the lens of Hidden Markov Models (HMM) to see what conclusions, if any, can be drawn.

Hidden Markov Models are similar to a standard Markov chain model but the where the current state is unknown. Instead of observing the actual state of the process, the only information available is the realization of some other output that is dependent on the current internal state. A somewhat contrived example would be trying to detect whether it is raining, or not, based on how many people you see with umbrellas. The hidden, unobservable state is the weather (raining or not) while the observable, realization of that state is the proportion of people carrying umbrellas (more people carry umbrellas if it’s raining).

Applying this concept to Bitcoin prices, there could be some internal state driving the change in price and different states produce different expected price changes. I assumed that the daily change in price followed a Log-normal distribution, which means that taking the logged value of daily returns should be normally distributed. This made the model slightly easier to interpret. I also used 3 internal states in an attempt to capture bear and bull states with differing volatility.

Below is a chart showing the most likely states during the 2017 and into the 2018 calendar years:

Here each of the three states are coloured. The blue state was characterized by positive average returns and low volatility. The red state also had generally positive returns but higher volatility. Finally, the green state had mostly negative returns and also high volatility.

I also ran a quick Shapiro-Wilk test on the log-valued daily returns which was unable to reject the null that daily returns come from a normal distribution. This means that there wasn’t enough evidence to disprove the assumption that price changes follow a Log-normal distribution.

This is all fine and good, but what would be really cool is if the fitted model could be used to predict the future price of Bitcoin. So I ran 10,000 30-day simulations to get an expected future price and a confidence interval. This is what it looks like:

This shows the predicted Bitcoin price, and the actual price change during the prediction interval. The shaded regions also represent the 95% and 80% confidence intervals, based on the 10,000 simulations. In this instance, the HMM was not exactly a great predictor. Bitcoin has been incredibly volatile and I think it’s extremely difficult to make any meaningful predictions using closing price alone.

If you’re interested in taking a closer look at the R code used to fit the HMM model and generate the charts, you can find it on my Github.

There's one guaranteed way to a good start to the new year: new computer hardware! Previously I had been hosting this site on an aging Raspberry Pi 2 and it was due time for an upgrade.

As much as I wanted to stay in the Raspberry Pi ecosystem and move to a Pi 3, Asus' Tinker Board sounded too good to pass up. Its biggest advantage over the Pi is a dedicated Gigabit ethernet adapter, perfect for a webserver. The Raspberry Pi shares its ethernet with the USB circuitry which means slow Megabit speeds. It can also quickly become saturated when reading data from an external USB drive and using the network at the same time.

Moving to new hardware also surfaced some bugs in minihttp so it was nice to further improve the server code as well. So far everything has been running great and I'm really pleased with the speed.

And of course, it's fitted with a new LCD display and a speaker. Similarly to the old server, the LCD cycles between temperature, CPU usage, and website hits. The speaker is set to play a chime whenever somebody uploads an image to our family picture frame.

It’s been a while since my last machine learning project: implementing a decision tree in Julia. This time I wanted to take a closer look at neural networks. I was recently shown an amazing book 'Neural Networks and Deep Learning' by Michael Nielson. He does a great job distilling the basics to a point where his explanations become intuitive. I won't be able to explain anything as well as he does so please check out his book.

The most basic neural networks are, as it turns out, surprisingly simple. It is possible to derive methods for building and training neural networks using only basic linear algebra and calculus. Neural networks have also been around for quite some time but it wasn’t until backpropagation was suggested as a way of training networks in the 70's that they really took off. The complexity of them stems somewhat from the sheer size of networks. Modern computer hardware and new scientific computing methods were required for neural networks to reach the popularity they have today.

Backpropagation is the key to training neural networks. Essentially, backpropagation takes the error at the output of a network and updates weights, within the network, based on how much they contributed to that error. By calculating the error from a sample and adjusting the weights accordingly over many, many iterations the network can be trained.

So in keeping with my previous project, I implemented a basic backpropogation algorithm in C for training on the popular MNIST dataset. I used a combination of the GNU Scientific Library and OpenBLAS for all the heavy number crunching. For the network itself I went with 2 hidden layers (4 total, including input and output layers) of 100 and 30 neurons. Below is the result after training on 50,000 images:

The green shows accuracy on training data and the blue shows the neural networks accuracy on a separate set of testing data. The x-axis shows the number of epochs, or the number of times the backpropagation went through all the training data and updated the network. Interestingly, after about 100 epochs the accuracy on the test data starts to decrease slightly. This is a sign that the network was overfitting to the training data. However, after around 180 epochs there is some disruption which ended up increasing the accuracy on both the training and testing data sets. Overall the accuracy was 99.74% and 97.14% on the training and testing data respectively.

As a final test, I got my lovely wife to draw any number on the computer (she chose '4'). I then fed this into the neural network to get see if it could identify what she wrote:

Clearly there is something to these neural networks after-all.

Thank you for reading. Please check out Michael's book if you want to know more about neural networks. Also check out the code I wrote for this network on my Github.