A lot of changes have happened to bittyblog over the past couple months. It has finally got to a place that I'm happy with so there probably won't be any more updates for a while.

So what has changed? Here's a quick list:

• Tags: both posts and pages can have tags associated with them. Adding tags to pages will have that page show all posts containing that tag. A pretty handy feature for creating subpages on a blog for specific topics.
• fastCGI: everything supports fastCGI now for faster response times.
• RSS: adding 'rss' to the query string will return the page results in RSS instead of HTML. RSS isn't something that I really use but I think it's still pretty popular so it's a good feature to have on blogs.
• Caching: bittyblog now has a built-in cache that can be activated for extra fast response times. I got about a 5-fold increase in the number of processed requests when testing on my desktop.
• Misc: lots of other miscellaneous changes and refactors to improve the code and speed.

So with bittyblog in a good spot I've finally launched my new site: LinuxGameNetwork (logo at the top of this post). A blog focusing on all topics related to Linux, gaming, and Linux gaming. My plan is to keep up frequent updates for 6 months and see what how the readership changes; after then I will probably re-evaluate what my goals for the site should be.

In the meantime please check it out if you are interested in Linux gaming and subscribe to the RSS feed if you're into that.

Lately I've had a lot of motivation to upgrade bittyblog. It started as a simple CGI app to host my personal weblog, however, I have a larger blogging project in mind that I would also like to use bittyblog for. So to get it ready, I've made several nice improvements.

Template Support

Previously, all the HTML had been hard-coded into the bittyblog's C source files. I tried to abstract this away as much as possible but it just became too much. There is a very nice mustache implementation that somebody wrote in C and that I was able to import into my project. Now I can manage HTML completely separate from the C code which greatly speeds up development and layout changes since a recompile is no longer necessary for HTML changes.

Primitive CMS

Old bittyblog had no way to manage posts or images from the browser. Everything was done by manually uploading pictures and editing the database over ssh which was very time consuming and clunky. Now I can add new posts and images easily from the new bbadmin.cgi page.

Setup Script

For a project that aims for simplicity, setting up bittyblog was actually quite a hassle. In an effort to help this I made a small install script that sets up the database and fills in global variables automatically. This has been really useful when setting up bittyblog on different machines. I got used to having a bunch of hard-coded variables that I never had to touch when developing but were nightmare when redeploying.

Of course, I have to eat what I grow, so my personal blog is now running the new version of bittyblog. Even though it looks the same to you in the browser, behind the scenes there have been a lot of nice improvements that will feed nicely into my other blogging project. Next on the list is a round of code cleanup and then improving the CMS system to make it easy working with a large number of posts and media.

Check out the new updates on github.

There's one guaranteed way to a good start to the new year: new computer hardware! Previously I had been hosting this site on an aging Raspberry Pi 2 and it was due time for an upgrade.

As much as I wanted to stay in the Raspberry Pi ecosystem and move to a Pi 3, Asus' Tinker Board sounded too good to pass up. Its biggest advantage over the Pi is a dedicated Gigabit ethernet adapter, perfect for a webserver. The Raspberry Pi shares its ethernet with the USB circuitry which means slow Megabit speeds. It can also quickly become saturated when reading data from an external USB drive and using the network at the same time.

Moving to new hardware also surfaced some bugs in minihttp so it was nice to further improve the server code as well. So far everything has been running great and I'm really pleased with the speed.

And of course, it's fitted with a new LCD display and a speaker. Similarly to the old server, the LCD cycles between temperature, CPU usage, and website hits. The speaker is set to play a chime whenever somebody uploads an image to our family picture frame.

It’s been a while since my last machine learning project: implementing a decision tree in Julia. This time I wanted to take a closer look at neural networks. I was recently shown an amazing book 'Neural Networks and Deep Learning' by Michael Nielson. He does a great job distilling the basics to a point where his explanations become intuitive. I won't be able to explain anything as well as he does so please check out his book.

The most basic neural networks are, as it turns out, surprisingly simple. It is possible to derive methods for building and training neural networks using only basic linear algebra and calculus. Neural networks have also been around for quite some time but it wasn’t until backpropagation was suggested as a way of training networks in the 70's that they really took off. The complexity of them stems somewhat from the sheer size of networks. Modern computer hardware and new scientific computing methods were required for neural networks to reach the popularity they have today.

Backpropagation is the key to training neural networks. Essentially, backpropagation takes the error at the output of a network and updates weights, within the network, based on how much they contributed to that error. By calculating the error from a sample and adjusting the weights accordingly over many, many iterations the network can be trained.

So in keeping with my previous project, I implemented a basic backpropogation algorithm in C for training on the popular MNIST dataset. I used a combination of the GNU Scientific Library and OpenBLAS for all the heavy number crunching. For the network itself I went with 2 hidden layers (4 total, including input and output layers) of 100 and 30 neurons. Below is the result after training on 50,000 images:

The green shows accuracy on training data and the blue shows the neural networks accuracy on a separate set of testing data. The x-axis shows the number of epochs, or the number of times the backpropagation went through all the training data and updated the network. Interestingly, after about 100 epochs the accuracy on the test data starts to decrease slightly. This is a sign that the network was overfitting to the training data. However, after around 180 epochs there is some disruption which ended up increasing the accuracy on both the training and testing data sets. Overall the accuracy was 99.74% and 97.14% on the training and testing data respectively.

As a final test, I got my lovely wife to draw any number on the computer (she chose '4'). I then fed this into the neural network to get see if it could identify what she wrote:

Clearly there is something to these neural networks after-all.

Thank you for reading. Please check out Michael's book if you want to know more about neural networks. Also check out the code I wrote for this network on my Github.