andre's blog

Binary self contained logs for modern high throughput systems

The number of large software systems in the world is growing steadily. Old first generation solutions are improved and made more sophisticated. Completely new areas open up for automation. More systems, more line of code. More complexity everywhere. And, inevitably, more questions about their behaviour. And, of course, more errors, from bugs in the systems themselves to malfunctioning of external dependencies and systems.

Bokeh graphs and pandas dataframe groupby object

Bokeh is a nice library, helping python web developers to visualise your data in the browser. It is on good terms with pandas, the statistical and data manipulation package beloved by data scientists. It can source points from a dataframe object directly. Unfortunately, it can't get a result of a group by object directly to display it as multiple lines, yet. But no worries, with just a few lines of code you can convince it to draw you a nice multiline graph. Take a look at the code snippet below:


Visual Studio 2017 and CMake

Microsoft is continuously improving its record with the open source community. First dumping massive chunks of .NET into github, then actually making very dedicated effort to clean it up and make it portable to Linux and Mac. Now comes another step, albeit smaller, in the same direction. Visual Studio 2017 will support CMake projects in a native way, without the need to generate .proj and .sln files first. This is great news, because it saves some effort for those working on cross-platform C++ products.

Cryptocurrency algo trading

The Internet is abuzz with the news about various crypto-currencies. From the old favourite BitCoin to the bold newcomers, such as Etherium and Ripple (not that new actually), lots of frantic trading and even more frenzied discussion is taking place. The new currencies are viewed as the end of era of the old money dominance over the finances of the world.

Zap all annoyances: images, colours, plugins, iframes etc

Many web sites' developers seem to think "the more the better" and clutter their pages with endless images, flashy colours, countless videos and flash animations. This is hugely distracting. For years I have been using a couple of buttons to zap these annoyances from this helpful page. And just now I realised I can combine a few of my favourite ones into one button. It removes images, hides iframes and plugins and makes the colour scheme black and white.

Heroku and django-bower

Heroku, a cloud solution for various application engines, supports django out of the box. It does a pretty good job of providing you with a deployment process which at one hand is relatively simple, and on the other hand allows them to control their resources.

Faster CSV parsing in Python with pandas

Recently I had to write a Python script which needed to parse large gzip-ed CSV files. First I reached for the standard csv module, which is quite straightforward to use. Unfortunately, it proved to be too slow. In fact, I gave up waiting for it to parse even a single file containing about 30 million rows! And my whole data set had more than 300 million rows altogether.